2024-04-22T13:08:36.779 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T13:08:36.779 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269 -- /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/orig.config.yaml 2024-04-22T13:08:36.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T13:08:36.930 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7668269' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4362 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 222/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUhPUtnAnuVmoEZ3iRJtrFKsflGVYPhvhnkfe2PB7Ag0xN5ICanyKozKdHGN2yAT3F9VYYvLeHxA8M94nL2Y5A= smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGaAVSbSXYPhZN1cfSzRXQc7yy6XyXtm1BuakNP+bVZOJ6uJGIsxEPX9417hPzBNf+AVGhfpto1/hw4QvLXbOgU= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjh7pHruRpGoGJ4NKY/f1AabTtQWDJfNFkk1L/+8zNZAi4PzIccBgAhR5UDOVFbL7S9RGP5lfqQh05Ls53SfVk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_12:36:42 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T13:08:36.930 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa; will attempt to use it 2024-04-22T13:08:36.931 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks 2024-04-22T13:08:36.931 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T13:08:36.933 INFO:teuthology.task.internal:Checking packages... 2024-04-22T13:08:36.955 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-22T13:08:36.955 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T13:08:36.955 INFO:teuthology.packaging:ref: None 2024-04-22T13:08:36.956 INFO:teuthology.packaging:tag: None 2024-04-22T13:08:36.956 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-22T13:08:36.956 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:08:36.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-22T13:08:37.294 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-22T13:08:37.295 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T13:08:37.302 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T13:08:37.302 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T13:08:37.308 INFO:teuthology.task.internal:Saving configuration 2024-04-22T13:08:37.330 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T13:08:37.336 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T13:08:37.360 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 13:01:57.419382', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUhPUtnAnuVmoEZ3iRJtrFKsflGVYPhvhnkfe2PB7Ag0xN5ICanyKozKdHGN2yAT3F9VYYvLeHxA8M94nL2Y5A='} 2024-04-22T13:08:37.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 13:01:57.416484', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGaAVSbSXYPhZN1cfSzRXQc7yy6XyXtm1BuakNP+bVZOJ6uJGIsxEPX9417hPzBNf+AVGhfpto1/hw4QvLXbOgU='} 2024-04-22T13:08:37.402 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 13:01:57.414790', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjh7pHruRpGoGJ4NKY/f1AabTtQWDJfNFkk1L/+8zNZAi4PzIccBgAhR5UDOVFbL7S9RGP5lfqQh05Ls53SfVk='} 2024-04-22T13:08:37.402 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T13:08:37.408 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-22T13:08:37.408 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-22T13:08:37.408 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-22T13:08:37.408 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T13:08:37.506 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1a058b70d0>, signals=[15]) 2024-04-22T13:08:37.506 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T13:08:37.513 INFO:teuthology.task.internal:Opening connections... 2024-04-22T13:08:37.513 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:08:37.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:08:37.594 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-04-22T13:08:37.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:08:37.666 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T13:08:37.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:08:37.742 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T13:08:37.748 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-04-22T13:08:37.767 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-04-22T13:08:37.767 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-04-22T13:08:37.825 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T13:08:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T13:08:37.826 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-04-22T13:08:37.847 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-04-22T13:08:37.865 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-04-22T13:08:37.866 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-04-22T13:08:37.923 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T13:08:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T13:08:37.925 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-04-22T13:08:37.947 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-04-22T13:08:37.964 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-04-22T13:08:37.964 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-04-22T13:08:38.021 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T13:08:38.022 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-04-22T13:08:38.022 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T13:08:38.022 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T13:08:38.022 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T13:08:38.022 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-04-22T13:08:38.038 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T13:08:38.048 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T13:08:38.053 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T13:08:38.054 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-04-22T13:08:38.056 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-04-22T13:08:38.058 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-04-22T13:08:38.076 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T13:08:38.082 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T13:08:38.082 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-04-22T13:08:38.114 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-04-22T13:08:38.116 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-04-22T13:08:38.144 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T13:08:38.206 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T13:08:38.220 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-22T13:08:38.220 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-22T13:08:38.220 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-22T13:08:38.220 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T13:08:38.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:08:38.238 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-04-22T13:08:38.296 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-04-22T13:08:38.297 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-437.el9.x86_64 2024-04-22T13:08:38.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-04-22T13:08:43.410 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-04-22T13:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 19 MB 00:07 2024-04-22T13:08:57.697 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-22T13:09:12.976 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 1.1 kB/s | 16 kB 00:14 2024-04-22T13:09:14.772 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-22T13:09:19.903 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-22T13:09:20.092 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:01 ago on Mon 22 Apr 2024 01:09:19 PM UTC. 2024-04-22T13:09:21.834 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T13:09:21.834 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T13:09:21.925 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-22T13:09:21.926 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-04-22T13:09:21.926 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-22T13:09:22.034 DEBUG:teuthology.orchestra.run.smithi005:> echo no | sudo yum reinstall kernel || true 2024-04-22T13:09:22.555 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 01:09:19 PM UTC. 2024-04-22T13:09:22.679 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:Reinstalling: 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:09:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 255 k 2024-04-22T13:09:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 0 2024-04-22T13:09:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:Operation aborted. 2024-04-22T13:09:22.763 INFO:teuthology.orchestra.run.smithi005.stdout:Is this ok [y/N]: 2024-04-22T13:09:22.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum reinstall -y kernel || true 2024-04-22T13:09:23.276 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 01:09:19 PM UTC. 2024-04-22T13:09:23.396 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:Reinstalling: 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 255 k 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 0 2024-04-22T13:09:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-04-22T13:09:24.022 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-437.el9.x86_64.rpm 732 kB/s | 255 kB 00:00 2024-04-22T13:09:24.022 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:09:24.022 INFO:teuthology.orchestra.run.smithi005.stdout:Total 408 kB/s | 255 kB 00:00 2024-04-22T13:09:24.045 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-04-22T13:09:24.259 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-04-22T13:09:24.259 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-04-22T13:09:24.264 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-04-22T13:09:24.265 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-04-22T13:09:24.395 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-04-22T13:09:24.489 INFO:teuthology.orchestra.run.smithi005.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T13:09:24.489 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T13:09:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T13:09:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T13:09:25.792 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T13:09:25.792 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:09:25.792 INFO:teuthology.orchestra.run.smithi005.stdout:Reinstalled: 2024-04-22T13:09:25.792 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T13:09:25.792 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:09:25.792 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-22T13:09:25.967 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T13:09:26.016 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T13:09:26.016 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T13:09:26.016 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T13:09:26.016 DEBUG:teuthology.parallel:result is None 2024-04-22T13:09:26.017 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T13:09:26.024 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T13:09:26.024 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T13:09:26.026 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T13:09:26.029 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T13:09:26.046 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T13:09:26.052 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T13:09:26.063 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T13:09:26.063 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T13:09:26.081 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T13:09:26.087 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T13:09:26.320 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T13:09:26.327 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T13:09:26.327 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T13:09:26.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:09:26.344 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T13:09:26.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:09:26.361 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T13:09:26.364 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T13:09:26.388 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T13:09:26.397 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:09:26.417 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:09:26.418 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:09:26.442 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:09:26.448 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:09:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:09:26.459 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T13:09:26.466 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T13:09:26.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T13:09:26.468 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T13:09:26.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T13:09:26.525 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T13:09:26.533 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T13:09:26.533 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T13:09:26.536 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T13:09:26.561 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T13:09:26.581 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:09:26.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:09:26.688 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:09:26.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:09:26.808 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:09:26.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T13:09:26.872 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:09:26.918 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:09:26.970 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:09:27.052 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:09:27.098 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:09:27.098 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T13:09:27.165 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:09:27.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:09:27.268 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:09:27.348 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:09:27.393 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:09:27.393 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T13:09:27.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-04-22T13:09:27.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-04-22T13:09:27.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-04-22T13:09:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:09:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:09:27.536 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:09:27.849 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T13:09:27.857 INFO:teuthology.task.internal:Starting timer... 2024-04-22T13:09:27.857 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T13:09:27.868 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T13:09:27.876 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-22T13:09:27.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-04-22T13:09:27.934 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-04-22T13:09:27.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-04-22T13:09:27.993 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-04-22T13:09:27.996 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-04-22T13:09:28.073 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-04-22T13:09:28.075 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T13:09:28.075 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-04-22T13:09:28.094 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-04-22T13:09:28.094 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-04-22T13:09:28.130 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-04-22T13:09:28.130 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-04-22T13:09:28.152 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-04-22T13:09:28.152 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2024-04-22T13:09:28.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:09:28.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:09:28.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:09:28.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:09:28.207 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:09:28.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:09:28.235 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T13:09:28.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-04-22T13:09:28.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-04-22T13:09:28.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-04-22T13:09:28.315 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T13:09:28.324 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-22T13:09:28.330 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-22T13:09:28.455 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T13:09:28.470 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T13:09:28.472 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi049.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T13:13:45.244 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2024-04-22T13:13:45.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-22T13:13:45.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:13:45.331 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-04-22T13:13:45.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-22T13:13:45.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-22T13:13:45.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:13:45.491 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-04-22T13:13:45.577 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-22T13:13:45.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-22T13:13:45.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:13:45.654 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-04-22T13:13:45.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-22T13:13:45.731 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T13:13:45.745 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T13:13:45.745 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T13:13:45.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:13:45.748 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T13:13:45.749 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:13:45.754 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T13:13:45.755 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:13:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T13:13:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T13:13:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T13:13:45.798 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T13:13:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T13:13:45.814 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T13:13:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-04-22T13:13:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-04-22T13:13:45.826 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-04-22T13:13:45.835 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-04-22T13:13:45.836 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-04-22T13:13:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T13:13:45.847 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-04-22T13:13:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T13:13:45.854 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T13:13:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T13:13:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T13:13:45.873 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T13:13:45.913 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:13:45.917 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:13:45.917 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-04-22T13:13:45.917 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.917 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.917 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.917 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.921 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:13:45.924 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:13:45.924 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-04-22T13:13:45.925 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.925 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.925 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.925 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.925 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:13:45.929 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:13:45.929 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-22T13:13:45.929 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.929 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.929 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.929 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:13:45.930 INFO:teuthology.run_tasks:Running task install... 2024-04-22T13:13:45.939 DEBUG:teuthology.task.install:project ceph 2024-04-22T13:13:45.940 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-22T13:13:45.940 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-22T13:13:45.940 INFO:teuthology.task.install:Using flavor: default 2024-04-22T13:13:45.946 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T13:13:45.947 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-22T13:13:45.947 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-22T13:13:45.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:13:45.950 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-22T13:13:45.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:13:45.952 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-22T13:13:45.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:13:46.104 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-22T13:13:46.104 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-22T13:13:46.108 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-22T13:13:46.108 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-22T13:13:46.235 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-22T13:13:46.235 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-22T13:13:46.316 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T13:13:46.316 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:13:46.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T13:13:46.323 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T13:13:46.323 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:13:46.323 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T13:13:46.342 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T13:13:46.342 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T13:13:46.345 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T13:13:46.345 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:13:46.345 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T13:13:46.349 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T13:13:46.349 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T13:13:46.371 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T13:13:46.371 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T13:13:46.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T13:13:46.415 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T13:13:46.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T13:13:46.483 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T13:13:46.490 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T13:13:46.508 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T13:13:46.546 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-04-22T13:13:46.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-04-22T13:13:46.554 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-04-22T13:13:46.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-04-22T13:13:46.572 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-04-22T13:13:46.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-04-22T13:13:46.781 INFO:teuthology.orchestra.run.smithi005.stdout:48 files removed 2024-04-22T13:13:46.788 INFO:teuthology.orchestra.run.smithi049.stdout:48 files removed 2024-04-22T13:13:46.798 INFO:teuthology.orchestra.run.smithi107.stdout:48 files removed 2024-04-22T13:13:46.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T13:13:46.815 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T13:13:46.825 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T13:13:47.561 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-22T13:13:47.693 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 138 kB/s | 83 kB 00:00 2024-04-22T13:13:47.725 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 130 kB/s | 83 kB 00:00 2024-04-22T13:13:48.086 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-22T13:13:48.118 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-22T13:13:48.159 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-22T13:13:48.535 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-22T13:13:48.584 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-22T13:13:48.637 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-04-22T13:13:49.286 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-22T13:13:51.112 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-04-22T13:13:52.103 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-22T13:13:54.212 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 19 MB 00:02 2024-04-22T13:13:58.729 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-22T13:14:05.245 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 842 kB/s | 6.2 MB 00:07 2024-04-22T13:14:06.654 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 2.4 kB/s | 16 kB 00:06 2024-04-22T13:14:06.804 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-22T13:14:07.515 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 436 kB/s | 8.0 MB 00:18 2024-04-22T13:14:07.764 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-22T13:14:09.124 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 21 MB 00:02 2024-04-22T13:14:10.110 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-22T13:14:12.763 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-22T13:14:14.330 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-22T13:14:14.872 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:14:14.873 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:14:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T13:14:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T13:14:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T13:14:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T13:14:14.888 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T13:14:14.890 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T13:14:14.891 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T13:14:14.950 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-22T13:14:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-22T13:14:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T13:14:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-22T13:14:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-22T13:14:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T13:14:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T13:14:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T13:14:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T13:14:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T13:14:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T13:14:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T13:14:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T13:14:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T13:14:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T13:14:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T13:14:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T13:14:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T13:14:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T13:14:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T13:14:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-04-22T13:14:14.976 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-04-22T13:14:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:14:14.978 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 218 M 2024-04-22T13:14:14.978 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T13:14:16.491 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:14:16.492 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:14:16.503 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T13:14:16.504 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T13:14:16.505 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T13:14:16.506 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T13:14:16.508 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T13:14:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.2 MB 00:02 2024-04-22T13:14:16.510 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T13:14:16.511 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T13:14:16.576 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-22T13:14:16.599 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-22T13:14:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-22T13:14:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-22T13:14:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-22T13:14:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T13:14:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T13:14:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T13:14:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T13:14:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T13:14:16.608 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T13:14:16.609 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T13:14:16.610 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:Install 148 Packages 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 2 Packages 2024-04-22T13:14:16.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:14:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 218 M 2024-04-22T13:14:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-22T13:14:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-22T13:14:20.830 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 21 MB 00:02 2024-04-22T13:14:22.072 INFO:teuthology.orchestra.run.smithi107.stdout:(1/150): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-22T13:14:23.105 INFO:teuthology.orchestra.run.smithi107.stdout:(2/150): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 862 kB/s | 890 kB 00:01 2024-04-22T13:14:23.255 INFO:teuthology.orchestra.run.smithi107.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 965 kB/s | 145 kB 00:00 2024-04-22T13:14:24.240 INFO:teuthology.orchestra.run.smithi049.stdout:(1/150): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-22T13:14:24.682 INFO:teuthology.orchestra.run.smithi049.stdout:(2/150): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 890 kB 00:00 2024-04-22T13:14:24.748 INFO:teuthology.orchestra.run.smithi049.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-22T13:14:25.190 INFO:teuthology.orchestra.run.smithi049.stdout:(4/150): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 5.3 MB/s | 2.4 MB 00:00 2024-04-22T13:14:25.316 INFO:teuthology.orchestra.run.smithi049.stdout:(5/150): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.3 MB/s | 5.4 MB 00:01 2024-04-22T13:14:25.441 INFO:teuthology.orchestra.run.smithi049.stdout:(6/150): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 6.0 MB/s | 1.5 MB 00:00 2024-04-22T13:14:25.680 INFO:teuthology.orchestra.run.smithi107.stdout:(4/150): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 997 kB/s | 2.4 MB 00:02 2024-04-22T13:14:25.750 INFO:teuthology.orchestra.run.smithi049.stdout:(7/150): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 4.7 MB 00:00 2024-04-22T13:14:25.914 INFO:teuthology.orchestra.run.smithi107.stdout:(5/150): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-22T13:14:26.143 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-22T13:14:26.674 INFO:teuthology.orchestra.run.smithi049.stdout:(8/150): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 11 MB/s | 11 MB 00:00 2024-04-22T13:14:26.741 INFO:teuthology.orchestra.run.smithi049.stdout:(9/150): ceph-selinux-19.0.0-3156.gd69f13d8.el7 378 kB/s | 25 kB 00:00 2024-04-22T13:14:26.925 INFO:teuthology.orchestra.run.smithi049.stdout:(10/150): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 11 MB/s | 17 MB 00:01 2024-04-22T13:14:26.984 INFO:teuthology.orchestra.run.smithi049.stdout:(11/150): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 225 kB 00:00 2024-04-22T13:14:27.068 INFO:teuthology.orchestra.run.smithi049.stdout:(12/150): cephfs-shell-19.0.0-3156.gd69f13d8.el 325 kB/s | 27 kB 00:00 2024-04-22T13:14:27.156 INFO:teuthology.orchestra.run.smithi049.stdout:(13/150): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-22T13:14:27.247 INFO:teuthology.orchestra.run.smithi107.stdout:(6/150): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 977 kB/s | 1.5 MB 00:01 2024-04-22T13:14:27.754 INFO:teuthology.orchestra.run.smithi049.stdout:(14/150): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.2 MB/s | 747 kB 00:00 2024-04-22T13:14:27.835 INFO:teuthology.orchestra.run.smithi049.stdout:(15/150): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 163 kB 00:00 2024-04-22T13:14:27.911 INFO:teuthology.orchestra.run.smithi049.stdout:(16/150): librados-devel-19.0.0-3156.gd69f13d8. 1.6 MB/s | 126 kB 00:00 2024-04-22T13:14:28.011 INFO:teuthology.orchestra.run.smithi049.stdout:(17/150): libradosstriper1-19.0.0-3156.gd69f13d 4.9 MB/s | 500 kB 00:00 2024-04-22T13:14:28.413 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:14:28.413 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:14:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T13:14:28.426 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T13:14:28.427 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T13:14:28.428 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T13:14:28.429 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T13:14:28.431 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T13:14:28.432 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T13:14:28.498 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-22T13:14:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-22T13:14:28.509 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-22T13:14:28.510 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-22T13:14:28.511 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-22T13:14:28.512 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T13:14:28.513 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T13:14:28.514 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T13:14:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T13:14:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T13:14:28.517 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T13:14:28.518 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 2 Packages 2024-04-22T13:14:28.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:14:28.520 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 218 M 2024-04-22T13:14:28.520 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-04-22T13:14:28.803 INFO:teuthology.orchestra.run.smithi049.stdout:(18/150): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.8 MB/s | 5.4 MB 00:00 2024-04-22T13:14:28.878 INFO:teuthology.orchestra.run.smithi049.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.gd6 605 kB/s | 45 kB 00:00 2024-04-22T13:14:28.970 INFO:teuthology.orchestra.run.smithi049.stdout:(20/150): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-22T13:14:29.053 INFO:teuthology.orchestra.run.smithi049.stdout:(21/150): python3-cephfs-19.0.0-3156.gd69f13d8. 1.9 MB/s | 161 kB 00:00 2024-04-22T13:14:29.145 INFO:teuthology.orchestra.run.smithi049.stdout:(22/150): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-22T13:14:29.237 INFO:teuthology.orchestra.run.smithi049.stdout:(23/150): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-22T13:14:29.397 INFO:teuthology.orchestra.run.smithi107.stdout:(7/150): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 4.7 MB 00:03 2024-04-22T13:14:29.607 INFO:teuthology.orchestra.run.smithi005.stdout:(1/150): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-22T13:14:29.679 INFO:teuthology.orchestra.run.smithi049.stdout:(24/150): ceph-test-19.0.0-3156.gd69f13d8.el7.x 17 MB/s | 49 MB 00:02 2024-04-22T13:14:29.704 INFO:teuthology.orchestra.run.smithi049.stdout:(25/150): python3-rgw-19.0.0-3156.gd69f13d8.el7 213 kB/s | 99 kB 00:00 2024-04-22T13:14:29.721 INFO:teuthology.orchestra.run.smithi049.stdout:(26/150): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 85 kB 00:00 2024-04-22T13:14:29.772 INFO:teuthology.orchestra.run.smithi049.stdout:(27/150): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 3.3 MB/s | 170 kB 00:00 2024-04-22T13:14:29.830 INFO:teuthology.orchestra.run.smithi049.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 587 kB/s | 34 kB 00:00 2024-04-22T13:14:29.897 INFO:teuthology.orchestra.run.smithi049.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.1 MB/s | 142 kB 00:00 2024-04-22T13:14:29.972 INFO:teuthology.orchestra.run.smithi049.stdout:(30/150): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 12 MB/s | 3.1 MB 00:00 2024-04-22T13:14:30.189 INFO:teuthology.orchestra.run.smithi049.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.gd69f1 12 MB/s | 3.5 MB 00:00 2024-04-22T13:14:30.248 INFO:teuthology.orchestra.run.smithi049.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.gd6 4.2 MB/s | 249 kB 00:00 2024-04-22T13:14:30.298 INFO:teuthology.orchestra.run.smithi049.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 988 kB/s | 49 kB 00:00 2024-04-22T13:14:30.349 INFO:teuthology.orchestra.run.smithi049.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3156.gd 330 kB/s | 16 kB 00:00 2024-04-22T13:14:30.407 INFO:teuthology.orchestra.run.smithi049.stdout:(35/150): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.6 MB/s | 273 kB 00:00 2024-04-22T13:14:30.491 INFO:teuthology.orchestra.run.smithi049.stdout:(36/150): cephadm-19.0.0-3156.gd69f13d8.el7.noa 8.9 MB/s | 762 kB 00:00 2024-04-22T13:14:30.533 INFO:teuthology.orchestra.run.smithi049.stdout:(37/150): cephfs-top-19.0.0-3156.gd69f13d8.el7. 440 kB/s | 18 kB 00:00 2024-04-22T13:14:30.675 INFO:teuthology.orchestra.run.smithi049.stdout:(38/150): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-22T13:14:30.736 INFO:teuthology.orchestra.run.smithi005.stdout:(2/150): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 785 kB/s | 890 kB 00:01 2024-04-22T13:14:30.859 INFO:teuthology.orchestra.run.smithi049.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 247 kB/s | 80 kB 00:00 2024-04-22T13:14:30.878 INFO:teuthology.orchestra.run.smithi005.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.0 MB/s | 145 kB 00:00 2024-04-22T13:14:30.984 INFO:teuthology.orchestra.run.smithi049.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-22T13:14:31.109 INFO:teuthology.orchestra.run.smithi049.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-22T13:14:31.251 INFO:teuthology.orchestra.run.smithi049.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-22T13:14:31.343 INFO:teuthology.orchestra.run.smithi049.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-22T13:14:31.427 INFO:teuthology.orchestra.run.smithi049.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-22T13:14:31.510 INFO:teuthology.orchestra.run.smithi049.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-22T13:14:31.544 INFO:teuthology.orchestra.run.smithi049.stdout:(46/150): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-04-22T13:14:31.644 INFO:teuthology.orchestra.run.smithi049.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-04-22T13:14:31.987 INFO:teuthology.orchestra.run.smithi005.stdout:(4/150): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.1 MB/s | 2.4 MB 00:01 2024-04-22T13:14:32.370 INFO:teuthology.orchestra.run.smithi005.stdout:(5/150): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.8 MB/s | 5.4 MB 00:03 2024-04-22T13:14:32.411 INFO:teuthology.orchestra.run.smithi049.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 795 kB/s | 689 kB 00:00 2024-04-22T13:14:32.471 INFO:teuthology.orchestra.run.smithi005.stdout:(6/150): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-22T13:14:32.536 INFO:teuthology.orchestra.run.smithi049.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-04-22T13:14:32.746 INFO:teuthology.orchestra.run.smithi049.stdout:(50/150): ceph-common-19.0.0-3156.gd69f13d8.el7 2.4 MB/s | 21 MB 00:08 2024-04-22T13:14:32.770 INFO:teuthology.orchestra.run.smithi049.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 92 kB/s | 21 kB 00:00 2024-04-22T13:14:32.812 INFO:teuthology.orchestra.run.smithi049.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 807 kB/s | 943 kB 00:01 2024-04-22T13:14:32.888 INFO:teuthology.orchestra.run.smithi049.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-04-22T13:14:33.013 INFO:teuthology.orchestra.run.smithi049.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-04-22T13:14:33.146 INFO:teuthology.orchestra.run.smithi049.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-04-22T13:14:33.238 INFO:teuthology.orchestra.run.smithi005.stdout:(7/150): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.4 MB/s | 4.7 MB 00:00 2024-04-22T13:14:33.272 INFO:teuthology.orchestra.run.smithi049.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-22T13:14:33.405 INFO:teuthology.orchestra.run.smithi049.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-04-22T13:14:33.464 INFO:teuthology.orchestra.run.smithi049.stdout:(58/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-04-22T13:14:33.606 INFO:teuthology.orchestra.run.smithi049.stdout:(59/150): flex-2.6.4-9.el9.x86_64.rpm 365 kB/s | 313 kB 00:00 2024-04-22T13:14:33.631 INFO:teuthology.orchestra.run.smithi049.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-22T13:14:33.665 INFO:teuthology.orchestra.run.smithi049.stdout:(61/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-04-22T13:14:33.732 INFO:teuthology.orchestra.run.smithi049.stdout:(62/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 202 kB/s | 25 kB 00:00 2024-04-22T13:14:33.757 INFO:teuthology.orchestra.run.smithi049.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-04-22T13:14:33.791 INFO:teuthology.orchestra.run.smithi049.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-22T13:14:33.874 INFO:teuthology.orchestra.run.smithi049.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-04-22T13:14:33.900 INFO:teuthology.orchestra.run.smithi049.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 247 kB/s | 35 kB 00:00 2024-04-22T13:14:34.597 INFO:teuthology.orchestra.run.smithi107.stdout:(8/150): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 2.0 MB/s | 11 MB 00:05 2024-04-22T13:14:34.663 INFO:teuthology.orchestra.run.smithi107.stdout:(9/150): ceph-selinux-19.0.0-3156.gd69f13d8.el7 378 kB/s | 25 kB 00:00 2024-04-22T13:14:34.700 INFO:teuthology.orchestra.run.smithi049.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 4.1 MB 00:00 2024-04-22T13:14:34.838 INFO:teuthology.orchestra.run.smithi005.stdout:(8/150): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 21 MB 00:05 2024-04-22T13:14:34.955 INFO:teuthology.orchestra.run.smithi005.stdout:(9/150): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 6.2 MB/s | 11 MB 00:01 2024-04-22T13:14:34.981 INFO:teuthology.orchestra.run.smithi005.stdout:(10/150): ceph-selinux-19.0.0-3156.gd69f13d8.el 177 kB/s | 25 kB 00:00 2024-04-22T13:14:35.105 INFO:teuthology.orchestra.run.smithi005.stdout:(11/150): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.8 MB/s | 225 kB 00:00 2024-04-22T13:14:35.133 INFO:teuthology.orchestra.run.smithi049.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-04-22T13:14:35.189 INFO:teuthology.orchestra.run.smithi005.stdout:(12/150): cephfs-shell-19.0.0-3156.gd69f13d8.el 324 kB/s | 27 kB 00:00 2024-04-22T13:14:35.259 INFO:teuthology.orchestra.run.smithi049.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-22T13:14:35.281 INFO:teuthology.orchestra.run.smithi005.stdout:(13/150): libcephfs-devel-19.0.0-3156.gd69f13d8 346 kB/s | 32 kB 00:00 2024-04-22T13:14:35.524 INFO:teuthology.orchestra.run.smithi005.stdout:(14/150): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.0 MB/s | 747 kB 00:00 2024-04-22T13:14:35.607 INFO:teuthology.orchestra.run.smithi005.stdout:(15/150): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 163 kB 00:00 2024-04-22T13:14:35.691 INFO:teuthology.orchestra.run.smithi005.stdout:(16/150): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-22T13:14:35.780 INFO:teuthology.orchestra.run.smithi107.stdout:(10/150): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 17 MB 00:08 2024-04-22T13:14:35.807 INFO:teuthology.orchestra.run.smithi005.stdout:(17/150): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-22T13:14:35.897 INFO:teuthology.orchestra.run.smithi107.stdout:(11/150): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 225 kB 00:00 2024-04-22T13:14:35.975 INFO:teuthology.orchestra.run.smithi107.stdout:(12/150): cephfs-shell-19.0.0-3156.gd69f13d8.el 360 kB/s | 27 kB 00:00 2024-04-22T13:14:36.031 INFO:teuthology.orchestra.run.smithi107.stdout:(13/150): libcephfs-devel-19.0.0-3156.gd69f13d8 545 kB/s | 32 kB 00:00 2024-04-22T13:14:36.109 INFO:teuthology.orchestra.run.smithi049.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 647 kB/s | 911 kB 00:01 2024-04-22T13:14:36.348 INFO:teuthology.orchestra.run.smithi107.stdout:(14/150): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 747 kB 00:00 2024-04-22T13:14:36.374 INFO:teuthology.orchestra.run.smithi005.stdout:(18/150): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.3 MB/s | 17 MB 00:03 2024-04-22T13:14:36.450 INFO:teuthology.orchestra.run.smithi005.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.gd6 602 kB/s | 45 kB 00:00 2024-04-22T13:14:36.465 INFO:teuthology.orchestra.run.smithi107.stdout:(15/150): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-22T13:14:36.516 INFO:teuthology.orchestra.run.smithi005.stdout:(20/150): python3-ceph-common-19.0.0-3156.gd69f 2.0 MB/s | 135 kB 00:00 2024-04-22T13:14:36.540 INFO:teuthology.orchestra.run.smithi107.stdout:(16/150): librados-devel-19.0.0-3156.gd69f13d8. 1.6 MB/s | 126 kB 00:00 2024-04-22T13:14:36.633 INFO:teuthology.orchestra.run.smithi005.stdout:(21/150): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.5 MB/s | 5.4 MB 00:00 2024-04-22T13:14:36.658 INFO:teuthology.orchestra.run.smithi005.stdout:(22/150): python3-cephfs-19.0.0-3156.gd69f13d8. 1.1 MB/s | 161 kB 00:00 2024-04-22T13:14:36.717 INFO:teuthology.orchestra.run.smithi005.stdout:(23/150): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-22T13:14:36.740 INFO:teuthology.orchestra.run.smithi107.stdout:(17/150): libradosstriper1-19.0.0-3156.gd69f13d 2.4 MB/s | 500 kB 00:00 2024-04-22T13:14:36.784 INFO:teuthology.orchestra.run.smithi005.stdout:(24/150): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 298 kB 00:00 2024-04-22T13:14:36.810 INFO:teuthology.orchestra.run.smithi005.stdout:(25/150): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 99 kB 00:00 2024-04-22T13:14:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:(26/150): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 85 kB 00:00 2024-04-22T13:14:36.919 INFO:teuthology.orchestra.run.smithi005.stdout:(27/150): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-22T13:14:37.002 INFO:teuthology.orchestra.run.smithi005.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 411 kB/s | 34 kB 00:00 2024-04-22T13:14:37.078 INFO:teuthology.orchestra.run.smithi005.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.8 MB/s | 142 kB 00:00 2024-04-22T13:14:37.303 INFO:teuthology.orchestra.run.smithi005.stdout:(30/150): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 6.3 MB/s | 3.1 MB 00:00 2024-04-22T13:14:37.709 INFO:teuthology.orchestra.run.smithi049.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 646 kB/s | 1.0 MB 00:01 2024-04-22T13:14:37.757 INFO:teuthology.orchestra.run.smithi107.stdout:(18/150): ceph-common-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 21 MB 00:16 2024-04-22T13:14:37.849 INFO:teuthology.orchestra.run.smithi107.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.gd6 495 kB/s | 45 kB 00:00 2024-04-22T13:14:37.951 INFO:teuthology.orchestra.run.smithi049.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 362 kB/s | 87 kB 00:00 2024-04-22T13:14:38.016 INFO:teuthology.orchestra.run.smithi107.stdout:(20/150): python3-ceph-common-19.0.0-3156.gd69f 808 kB/s | 135 kB 00:00 2024-04-22T13:14:38.111 INFO:teuthology.orchestra.run.smithi005.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.4 MB/s | 3.5 MB 00:01 2024-04-22T13:14:38.166 INFO:teuthology.orchestra.run.smithi107.stdout:(21/150): python3-cephfs-19.0.0-3156.gd69f13d8. 1.1 MB/s | 161 kB 00:00 2024-04-22T13:14:38.295 INFO:teuthology.orchestra.run.smithi005.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.gd6 1.3 MB/s | 249 kB 00:00 2024-04-22T13:14:38.387 INFO:teuthology.orchestra.run.smithi005.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 539 kB/s | 49 kB 00:00 2024-04-22T13:14:38.400 INFO:teuthology.orchestra.run.smithi107.stdout:(22/150): python3-rados-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 320 kB 00:00 2024-04-22T13:14:38.459 INFO:teuthology.orchestra.run.smithi049.stdout:(73/150): openblas-openmp-0.3.21-2.el9.x86_64.r 1.0 MB/s | 4.7 MB 00:04 2024-04-22T13:14:38.479 INFO:teuthology.orchestra.run.smithi005.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3156.gd 180 kB/s | 16 kB 00:00 2024-04-22T13:14:38.571 INFO:teuthology.orchestra.run.smithi005.stdout:(35/150): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-04-22T13:14:38.601 INFO:teuthology.orchestra.run.smithi049.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-22T13:14:38.625 INFO:teuthology.orchestra.run.smithi107.stdout:(23/150): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 298 kB 00:00 2024-04-22T13:14:38.671 INFO:teuthology.orchestra.run.smithi005.stdout:(36/150): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 273 kB 00:00 2024-04-22T13:14:38.726 INFO:teuthology.orchestra.run.smithi107.stdout:(24/150): python3-rgw-19.0.0-3156.gd69f13d8.el7 994 kB/s | 99 kB 00:00 2024-04-22T13:14:38.743 INFO:teuthology.orchestra.run.smithi049.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-22T13:14:38.762 INFO:teuthology.orchestra.run.smithi005.stdout:(37/150): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.9 MB/s | 762 kB 00:00 2024-04-22T13:14:38.779 INFO:teuthology.orchestra.run.smithi005.stdout:(38/150): cephfs-top-19.0.0-3156.gd69f13d8.el7. 168 kB/s | 18 kB 00:00 2024-04-22T13:14:38.817 INFO:teuthology.orchestra.run.smithi107.stdout:(25/150): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 932 kB/s | 85 kB 00:00 2024-04-22T13:14:38.877 INFO:teuthology.orchestra.run.smithi049.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-04-22T13:14:38.960 INFO:teuthology.orchestra.run.smithi107.stdout:(26/150): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 5.4 MB 00:02 2024-04-22T13:14:39.011 INFO:teuthology.orchestra.run.smithi049.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-04-22T13:14:39.051 INFO:teuthology.orchestra.run.smithi107.stdout:(27/150): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 170 kB 00:00 2024-04-22T13:14:39.144 INFO:teuthology.orchestra.run.smithi107.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-22T13:14:39.144 INFO:teuthology.orchestra.run.smithi049.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-22T13:14:39.235 INFO:teuthology.orchestra.run.smithi107.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-22T13:14:39.278 INFO:teuthology.orchestra.run.smithi049.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-04-22T13:14:39.380 INFO:teuthology.orchestra.run.smithi005.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 130 kB/s | 80 kB 00:00 2024-04-22T13:14:39.622 INFO:teuthology.orchestra.run.smithi005.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 809 kB/s | 195 kB 00:00 2024-04-22T13:14:39.797 INFO:teuthology.orchestra.run.smithi005.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 790 kB/s | 803 kB 00:01 2024-04-22T13:14:39.856 INFO:teuthology.orchestra.run.smithi005.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-22T13:14:39.973 INFO:teuthology.orchestra.run.smithi005.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 912 kB/s | 106 kB 00:00 2024-04-22T13:14:40.098 INFO:teuthology.orchestra.run.smithi005.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-22T13:14:40.140 INFO:teuthology.orchestra.run.smithi005.stdout:(45/150): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-04-22T13:14:40.224 INFO:teuthology.orchestra.run.smithi005.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 998 kB/s | 125 kB 00:00 2024-04-22T13:14:40.265 INFO:teuthology.orchestra.run.smithi005.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-04-22T13:14:40.336 INFO:teuthology.orchestra.run.smithi107.stdout:(30/150): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.0 MB/s | 3.1 MB 00:01 2024-04-22T13:14:40.536 INFO:teuthology.orchestra.run.smithi107.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.7 MB/s | 3.5 MB 00:01 2024-04-22T13:14:40.658 INFO:teuthology.orchestra.run.smithi005.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-04-22T13:14:40.686 INFO:teuthology.orchestra.run.smithi107.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.gd6 1.6 MB/s | 249 kB 00:00 2024-04-22T13:14:40.758 INFO:teuthology.orchestra.run.smithi005.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-04-22T13:14:40.769 INFO:teuthology.orchestra.run.smithi107.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 592 kB/s | 49 kB 00:00 2024-04-22T13:14:40.783 INFO:teuthology.orchestra.run.smithi005.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 855 kB/s | 107 kB 00:00 2024-04-22T13:14:40.825 INFO:teuthology.orchestra.run.smithi005.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-22T13:14:40.845 INFO:teuthology.orchestra.run.smithi107.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-22T13:14:40.892 INFO:teuthology.orchestra.run.smithi005.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-04-22T13:14:40.950 INFO:teuthology.orchestra.run.smithi005.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-22T13:14:40.995 INFO:teuthology.orchestra.run.smithi107.stdout:(35/150): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 273 kB 00:00 2024-04-22T13:14:41.034 INFO:teuthology.orchestra.run.smithi005.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-04-22T13:14:41.118 INFO:teuthology.orchestra.run.smithi005.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-22T13:14:41.144 INFO:teuthology.orchestra.run.smithi049.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.7 MB/s | 5.1 MB 00:01 2024-04-22T13:14:41.268 INFO:teuthology.orchestra.run.smithi005.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-22T13:14:41.286 INFO:teuthology.orchestra.run.smithi049.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-22T13:14:41.345 INFO:teuthology.orchestra.run.smithi107.stdout:(36/150): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.1 MB/s | 762 kB 00:00 2024-04-22T13:14:41.360 INFO:teuthology.orchestra.run.smithi005.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 495 kB/s | 45 kB 00:00 2024-04-22T13:14:41.428 INFO:teuthology.orchestra.run.smithi049.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-22T13:14:41.462 INFO:teuthology.orchestra.run.smithi107.stdout:(37/150): cephfs-top-19.0.0-3156.gd69f13d8.el7. 156 kB/s | 18 kB 00:00 2024-04-22T13:14:41.562 INFO:teuthology.orchestra.run.smithi049.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 578 kB/s | 77 kB 00:00 2024-04-22T13:14:41.687 INFO:teuthology.orchestra.run.smithi049.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 338 kB/s | 42 kB 00:00 2024-04-22T13:14:41.710 INFO:teuthology.orchestra.run.smithi005.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-04-22T13:14:41.721 INFO:teuthology.orchestra.run.smithi107.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 311 kB/s | 80 kB 00:00 2024-04-22T13:14:41.829 INFO:teuthology.orchestra.run.smithi049.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-04-22T13:14:41.963 INFO:teuthology.orchestra.run.smithi049.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-22T13:14:42.038 INFO:teuthology.orchestra.run.smithi049.stdout:(87/150): postgresql-server-13.14-1.el9.x86_64. 873 kB/s | 5.8 MB 00:06 2024-04-22T13:14:42.052 INFO:teuthology.orchestra.run.smithi005.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-04-22T13:14:42.144 INFO:teuthology.orchestra.run.smithi005.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-04-22T13:14:42.163 INFO:teuthology.orchestra.run.smithi049.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 408 kB/s | 51 kB 00:00 2024-04-22T13:14:42.219 INFO:teuthology.orchestra.run.smithi005.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 633 kB/s | 47 kB 00:00 2024-04-22T13:14:42.280 INFO:teuthology.orchestra.run.smithi049.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 54 kB 00:00 2024-04-22T13:14:42.288 INFO:teuthology.orchestra.run.smithi107.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-22T13:14:42.345 INFO:teuthology.orchestra.run.smithi005.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-22T13:14:42.629 INFO:teuthology.orchestra.run.smithi107.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 572 kB/s | 195 kB 00:00 2024-04-22T13:14:42.755 INFO:teuthology.orchestra.run.smithi107.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-22T13:14:42.788 INFO:teuthology.orchestra.run.smithi005.stdout:(63/150): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.3 MB/s | 49 MB 00:07 2024-04-22T13:14:42.853 INFO:teuthology.orchestra.run.smithi005.stdout:(64/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.5 MB/s | 3.0 MB 00:01 2024-04-22T13:14:42.879 INFO:teuthology.orchestra.run.smithi005.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 353 kB/s | 188 kB 00:00 2024-04-22T13:14:42.996 INFO:teuthology.orchestra.run.smithi005.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 170 kB/s | 35 kB 00:00 2024-04-22T13:14:43.380 INFO:teuthology.orchestra.run.smithi107.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-04-22T13:14:43.405 INFO:teuthology.orchestra.run.smithi049.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 193 kB/s | 279 kB 00:01 2024-04-22T13:14:43.472 INFO:teuthology.orchestra.run.smithi107.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-22T13:14:43.522 INFO:teuthology.orchestra.run.smithi049.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-04-22T13:14:43.548 INFO:teuthology.orchestra.run.smithi107.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-22T13:14:43.614 INFO:teuthology.orchestra.run.smithi107.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-22T13:14:43.647 INFO:teuthology.orchestra.run.smithi049.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 354 kB/s | 44 kB 00:00 2024-04-22T13:14:43.773 INFO:teuthology.orchestra.run.smithi107.stdout:(46/150): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-04-22T13:14:43.789 INFO:teuthology.orchestra.run.smithi049.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-22T13:14:43.798 INFO:teuthology.orchestra.run.smithi107.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-04-22T13:14:43.846 INFO:teuthology.orchestra.run.smithi005.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-04-22T13:14:43.915 INFO:teuthology.orchestra.run.smithi049.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 287 kB/s | 36 kB 00:00 2024-04-22T13:14:44.040 INFO:teuthology.orchestra.run.smithi049.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 378 kB/s | 47 kB 00:00 2024-04-22T13:14:44.088 INFO:teuthology.orchestra.run.smithi005.stdout:(68/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 4.1 MB 00:01 2024-04-22T13:14:44.174 INFO:teuthology.orchestra.run.smithi049.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-22T13:14:44.222 INFO:teuthology.orchestra.run.smithi005.stdout:(69/150): openblas-openmp-0.3.21-2.el9.x86_64.r 3.5 MB/s | 4.7 MB 00:01 2024-04-22T13:14:44.247 INFO:teuthology.orchestra.run.smithi005.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 841 kB/s | 133 kB 00:00 2024-04-22T13:14:44.307 INFO:teuthology.orchestra.run.smithi049.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 433 kB/s | 58 kB 00:00 2024-04-22T13:14:44.372 INFO:teuthology.orchestra.run.smithi005.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-04-22T13:14:44.432 INFO:teuthology.orchestra.run.smithi107.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-04-22T13:14:44.449 INFO:teuthology.orchestra.run.smithi049.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 662 kB/s | 94 kB 00:00 2024-04-22T13:14:44.557 INFO:teuthology.orchestra.run.smithi107.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-04-22T13:14:44.575 INFO:teuthology.orchestra.run.smithi049.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 509 kB/s | 64 kB 00:00 2024-04-22T13:14:44.589 INFO:teuthology.orchestra.run.smithi005.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 404 kB/s | 87 kB 00:00 2024-04-22T13:14:44.622 INFO:teuthology.orchestra.run.smithi005.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-04-22T13:14:44.641 INFO:teuthology.orchestra.run.smithi107.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 258 kB/s | 21 kB 00:00 2024-04-22T13:14:44.700 INFO:teuthology.orchestra.run.smithi049.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-04-22T13:14:44.774 INFO:teuthology.orchestra.run.smithi107.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-22T13:14:44.789 INFO:teuthology.orchestra.run.smithi005.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-22T13:14:44.808 INFO:teuthology.orchestra.run.smithi107.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 934 kB/s | 943 kB 00:01 2024-04-22T13:14:44.864 INFO:teuthology.orchestra.run.smithi005.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-22T13:14:44.875 INFO:teuthology.orchestra.run.smithi107.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 325 kB/s | 33 kB 00:00 2024-04-22T13:14:44.948 INFO:teuthology.orchestra.run.smithi005.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 307 kB/s | 26 kB 00:00 2024-04-22T13:14:44.992 INFO:teuthology.orchestra.run.smithi107.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-04-22T13:14:45.023 INFO:teuthology.orchestra.run.smithi005.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-04-22T13:14:45.107 INFO:teuthology.orchestra.run.smithi005.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-22T13:14:45.117 INFO:teuthology.orchestra.run.smithi107.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-22T13:14:45.182 INFO:teuthology.orchestra.run.smithi005.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-22T13:14:45.326 INFO:teuthology.orchestra.run.smithi107.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 768 kB/s | 160 kB 00:00 2024-04-22T13:14:45.366 INFO:teuthology.orchestra.run.smithi005.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-04-22T13:14:45.434 INFO:teuthology.orchestra.run.smithi107.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-04-22T13:14:45.458 INFO:teuthology.orchestra.run.smithi005.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-22T13:14:45.467 INFO:teuthology.orchestra.run.smithi049.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-04-22T13:14:45.501 INFO:teuthology.orchestra.run.smithi107.stdout:(58/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.3 MB/s | 3.0 MB 00:00 2024-04-22T13:14:45.550 INFO:teuthology.orchestra.run.smithi005.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-22T13:14:45.642 INFO:teuthology.orchestra.run.smithi005.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 843 kB/s | 77 kB 00:00 2024-04-22T13:14:45.736 INFO:teuthology.orchestra.run.smithi005.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 450 kB/s | 42 kB 00:00 2024-04-22T13:14:45.811 INFO:teuthology.orchestra.run.smithi005.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-22T13:14:45.903 INFO:teuthology.orchestra.run.smithi005.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-22T13:14:45.991 INFO:teuthology.orchestra.run.smithi005.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 279 kB 00:00 2024-04-22T13:14:46.079 INFO:teuthology.orchestra.run.smithi005.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 586 kB/s | 51 kB 00:00 2024-04-22T13:14:46.145 INFO:teuthology.orchestra.run.smithi005.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 807 kB/s | 54 kB 00:00 2024-04-22T13:14:46.437 INFO:teuthology.orchestra.run.smithi005.stdout:(90/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-04-22T13:14:46.485 INFO:teuthology.orchestra.run.smithi107.stdout:(59/150): ceph-test-19.0.0-3156.gd69f13d8.el7.x 4.2 MB/s | 49 MB 00:11 2024-04-22T13:14:46.504 INFO:teuthology.orchestra.run.smithi005.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-04-22T13:14:46.518 INFO:teuthology.orchestra.run.smithi107.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 611 kB/s | 662 kB 00:01 2024-04-22T13:14:46.529 INFO:teuthology.orchestra.run.smithi005.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-04-22T13:14:46.543 INFO:teuthology.orchestra.run.smithi107.stdout:(61/150): libtool-2.4.6-45.el9.x86_64.rpm 559 kB/s | 582 kB 00:01 2024-04-22T13:14:46.572 INFO:teuthology.orchestra.run.smithi005.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-04-22T13:14:46.577 INFO:teuthology.orchestra.run.smithi107.stdout:(62/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-22T13:14:46.611 INFO:teuthology.orchestra.run.smithi107.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-04-22T13:14:46.647 INFO:teuthology.orchestra.run.smithi005.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 479 kB/s | 36 kB 00:00 2024-04-22T13:14:46.672 INFO:teuthology.orchestra.run.smithi005.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-22T13:14:46.722 INFO:teuthology.orchestra.run.smithi005.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-22T13:14:46.753 INFO:teuthology.orchestra.run.smithi107.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-22T13:14:46.798 INFO:teuthology.orchestra.run.smithi005.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-22T13:14:46.825 INFO:teuthology.orchestra.run.smithi005.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-04-22T13:14:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 146 kB/s | 35 kB 00:00 2024-04-22T13:14:46.873 INFO:teuthology.orchestra.run.smithi005.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-22T13:14:46.878 INFO:teuthology.orchestra.run.smithi107.stdout:(66/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 875 kB/s | 292 kB 00:00 2024-04-22T13:14:46.898 INFO:teuthology.orchestra.run.smithi005.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-22T13:14:46.949 INFO:teuthology.orchestra.run.smithi005.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-04-22T13:14:47.232 INFO:teuthology.orchestra.run.smithi005.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-04-22T13:14:47.541 INFO:teuthology.orchestra.run.smithi005.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-22T13:14:47.659 INFO:teuthology.orchestra.run.smithi049.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 394 kB/s | 863 kB 00:02 2024-04-22T13:14:47.762 INFO:teuthology.orchestra.run.smithi107.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 4.7 MB/s | 4.7 MB 00:01 2024-04-22T13:14:47.804 INFO:teuthology.orchestra.run.smithi107.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-04-22T13:14:47.841 INFO:teuthology.orchestra.run.smithi005.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 901 kB/s | 548 kB 00:00 2024-04-22T13:14:47.920 INFO:teuthology.orchestra.run.smithi107.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-22T13:14:47.924 INFO:teuthology.orchestra.run.smithi005.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 234 kB/s | 19 kB 00:00 2024-04-22T13:14:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 2.6 MB/s | 911 kB 00:00 2024-04-22T13:14:48.167 INFO:teuthology.orchestra.run.smithi005.stdout:(106/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.0 MB/s | 16 MB 00:02 2024-04-22T13:14:48.192 INFO:teuthology.orchestra.run.smithi005.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 475 kB/s | 309 kB 00:00 2024-04-22T13:14:48.292 INFO:teuthology.orchestra.run.smithi005.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2024-04-22T13:14:48.388 INFO:teuthology.orchestra.run.smithi107.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-04-22T13:14:48.401 INFO:teuthology.orchestra.run.smithi005.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 622 kB/s | 67 kB 00:00 2024-04-22T13:14:48.409 INFO:teuthology.orchestra.run.smithi049.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 731 kB/s | 548 kB 00:00 2024-04-22T13:14:48.434 INFO:teuthology.orchestra.run.smithi005.stdout:(110/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-22T13:14:48.510 INFO:teuthology.orchestra.run.smithi005.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-22T13:14:48.521 INFO:teuthology.orchestra.run.smithi107.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 656 kB/s | 87 kB 00:00 2024-04-22T13:14:48.559 INFO:teuthology.orchestra.run.smithi049.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-04-22T13:14:48.610 INFO:teuthology.orchestra.run.smithi005.stdout:(112/150): libarrow-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-22T13:14:48.659 INFO:teuthology.orchestra.run.smithi049.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-22T13:14:48.760 INFO:teuthology.orchestra.run.smithi005.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-04-22T13:14:48.836 INFO:teuthology.orchestra.run.smithi005.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-04-22T13:14:48.869 INFO:teuthology.orchestra.run.smithi005.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-22T13:14:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-22T13:14:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:(117/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-22T13:14:49.201 INFO:teuthology.orchestra.run.smithi049.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-04-22T13:14:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-22T13:14:49.302 INFO:teuthology.orchestra.run.smithi049.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-22T13:14:49.303 INFO:teuthology.orchestra.run.smithi005.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-22T13:14:49.347 INFO:teuthology.orchestra.run.smithi107.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 4.1 MB/s | 5.8 MB 00:01 2024-04-22T13:14:49.387 INFO:teuthology.orchestra.run.smithi005.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-04-22T13:14:49.402 INFO:teuthology.orchestra.run.smithi049.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-22T13:14:49.412 INFO:teuthology.orchestra.run.smithi005.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-22T13:14:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-04-22T13:14:49.521 INFO:teuthology.orchestra.run.smithi005.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-22T13:14:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-04-22T13:14:49.546 INFO:teuthology.orchestra.run.smithi005.stdout:(123/150): python3-cheroot-8.6.0-4.el9.noarch.r 254 kB/s | 172 kB 00:00 2024-04-22T13:14:49.630 INFO:teuthology.orchestra.run.smithi005.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-22T13:14:49.663 INFO:teuthology.orchestra.run.smithi005.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-22T13:14:49.728 INFO:teuthology.orchestra.run.smithi049.stdout:(110/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-04-22T13:14:49.730 INFO:teuthology.orchestra.run.smithi005.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-22T13:14:49.753 INFO:teuthology.orchestra.run.smithi049.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 647 kB/s | 151 kB 00:00 2024-04-22T13:14:49.756 INFO:teuthology.orchestra.run.smithi005.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-22T13:14:49.806 INFO:teuthology.orchestra.run.smithi107.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 930 kB/s | 248 kB 00:00 2024-04-22T13:14:49.831 INFO:teuthology.orchestra.run.smithi005.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-22T13:14:49.853 INFO:teuthology.orchestra.run.smithi049.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-04-22T13:14:49.878 INFO:teuthology.orchestra.run.smithi049.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-22T13:14:49.940 INFO:teuthology.orchestra.run.smithi005.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-04-22T13:14:49.947 INFO:teuthology.orchestra.run.smithi107.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 181 kB/s | 26 kB 00:00 2024-04-22T13:14:49.962 INFO:teuthology.orchestra.run.smithi049.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 405 kB/s | 43 kB 00:00 2024-04-22T13:14:49.981 INFO:teuthology.orchestra.run.smithi005.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-22T13:14:49.987 INFO:teuthology.orchestra.run.smithi049.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-22T13:14:50.006 INFO:teuthology.orchestra.run.smithi107.stdout:(77/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.3 MB/s | 4.1 MB 00:03 2024-04-22T13:14:50.049 INFO:teuthology.orchestra.run.smithi005.stdout:(131/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-04-22T13:14:50.054 INFO:teuthology.orchestra.run.smithi049.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-04-22T13:14:50.074 INFO:teuthology.orchestra.run.smithi005.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-04-22T13:14:50.081 INFO:teuthology.orchestra.run.smithi107.stdout:(78/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-04-22T13:14:50.087 INFO:teuthology.orchestra.run.smithi049.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-22T13:14:50.099 INFO:teuthology.orchestra.run.smithi005.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2024-04-22T13:14:50.107 INFO:teuthology.orchestra.run.smithi107.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 19 kB 00:00 2024-04-22T13:14:50.149 INFO:teuthology.orchestra.run.smithi005.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-22T13:14:50.163 INFO:teuthology.orchestra.run.smithi049.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-22T13:14:50.183 INFO:teuthology.orchestra.run.smithi005.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-22T13:14:50.197 INFO:teuthology.orchestra.run.smithi049.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-22T13:14:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 859 kB/s | 172 kB 00:00 2024-04-22T13:14:50.208 INFO:teuthology.orchestra.run.smithi005.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-22T13:14:50.232 INFO:teuthology.orchestra.run.smithi107.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 231 kB/s | 35 kB 00:00 2024-04-22T13:14:50.280 INFO:teuthology.orchestra.run.smithi049.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-22T13:14:50.283 INFO:teuthology.orchestra.run.smithi005.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-22T13:14:50.308 INFO:teuthology.orchestra.run.smithi005.stdout:(138/150): python3-pyperclip-1.8.2-2.el9.noarch 177 kB/s | 28 kB 00:00 2024-04-22T13:14:50.316 INFO:teuthology.orchestra.run.smithi049.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-04-22T13:14:50.334 INFO:teuthology.orchestra.run.smithi005.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-22T13:14:50.393 INFO:teuthology.orchestra.run.smithi005.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-04-22T13:14:50.451 INFO:teuthology.orchestra.run.smithi005.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-04-22T13:14:50.457 INFO:teuthology.orchestra.run.smithi107.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-04-22T13:14:50.534 INFO:teuthology.orchestra.run.smithi005.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-22T13:14:50.541 INFO:teuthology.orchestra.run.smithi107.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 928 kB/s | 77 kB 00:00 2024-04-22T13:14:50.541 INFO:teuthology.orchestra.run.smithi049.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-04-22T13:14:50.560 INFO:teuthology.orchestra.run.smithi005.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-22T13:14:50.566 INFO:teuthology.orchestra.run.smithi049.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 578 kB/s | 144 kB 00:00 2024-04-22T13:14:50.625 INFO:teuthology.orchestra.run.smithi107.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 567 kB/s | 222 kB 00:00 2024-04-22T13:14:50.635 INFO:teuthology.orchestra.run.smithi005.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-22T13:14:50.650 INFO:teuthology.orchestra.run.smithi107.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-04-22T13:14:50.652 INFO:teuthology.orchestra.run.smithi005.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-22T13:14:50.683 INFO:teuthology.orchestra.run.smithi049.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-22T13:14:50.717 INFO:teuthology.orchestra.run.smithi049.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 118 kB/s | 18 kB 00:00 2024-04-22T13:14:50.744 INFO:teuthology.orchestra.run.smithi005.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-22T13:14:50.769 INFO:teuthology.orchestra.run.smithi005.stdout:(147/150): python3-tempora-5.0.0-2.el9.noarch.r 78 kB/s | 36 kB 00:00 2024-04-22T13:14:50.817 INFO:teuthology.orchestra.run.smithi107.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 953 kB/s | 159 kB 00:00 2024-04-22T13:14:50.817 INFO:teuthology.orchestra.run.smithi049.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-22T13:14:50.842 INFO:teuthology.orchestra.run.smithi049.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-04-22T13:14:50.859 INFO:teuthology.orchestra.run.smithi107.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-04-22T13:14:50.943 INFO:teuthology.orchestra.run.smithi049.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-22T13:14:50.984 INFO:teuthology.orchestra.run.smithi107.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 408 kB/s | 51 kB 00:00 2024-04-22T13:14:50.984 INFO:teuthology.orchestra.run.smithi049.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-22T13:14:51.067 INFO:teuthology.orchestra.run.smithi107.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-04-22T13:14:51.085 INFO:teuthology.orchestra.run.smithi049.stdout:(130/150): python3-babel-2.9.1-2.el9.noarch.rpm 465 kB/s | 6.0 MB 00:13 2024-04-22T13:14:51.101 INFO:teuthology.orchestra.run.smithi107.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 459 kB/s | 54 kB 00:00 2024-04-22T13:14:51.110 INFO:teuthology.orchestra.run.smithi049.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 281 kB/s | 46 kB 00:00 2024-04-22T13:14:51.135 INFO:teuthology.orchestra.run.smithi049.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 523 kB/s | 79 kB 00:00 2024-04-22T13:14:51.194 INFO:teuthology.orchestra.run.smithi049.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 533 kB/s | 58 kB 00:00 2024-04-22T13:14:51.218 INFO:teuthology.orchestra.run.smithi107.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-04-22T13:14:51.252 INFO:teuthology.orchestra.run.smithi049.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-22T13:14:51.277 INFO:teuthology.orchestra.run.smithi049.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-04-22T13:14:51.303 INFO:teuthology.orchestra.run.smithi049.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-04-22T13:14:51.345 INFO:teuthology.orchestra.run.smithi005.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-22T13:14:51.351 INFO:teuthology.orchestra.run.smithi107.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 331 kB/s | 44 kB 00:00 2024-04-22T13:14:51.361 INFO:teuthology.orchestra.run.smithi049.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 339 kB/s | 28 kB 00:00 2024-04-22T13:14:51.387 INFO:teuthology.orchestra.run.smithi049.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-22T13:14:51.412 INFO:teuthology.orchestra.run.smithi049.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-22T13:14:51.454 INFO:teuthology.orchestra.run.smithi049.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-04-22T13:14:51.479 INFO:teuthology.orchestra.run.smithi049.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-04-22T13:14:51.510 INFO:teuthology.orchestra.run.smithi107.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-04-22T13:14:51.521 INFO:teuthology.orchestra.run.smithi049.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-22T13:14:51.563 INFO:teuthology.orchestra.run.smithi049.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-22T13:14:51.588 INFO:teuthology.orchestra.run.smithi049.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-22T13:14:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-22T13:14:51.644 INFO:teuthology.orchestra.run.smithi107.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 269 kB/s | 36 kB 00:00 2024-04-22T13:14:51.672 INFO:teuthology.orchestra.run.smithi049.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-22T13:14:51.697 INFO:teuthology.orchestra.run.smithi049.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-22T13:14:51.777 INFO:teuthology.orchestra.run.smithi107.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 356 kB/s | 47 kB 00:00 2024-04-22T13:14:51.881 INFO:teuthology.orchestra.run.smithi049.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-04-22T13:14:51.961 INFO:teuthology.orchestra.run.smithi107.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-22T13:14:52.040 INFO:teuthology.orchestra.run.smithi049.stdout:(149/150): librbd1-19.0.0-3156.gd69f13d8.el7.x8 9.2 MB/s | 3.1 MB 00:00 2024-04-22T13:14:52.103 INFO:teuthology.orchestra.run.smithi107.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 408 kB/s | 58 kB 00:00 2024-04-22T13:14:52.120 INFO:teuthology.orchestra.run.smithi005.stdout:(149/150): librados2-19.0.0-3156.gd69f13d8.el7. 2.5 MB/s | 3.4 MB 00:01 2024-04-22T13:14:52.132 INFO:teuthology.orchestra.run.smithi049.stdout:(150/150): librados2-19.0.0-3156.gd69f13d8.el7. 7.4 MB/s | 3.4 MB 00:00 2024-04-22T13:14:52.136 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:14:52.136 INFO:teuthology.orchestra.run.smithi049.stdout:Total 6.1 MB/s | 218 MB 00:35 2024-04-22T13:14:52.203 INFO:teuthology.orchestra.run.smithi107.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 939 kB/s | 94 kB 00:00 2024-04-22T13:14:52.228 INFO:teuthology.orchestra.run.smithi005.stdout:(150/150): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.2 MB/s | 3.1 MB 00:01 2024-04-22T13:14:52.233 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:14:52.234 INFO:teuthology.orchestra.run.smithi005.stdout:Total 9.2 MB/s | 218 MB 00:23 2024-04-22T13:14:52.270 INFO:teuthology.orchestra.run.smithi107.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 957 kB/s | 64 kB 00:00 2024-04-22T13:14:52.356 INFO:teuthology.orchestra.run.smithi107.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 524 kB/s | 45 kB 00:00 2024-04-22T13:14:52.422 INFO:teuthology.orchestra.run.smithi107.stdout:(101/150): python3-numpy-1.20.1-5.el9.x86_64.rp 2.2 MB/s | 5.1 MB 00:02 2024-04-22T13:14:53.048 INFO:teuthology.orchestra.run.smithi107.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-04-22T13:14:53.195 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-22T13:14:53.315 INFO:teuthology.orchestra.run.smithi107.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-22T13:14:53.332 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-22T13:14:53.332 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-22T13:14:53.345 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-04-22T13:14:53.381 INFO:teuthology.orchestra.run.smithi107.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-04-22T13:14:53.482 INFO:teuthology.orchestra.run.smithi107.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-22T13:14:53.483 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-04-22T13:14:53.483 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-04-22T13:14:53.724 INFO:teuthology.orchestra.run.smithi107.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-22T13:14:53.774 INFO:teuthology.orchestra.run.smithi107.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-04-22T13:14:53.824 INFO:teuthology.orchestra.run.smithi107.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-04-22T13:14:53.883 INFO:teuthology.orchestra.run.smithi107.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-22T13:14:53.942 INFO:teuthology.orchestra.run.smithi107.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-22T13:14:54.017 INFO:teuthology.orchestra.run.smithi107.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-22T13:14:54.067 INFO:teuthology.orchestra.run.smithi107.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-04-22T13:14:54.109 INFO:teuthology.orchestra.run.smithi107.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-22T13:14:54.160 INFO:teuthology.orchestra.run.smithi107.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-04-22T13:14:54.201 INFO:teuthology.orchestra.run.smithi107.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 361 kB/s | 15 kB 00:00 2024-04-22T13:14:54.260 INFO:teuthology.orchestra.run.smithi107.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-22T13:14:54.310 INFO:teuthology.orchestra.run.smithi107.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-22T13:14:54.419 INFO:teuthology.orchestra.run.smithi107.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-22T13:14:54.477 INFO:teuthology.orchestra.run.smithi107.stdout:(119/150): protobuf-compiler-3.14.0-13.el9.x86_ 420 kB/s | 863 kB 00:02 2024-04-22T13:14:54.503 INFO:teuthology.orchestra.run.smithi107.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-04-22T13:14:54.612 INFO:teuthology.orchestra.run.smithi107.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-22T13:14:54.670 INFO:teuthology.orchestra.run.smithi107.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-22T13:14:54.703 INFO:teuthology.orchestra.run.smithi107.stdout:(123/150): python3-google-auth-2.28.2-1.el9.noa 899 kB/s | 203 kB 00:00 2024-04-22T13:14:54.721 INFO:teuthology.orchestra.run.smithi107.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-04-22T13:14:54.746 INFO:teuthology.orchestra.run.smithi107.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-04-22T13:14:54.762 INFO:teuthology.orchestra.run.smithi107.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 554 kB/s | 23 kB 00:00 2024-04-22T13:14:54.788 INFO:teuthology.orchestra.run.smithi107.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-04-22T13:14:54.814 INFO:teuthology.orchestra.run.smithi107.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 388 kB/s | 20 kB 00:00 2024-04-22T13:14:54.864 INFO:teuthology.orchestra.run.smithi107.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-22T13:14:54.889 INFO:teuthology.orchestra.run.smithi107.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-04-22T13:14:54.915 INFO:teuthology.orchestra.run.smithi107.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-22T13:14:54.956 INFO:teuthology.orchestra.run.smithi107.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-04-22T13:14:54.982 INFO:teuthology.orchestra.run.smithi107.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-22T13:14:55.015 INFO:teuthology.orchestra.run.smithi107.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-22T13:14:55.041 INFO:teuthology.orchestra.run.smithi107.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-22T13:14:55.099 INFO:teuthology.orchestra.run.smithi107.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2024-04-22T13:14:55.124 INFO:teuthology.orchestra.run.smithi107.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-22T13:14:55.150 INFO:teuthology.orchestra.run.smithi107.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-22T13:14:55.153 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-22T13:14:55.153 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-22T13:14:55.184 INFO:teuthology.orchestra.run.smithi107.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-22T13:14:55.209 INFO:teuthology.orchestra.run.smithi107.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 609 kB/s | 36 kB 00:00 2024-04-22T13:14:55.242 INFO:teuthology.orchestra.run.smithi107.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 872 kB/s | 51 kB 00:00 2024-04-22T13:14:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-22T13:14:55.310 INFO:teuthology.orchestra.run.smithi107.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-22T13:14:55.312 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-04-22T13:14:55.312 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-04-22T13:14:55.343 INFO:teuthology.orchestra.run.smithi107.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-22T13:14:55.368 INFO:teuthology.orchestra.run.smithi107.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-22T13:14:55.394 INFO:teuthology.orchestra.run.smithi107.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-22T13:14:55.569 INFO:teuthology.orchestra.run.smithi107.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-22T13:14:56.677 INFO:teuthology.orchestra.run.smithi107.stdout:(148/150): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-22T13:14:56.727 INFO:teuthology.orchestra.run.smithi107.stdout:(149/150): librados2-19.0.0-3156.gd69f13d8.el7. 2.5 MB/s | 3.4 MB 00:01 2024-04-22T13:14:57.736 INFO:teuthology.orchestra.run.smithi107.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.4 MB/s | 16 MB 00:06 2024-04-22T13:14:57.742 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:14:57.742 INFO:teuthology.orchestra.run.smithi107.stdout:Total 5.1 MB/s | 218 MB 00:42 2024-04-22T13:14:58.751 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T13:14:58.877 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T13:14:58.877 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T13:15:00.600 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T13:15:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T13:15:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-22T13:15:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-04-22T13:15:01.226 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T13:15:01.251 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T13:15:01.282 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T13:15:01.295 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T13:15:01.306 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T13:15:01.350 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T13:15:01.643 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T13:15:01.744 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T13:15:01.762 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:15:01.798 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:15:01.885 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:15:01.906 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:15:01.911 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:01.987 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:02.009 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:02.029 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:15:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:15:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T13:15:02.079 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T13:15:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T13:15:02.105 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T13:15:02.244 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T13:15:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T13:15:02.314 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T13:15:02.327 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T13:15:02.342 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:02.362 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:02.448 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:02.476 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:15:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:15:02.538 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:15:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:15:02.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:15:02.578 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:15:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:15:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:15:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:15:02.785 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:15:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:15:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:15:02.904 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T13:15:02.943 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T13:15:03.001 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T13:15:03.034 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T13:15:03.041 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T13:15:03.078 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T13:15:03.084 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T13:15:03.133 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T13:15:03.170 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T13:15:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T13:15:03.211 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T13:15:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T13:15:03.254 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T13:15:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T13:15:03.292 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T13:15:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T13:15:03.333 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T13:15:03.360 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T13:15:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T13:15:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T13:15:03.446 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T13:15:03.475 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T13:15:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T13:15:03.511 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T13:15:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T13:15:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T13:15:03.601 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T13:15:03.638 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 30/152 2024-04-22T13:15:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T13:15:03.682 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 31/152 2024-04-22T13:15:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 30/152 2024-04-22T13:15:03.702 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 32/152 2024-04-22T13:15:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 31/152 2024-04-22T13:15:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 32/152 2024-04-22T13:15:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T13:15:03.826 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T13:15:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T13:15:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T13:15:03.903 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T13:15:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T13:15:04.050 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T13:15:04.104 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T13:15:04.121 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T13:15:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T13:15:04.666 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T13:15:04.751 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T13:15:04.764 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T13:15:04.785 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T13:15:04.829 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T13:15:04.849 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T13:15:04.867 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 42/152 2024-04-22T13:15:04.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T13:15:04.900 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T13:15:04.924 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T13:15:04.957 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T13:15:04.963 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 42/152 2024-04-22T13:15:04.996 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T13:15:05.031 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T13:15:05.054 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T13:15:05.067 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T13:15:05.104 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T13:15:05.129 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T13:15:05.152 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T13:15:05.171 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T13:15:05.186 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T13:15:05.208 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T13:15:05.222 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T13:15:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T13:15:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T13:15:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T13:15:05.335 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T13:15:05.377 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T13:15:05.408 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T13:15:05.454 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T13:15:05.483 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T13:15:05.524 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T13:15:05.527 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T13:15:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T13:15:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T13:15:05.623 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T13:15:05.646 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T13:15:05.658 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T13:15:05.698 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T13:15:05.731 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T13:15:05.747 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T13:15:05.755 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T13:15:05.786 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T13:15:05.827 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T13:15:05.834 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T13:15:05.860 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T13:15:05.884 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T13:15:05.965 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T13:15:05.972 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T13:15:06.005 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T13:15:06.033 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T13:15:06.068 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T13:15:06.104 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T13:15:06.112 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T13:15:06.142 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T13:15:06.150 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T13:15:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T13:15:06.189 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T13:15:06.210 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T13:15:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T13:15:06.245 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T13:15:06.258 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T13:15:06.278 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T13:15:06.295 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T13:15:06.320 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T13:15:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T13:15:06.363 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T13:15:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T13:15:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T13:15:06.432 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T13:15:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T13:15:06.489 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T13:15:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T13:15:06.506 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T13:15:06.516 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T13:15:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T13:15:06.580 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T13:15:06.581 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T13:15:06.605 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T13:15:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T13:15:06.656 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T13:15:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T13:15:06.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T13:15:06.694 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T13:15:06.743 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T13:15:06.752 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T13:15:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T13:15:06.932 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T13:15:06.992 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T13:15:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:15:07.211 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:15:07.240 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:07.315 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:07.356 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:15:07.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T13:15:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T13:15:07.533 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T13:15:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T13:15:07.591 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T13:15:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T13:15:07.624 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T13:15:07.653 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T13:15:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T13:15:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:15:07.791 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T13:15:07.802 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T13:15:07.836 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T13:15:07.842 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:15:07.864 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:15:07.881 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T13:15:07.885 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:15:07.918 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T13:15:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T13:15:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T13:15:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:15:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:15:08.089 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T13:15:08.102 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T13:15:08.129 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T13:15:08.131 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T13:15:08.170 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T13:15:08.216 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T13:15:08.258 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T13:15:08.295 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T13:15:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T13:15:08.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T13:15:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T13:15:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T13:15:08.415 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T13:15:08.446 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T13:15:08.498 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T13:15:08.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T13:15:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T13:15:08.629 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T13:15:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T13:15:08.736 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T13:15:08.780 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T13:15:08.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T13:15:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 30/152 2024-04-22T13:15:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 31/152 2024-04-22T13:15:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 32/152 2024-04-22T13:15:09.043 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T13:15:09.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T13:15:09.114 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T13:15:09.138 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T13:15:09.170 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T13:15:09.226 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T13:15:09.257 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T13:15:09.323 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T13:15:09.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T13:15:09.899 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T13:15:09.977 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T13:15:10.004 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T13:15:10.046 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T13:15:10.085 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 42/152 2024-04-22T13:15:10.118 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T13:15:10.172 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T13:15:10.246 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T13:15:10.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T13:15:10.313 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T13:15:10.362 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T13:15:10.396 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T13:15:10.439 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T13:15:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T13:15:10.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T13:15:10.618 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T13:15:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T13:15:10.644 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T13:15:10.662 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T13:15:10.672 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T13:15:10.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T13:15:10.700 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T13:15:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T13:15:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T13:15:10.741 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T13:15:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T13:15:10.837 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T13:15:10.883 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T13:15:10.924 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T13:15:10.949 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T13:15:10.974 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T13:15:11.029 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T13:15:11.048 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T13:15:11.068 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:15:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T13:15:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:15:11.143 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:15:11.173 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 98/152 2024-04-22T13:15:11.176 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T13:15:11.209 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T13:15:11.214 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T13:15:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:15:11.242 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T13:15:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 98/152 2024-04-22T13:15:11.276 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T13:15:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T13:15:11.321 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T13:15:11.358 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T13:15:11.389 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T13:15:11.419 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T13:15:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T13:15:11.487 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T13:15:11.528 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T13:15:11.563 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T13:15:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T13:15:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T13:15:11.718 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T13:15:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T13:15:11.746 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:15:11.772 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T13:15:11.800 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T13:15:11.818 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:15:11.839 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:15:11.847 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 102/152 2024-04-22T13:15:11.870 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T13:15:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:15:11.917 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T13:15:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 102/152 2024-04-22T13:15:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T13:15:12.690 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T13:15:12.723 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T13:15:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T13:15:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T13:15:12.960 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T13:15:13.008 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T13:15:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T13:15:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T13:15:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T13:15:13.931 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T13:15:14.222 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T13:15:14.227 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:14.261 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:14.350 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:14.387 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:14.835 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:14.872 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:15:14.950 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:15:14.950 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T13:15:14.950 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T13:15:14.950 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:14.960 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:14.982 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:15:14.999 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:15:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:15:15.091 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T13:15:15.091 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T13:15:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:15.117 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:15:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T13:15:15.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T13:15:15.579 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T13:15:15.605 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T13:15:15.915 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T13:15:15.967 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:15:16.057 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:15:16.088 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 98/152 2024-04-22T13:15:16.124 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T13:15:16.602 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T13:15:16.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:15:16.708 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:15:16.730 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 102/152 2024-04-22T13:15:19.056 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:19.101 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:19.649 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:15:19.690 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:15:19.782 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:15:19.782 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T13:15:19.782 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T13:15:19.782 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:19.816 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:15:26.743 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-04-22T13:15:26.744 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:26.928 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:15:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:15:26.945 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:26.945 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T13:15:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:27.123 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-04-22T13:15:27.124 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:15:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:15:27.345 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:27.345 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T13:15:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:28.498 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:15:28.525 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:15:28.526 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:28.526 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T13:15:28.526 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:15:28.526 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:15:28.526 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:28.766 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:15:28.789 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:15:28.789 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:28.789 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T13:15:28.789 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:15:28.789 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:15:28.790 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:28.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:15:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:15:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T13:15:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:15:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:15:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:29.148 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:15:29.172 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:15:29.172 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:29.172 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T13:15:29.172 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:15:29.173 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:15:29.173 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:29.258 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:15:29.281 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:15:29.282 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:29.282 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T13:15:29.282 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:15:29.282 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:15:29.282 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:29.389 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T13:15:29.447 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T13:15:29.519 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T13:15:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:15:29.664 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:15:29.664 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:29.664 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T13:15:29.664 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:15:29.664 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:15:29.664 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:29.714 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T13:15:29.759 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T13:15:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T13:15:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T13:15:29.848 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T13:15:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T13:15:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T13:15:30.081 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T13:15:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T13:15:30.118 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T13:15:30.150 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T13:15:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T13:15:30.177 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T13:15:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T13:15:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T13:15:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T13:15:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T13:15:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T13:15:30.573 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:15:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T13:15:30.600 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:15:30.633 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 122/152 2024-04-22T13:15:30.723 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 123/152 2024-04-22T13:15:30.790 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:15:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:15:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:15:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 122/152 2024-04-22T13:15:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 123/152 2024-04-22T13:15:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-04-22T13:15:31.229 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-04-22T13:15:31.230 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:31.413 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:15:31.426 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:15:31.426 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:31.426 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T13:15:31.426 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:31.905 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:15:31.934 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:15:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:15:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:15:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:15:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:15:32.906 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:32.906 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T13:15:32.906 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:15:32.906 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:15:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:15:32.984 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:15:33.093 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:15:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:15:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:15:33.154 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:33.154 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T13:15:33.154 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:15:33.154 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:15:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 127/152 2024-04-22T13:15:33.303 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:15:33.327 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:15:33.327 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:33.328 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T13:15:33.328 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:15:33.328 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:15:33.328 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:33.378 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:15:33.379 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:15:33.390 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:15:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:15:33.493 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/152 2024-04-22T13:15:33.517 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:15:33.545 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 131/152 2024-04-22T13:15:33.599 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:15:33.621 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:15:33.621 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:33.621 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T13:15:33.621 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:15:33.621 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:15:33.621 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 127/152 2024-04-22T13:15:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:15:33.720 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T13:15:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:15:33.743 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:33.743 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T13:15:33.743 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:15:33.743 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:15:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:33.777 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T13:15:33.794 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:15:33.806 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:15:33.832 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T13:15:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/152 2024-04-22T13:15:33.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 131/152 2024-04-22T13:15:34.026 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T13:15:34.074 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T13:15:34.152 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T13:15:34.248 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T13:15:34.388 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T13:15:34.433 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T13:15:34.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T13:15:34.494 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T13:15:34.531 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T13:15:34.565 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:15:34.588 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:15:34.589 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:34.589 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T13:15:34.589 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:15:34.589 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:15:34.589 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:34.632 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:15:34.655 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:15:34.655 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:34.656 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T13:15:34.656 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:34.708 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:15:34.731 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:15:34.731 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:34.731 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T13:15:34.731 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:15:34.732 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:15:34.732 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:15:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T13:15:34.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:15:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:15:34.987 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:15:34.987 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:34.987 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T13:15:34.987 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:15:34.987 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:15:34.987 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:34.990 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 122/152 2024-04-22T13:15:35.030 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:15:35.050 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:15:35.051 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 123/152 2024-04-22T13:15:35.054 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:15:35.054 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:35.054 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T13:15:35.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:15:35.074 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:35.074 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T13:15:35.074 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:15:35.074 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:15:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:15:35.106 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:15:35.132 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:15:35.133 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:35.133 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T13:15:35.133 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:15:35.133 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:15:35.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:35.448 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:15:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:15:35.473 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:35.473 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T13:15:35.473 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:15:35.473 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:15:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:15:36.149 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:15:37.111 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:15:37.141 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:15:37.242 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:15:37.412 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 127/152 2024-04-22T13:15:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:15:37.466 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:15:37.466 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:37.467 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T13:15:37.467 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:15:37.467 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:15:37.467 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:37.512 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:15:37.522 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:15:37.621 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/152 2024-04-22T13:15:37.670 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 131/152 2024-04-22T13:15:38.589 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T13:15:38.614 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:15:38.636 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:15:38.636 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:38.636 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T13:15:38.636 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:15:38.636 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:15:38.636 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:38.689 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:15:38.710 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:15:38.710 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:38.710 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T13:15:38.710 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:38.756 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:15:38.778 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:15:38.778 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:38.778 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T13:15:38.778 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:15:38.778 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:15:38.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:39.082 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:15:39.104 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:15:39.104 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:15:39.104 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T13:15:39.104 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:15:39.104 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:15:39.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/152 2024-04-22T13:15:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/152 2024-04-22T13:15:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/152 2024-04-22T13:15:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T13:15:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T13:15:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/152 2024-04-22T13:15:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 142/152 2024-04-22T13:15:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/152 2024-04-22T13:15:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T13:15:40.410 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/152 2024-04-22T13:15:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 144/152 2024-04-22T13:15:40.441 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T13:15:40.468 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T13:15:40.672 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 142/152 2024-04-22T13:15:40.714 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T13:15:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 144/152 2024-04-22T13:15:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:41.416 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:41.437 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:42.002 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 146/152 2024-04-22T13:15:42.098 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T13:15:42.140 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T13:15:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T13:15:42.261 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T13:15:42.262 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:15:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 146/152 2024-04-22T13:15:42.345 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:15:42.345 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:42.429 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T13:15:42.472 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T13:15:42.567 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T13:15:42.608 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T13:15:42.608 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:15:42.700 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:15:42.700 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:43.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/152 2024-04-22T13:15:43.751 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/152 2024-04-22T13:15:43.793 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/152 2024-04-22T13:15:43.825 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T13:15:43.860 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T13:15:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 142/152 2024-04-22T13:15:44.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T13:15:44.120 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 144/152 2024-04-22T13:15:44.787 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:44.826 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:44.846 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:15:45.625 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 146/152 2024-04-22T13:15:45.722 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T13:15:45.762 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T13:15:45.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T13:15:45.876 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T13:15:45.876 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:15:45.959 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:15:45.960 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/152 2024-04-22T13:15:48.095 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/152 2024-04-22T13:15:48.096 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/152 2024-04-22T13:15:48.097 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T13:15:48.098 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T13:15:48.099 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T13:15:48.100 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T13:15:48.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T13:15:48.102 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T13:15:48.103 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T13:15:48.104 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T13:15:48.105 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/152 2024-04-22T13:15:48.754 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/152 2024-04-22T13:15:48.755 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/152 2024-04-22T13:15:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/152 2024-04-22T13:15:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/152 2024-04-22T13:15:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/152 2024-04-22T13:15:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/152 2024-04-22T13:15:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/152 2024-04-22T13:15:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/152 2024-04-22T13:15:48.768 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T13:15:48.769 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T13:15:48.770 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T13:15:48.771 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T13:15:48.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T13:15:48.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T13:15:48.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T13:15:48.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T13:15:48.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T13:15:48.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/152 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-22T13:15:50.281 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.282 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.283 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.284 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T13:15:50.285 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T13:15:50.286 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T13:15:50.286 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T13:15:50.287 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T13:15:50.288 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T13:15:50.289 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T13:15:50.290 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T13:15:50.291 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T13:15:50.292 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T13:15:50.293 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T13:15:50.294 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T13:15:50.294 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-22T13:15:50.294 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-22T13:15:50.654 DEBUG:teuthology.parallel:result is None 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T13:15:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T13:15:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T13:15:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T13:15:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T13:15:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T13:15:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T13:15:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T13:15:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T13:15:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:15:50.921 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-22T13:15:51.262 DEBUG:teuthology.parallel:result is None 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/152 2024-04-22T13:15:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/152 2024-04-22T13:15:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/152 2024-04-22T13:15:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T13:15:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T13:15:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T13:15:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T13:15:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T13:15:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T13:15:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T13:15:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T13:15:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T13:15:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T13:15:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T13:15:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T13:15:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T13:15:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T13:15:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T13:15:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T13:15:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/152 2024-04-22T13:15:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T13:15:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/152 2024-04-22T13:15:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:15:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:53.928 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-04-22T13:15:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T13:15:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T13:15:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T13:15:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T13:15:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T13:15:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T13:15:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T13:15:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T13:15:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T13:15:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T13:15:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T13:15:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:15:53.941 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T13:15:54.314 DEBUG:teuthology.parallel:result is None 2024-04-22T13:15:54.315 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:15:54.463 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T13:15:54.490 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-22T13:15:54.490 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-22T13:15:54.490 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-22T13:15:54.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:15:54.636 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T13:15:54.662 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-22T13:15:54.662 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-22T13:15:54.662 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-22T13:15:54.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:15:55.019 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T13:15:55.040 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-22T13:15:55.040 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-22T13:15:55.040 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-22T13:15:55.042 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T13:15:55.042 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:15:55.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T13:15:55.075 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:15:55.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T13:15:55.107 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:15:55.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T13:15:55.132 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T13:15:55.132 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:15:55.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T13:15:55.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T13:15:55.229 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:15:55.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T13:15:55.257 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T13:15:55.324 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:15:55.325 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T13:15:55.351 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T13:15:55.415 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T13:15:55.415 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:15:55.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T13:15:55.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T13:15:55.512 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:15:55.512 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T13:15:55.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T13:15:55.608 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:15:55.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T13:15:55.631 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T13:15:55.694 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T13:15:55.694 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:15:55.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T13:15:55.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T13:15:55.793 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:15:55.793 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T13:15:55.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T13:15:55.890 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:15:55.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T13:15:55.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T13:15:55.979 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-22T13:15:56.088 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-22T13:15:56.088 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:15:56.089 INFO:tasks.cephadm:Cluster fsid is 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:15:56.089 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-22T13:15:56.089 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.49', 'mon.c': '172.21.15.107'} 2024-04-22T13:15:56.089 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-04-22T13:15:56.089 INFO:tasks.cephadm:First mgr is x 2024-04-22T13:15:56.089 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-22T13:15:56.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-04-22T13:15:56.121 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-04-22T13:15:56.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-04-22T13:15:56.176 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-22T13:15:56.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:15:56.519 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-22T13:15:56.898 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:15:56.900 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-22T13:15:56.900 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:57.936 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 22 13:15 /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:57.936 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:58.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 22 13:15 /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:58.765 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:59.611 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 22 13:15 /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:59.611 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:59.631 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:59.650 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T13:15:59.686 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-22T13:15:59.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-22T13:15:59.689 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-22T13:15:59.693 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-22T13:16:01.239 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:16:01.243 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:16:01.292 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:16:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T13:16:35.526 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-22T13:16:35.526 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-22T13:16:35.526 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-04-22T13:16:35.526 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-22T13:16:35.526 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-22T13:16:35.527 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T13:17:01.399 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:17:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-22T13:17:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-22T13:17:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-04-22T13:17:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-22T13:17:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:17:01.400 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-22T13:17:03.016 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-22T13:17:03.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-04-22T13:17:03.076 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-04-22T13:17:03.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-04-22T13:17:03.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-04-22T13:17:03.165 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-04-22T13:17:03.190 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-04-22T13:17:03.217 INFO:tasks.cephadm:Writing seed config... 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-22T13:17:03.219 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-22T13:17:03.220 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-22T13:17:03.221 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-22T13:17:03.222 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-22T13:17:03.222 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-22T13:17:03.222 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-22T13:17:03.222 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-22T13:17:03.222 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-22T13:17:03.222 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-22T13:17:03.223 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:17:03.223 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-22T13:17:03.240 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 73e95e94-00aa-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-22T13:17:03.241 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a.service 2024-04-22T13:17:03.283 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x.service 2024-04-22T13:17:03.325 INFO:tasks.cephadm:Bootstrapping... 2024-04-22T13:17:03.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:17:03.527 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:17:03.527 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '73e95e94-00aa-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-04-22T13:17:03.528 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-22T13:17:03.528 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-04-22T13:17:03.555 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T13:17:03.555 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-04-22T13:17:03.555 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-04-22T13:17:03.562 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T13:17:03.562 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T13:17:03.569 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T13:17:03.569 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-22T13:17:03.575 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-04-22T13:17:03.582 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-04-22T13:17:03.582 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-04-22T13:17:03.582 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-04-22T13:17:03.605 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T13:17:03.606 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-22T13:17:03.606 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-04-22T13:17:03.606 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-04-22T13:17:03.612 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T13:17:03.612 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T13:17:03.618 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T13:17:03.618 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-22T13:17:03.624 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-04-22T13:17:03.631 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-04-22T13:17:03.631 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-04-22T13:17:03.631 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-04-22T13:17:03.631 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:03.632 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 139705411779744 on /run/cephadm/73e95e94-00aa-11ef-bc93-c7b262605968.lock 2024-04-22T13:17:03.632 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 139705411779744 acquired on /run/cephadm/73e95e94-00aa-11ef-bc93-c7b262605968.lock 2024-04-22T13:17:03.632 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-04-22T13:17:03.632 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-04-22T13:17:03.632 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-04-22T13:17:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-04-22T13:17:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-04-22T13:17:03.639 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-22T13:17:03.639 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-22T13:17:03.639 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-22T13:17:03.641 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-22T13:17:03.641 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-22T13:17:03.641 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T13:17:03.641 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-22T13:17:03.641 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-22T13:17:03.641 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T13:17:03.642 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-04-22T13:17:03.642 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-04-22T13:17:03.642 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-22T13:17:03.643 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-22T13:17:03.643 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-22T13:17:04.168 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-22T13:17:05.039 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:17:05.039 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:17:05.039 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-04-22T13:17:05.953 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-04-22T13:17:05.953 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-04-22T13:17:06.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBSYyZmnAZjDxAA9cxn6mwocQ5gwr+2FCIy9w== 2024-04-22T13:17:07.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBTYyZmfu1iCRAAQshP4iED6W91AG4cIpqAUA== 2024-04-22T13:17:08.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBUYyZmWDV+CRAAQbvr5Ia0vU3btPsKqPyikw== 2024-04-22T13:17:08.861 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-04-22T13:17:09.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:17:09.738 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-04-22T13:17:10.685 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-04-22T13:17:10.913 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-22T13:17:11.056 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-22T13:17:11.255 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-73e95e94-00aa-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-73e95e94-00aa-11ef-bc93-c7b262605968.target. 2024-04-22T13:17:11.255 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-73e95e94-00aa-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-73e95e94-00aa-11ef-bc93-c7b262605968.target. 2024-04-22T13:17:11.529 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a 2024-04-22T13:17:11.529 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-22T13:17:11.713 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-73e95e94-00aa-11ef-bc93-c7b262605968.target.wants/ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-73e95e94-00aa-11ef-bc93-c7b262605968@.service. 2024-04-22T13:17:11.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:11 smithi005 systemd[1]: Starting Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:17:12.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:11 smithi005 podman[27615]: 2024-04-22 13:17:11.978929644 +0000 UTC m=+0.106867569 container create 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-22T13:17:12.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:11 smithi005 podman[27615]: 2024-04-22 13:17:11.8956942 +0000 UTC m=+0.023632134 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:17:12.252 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:17:12.252 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-22T13:17:12.259 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:17:12.259 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-22T13:17:12.260 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-22T13:17:12.260 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T13:17:12.260 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-04-22T13:17:12.260 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-04-22T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:12 smithi005 podman[27615]: 2024-04-22 13:17:12.164757079 +0000 UTC m=+0.292695020 container init 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-22T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:12 smithi005 podman[27615]: 2024-04-22 13:17:12.168182148 +0000 UTC m=+0.296120082 container start 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:12 smithi005 ceph-mon[27633]: mkfs 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:12 smithi005 ceph-mon[27633]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-22T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:12 smithi005 bash[27615]: 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 2024-04-22T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:12 smithi005 systemd[1]: Started Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:17:13.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-22T13:17:13.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: monmap epoch 1 2024-04-22T13:17:13.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: last_changed 2024-04-22T13:17:09.176897+0000 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: min_mon_release 19 (squid) 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: election_strategy: 1 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: fsmap 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: mgrmap e1: no daemons active 2024-04-22T13:17:13.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:13 smithi005 ceph-mon[27633]: from='client.? 172.21.15.5:0/1080081805' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-22T13:17:13.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.575858s) 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-04-22T13:17:13.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb46726b700 1 Processor -- start 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb46726b700 1 -- start start 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb46726b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46010b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb46726b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb46007be60 con 0x7fb460108fd0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb465007700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46010b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb465007700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46010b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45412/0 (socket says 172.21.15.5:45412) 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.702+0000 7fb465007700 1 -- 172.21.15.5:0/1651521178 learned_addr learned my addr 172.21.15.5:0/1651521178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.703+0000 7fb465007700 1 -- 172.21.15.5:0/1651521178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46010b900 con 0x7fb460108fd0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.703+0000 7fb465007700 1 --2- 172.21.15.5:0/1651521178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46010b3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0098a0 tx=0x7fb45c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfeebcf1e5bb8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.703+0000 7fb457fff700 1 -- 172.21.15.5:0/1651521178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c004160 con 0x7fb460108fd0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.703+0000 7fb457fff700 1 -- 172.21.15.5:0/1651521178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c004300 con 0x7fb460108fd0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.703+0000 7fb457fff700 1 -- 172.21.15.5:0/1651521178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c0036a0 con 0x7fb460108fd0 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 -- 172.21.15.5:0/1651521178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 msgr2=0x7fb46010b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 --2- 172.21.15.5:0/1651521178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46010b3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0098a0 tx=0x7fb45c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 -- 172.21.15.5:0/1651521178 shutdown_connections 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 --2- 172.21.15.5:0/1651521178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46010b3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 -- 172.21.15.5:0/1651521178 >> 172.21.15.5:0/1651521178 conn(0x7fb4601029a0 msgr2=0x7fb460104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 -- 172.21.15.5:0/1651521178 shutdown_connections 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 -- 172.21.15.5:0/1651521178 wait complete. 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 Processor -- start 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.704+0000 7fb46726b700 1 -- start start 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb46726b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb46726b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb46010bd70 con 0x7fb460108fd0 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb465007700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb465007700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45418/0 (socket says 172.21.15.5:45418) 2024-04-22T13:17:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb465007700 1 -- 172.21.15.5:0/1080081805 learned_addr learned my addr 172.21.15.5:0/1080081805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb465007700 1 -- 172.21.15.5:0/1080081805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46019c1e0 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb465007700 1 --2- 172.21.15.5:0/1080081805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46019bca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb45c000c00 tx=0x7fb45c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c003db0 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c011630 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46019c470 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c011950 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.705+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46019f160 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.706+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb45c01a460 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.706+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb45c01b780 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.706+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4600640a0 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.709+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb45c01ba20 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.782+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb46002e1b0 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.782+0000 7fb4567fc700 1 -- 172.21.15.5:0/1080081805 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb45c01bc00 con 0x7fb460108fd0 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 msgr2=0x7fb46019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:13.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 --2- 172.21.15.5:0/1080081805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46019bca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb45c000c00 tx=0x7fb45c004750 comp rx=0 tx=0).stop 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 shutdown_connections 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 --2- 172.21.15.5:0/1080081805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb460108fd0 0x7fb46019bca0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 >> 172.21.15.5:0/1080081805 conn(0x7fb4601029a0 msgr2=0x7fb460104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 shutdown_connections 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:12.783+0000 7fb46726b700 1 -- 172.21.15.5:0/1080081805 wait complete. 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-04-22T13:17:13.376 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-04-22T13:17:14.496 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:14 smithi005 ceph-mon[27633]: from='client.? 172.21.15.5:0/3899778317' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T13:17:14.496 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:14 smithi005 ceph-mon[27633]: from='client.? 172.21.15.5:0/3899778317' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-22T13:17:14.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:14.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T13:17:14.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:14.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T13:17:14.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.830+0000 7fda4727f700 1 Processor -- start 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.831+0000 7fda4727f700 1 -- start start 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.831+0000 7fda4727f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda40108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.831+0000 7fda4727f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda40108f30 con 0x7fda40108540 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.831+0000 7fda4501b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda40108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.831+0000 7fda4501b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda40108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45432/0 (socket says 172.21.15.5:45432) 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.831+0000 7fda4501b700 1 -- 172.21.15.5:0/356643072 learned_addr learned my addr 172.21.15.5:0/356643072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4501b700 1 -- 172.21.15.5:0/356643072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda401096f0 con 0x7fda40108540 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4501b700 1 --2- 172.21.15.5:0/356643072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda40108960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fda300098a0 tx=0x7fda30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8226f1df8d099a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda37fff700 1 -- 172.21.15.5:0/356643072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda30004160 con 0x7fda40108540 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda37fff700 1 -- 172.21.15.5:0/356643072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda30004300 con 0x7fda40108540 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda37fff700 1 -- 172.21.15.5:0/356643072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda300036a0 con 0x7fda40108540 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4727f700 1 -- 172.21.15.5:0/356643072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 msgr2=0x7fda40108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:14.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4727f700 1 --2- 172.21.15.5:0/356643072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda40108960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fda300098a0 tx=0x7fda30009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4727f700 1 -- 172.21.15.5:0/356643072 shutdown_connections 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4727f700 1 --2- 172.21.15.5:0/356643072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda40108960 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.832+0000 7fda4727f700 1 -- 172.21.15.5:0/356643072 >> 172.21.15.5:0/356643072 conn(0x7fda40103b40 msgr2=0x7fda40105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4727f700 1 -- 172.21.15.5:0/356643072 shutdown_connections 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4727f700 1 -- 172.21.15.5:0/356643072 wait complete. 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4727f700 1 Processor -- start 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4727f700 1 -- start start 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4727f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda4007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4727f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda40109c20 con 0x7fda40108540 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4501b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda4007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4501b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda4007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45444/0 (socket says 172.21.15.5:45444) 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.833+0000 7fda4501b700 1 -- 172.21.15.5:0/3899778317 learned_addr learned my addr 172.21.15.5:0/3899778317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda4501b700 1 -- 172.21.15.5:0/3899778317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4007e0c0 con 0x7fda40108540 2024-04-22T13:17:14.509 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda4501b700 1 --2- 172.21.15.5:0/3899778317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda4007db80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fda30000c00 tx=0x7fda30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda30003db0 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda30011630 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4007c2b0 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda30011950 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4007c7f0 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda3001a460 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.834+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda3001b780 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.835+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda400640a0 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.837+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fda30031020 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.908+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fda40195190 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.912+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+490 (secure 0 0 0) 0x7fda3001b9d0 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda367fc700 1 -- 172.21.15.5:0/3899778317 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fda3001acd0 con 0x7fda40108540 2024-04-22T13:17:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 msgr2=0x7fda4007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 --2- 172.21.15.5:0/3899778317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda4007db80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fda30000c00 tx=0x7fda30004750 comp rx=0 tx=0).stop 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 shutdown_connections 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 --2- 172.21.15.5:0/3899778317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda40108540 0x7fda4007db80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 >> 172.21.15.5:0/3899778317 conn(0x7fda40103b40 msgr2=0x7fda401928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 shutdown_connections 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:13.913+0000 7fda4727f700 1 -- 172.21.15.5:0/3899778317 wait complete. 2024-04-22T13:17:14.511 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-04-22T13:17:15.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:15 smithi005 ceph-mon[27633]: from='client.? 172.21.15.5:0/374051192' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.907+0000 7fe132ec4700 1 Processor -- start 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.907+0000 7fe132ec4700 1 -- start start 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.907+0000 7fe132ec4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.907+0000 7fe132ec4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c108f60 con 0x7fe12c108570 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe130c60700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe130c60700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45452/0 (socket says 172.21.15.5:45452) 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe130c60700 1 -- 172.21.15.5:0/3777660855 learned_addr learned my addr 172.21.15.5:0/3777660855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe130c60700 1 -- 172.21.15.5:0/3777660855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c109720 con 0x7fe12c108570 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe130c60700 1 --2- 172.21.15.5:0/3777660855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c108990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe124009e80 tx=0x7fe12400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743d4d41784298e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe12b7fe700 1 -- 172.21.15.5:0/3777660855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe124017070 con 0x7fe12c108570 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe12b7fe700 1 -- 172.21.15.5:0/3777660855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe12400b8c0 con 0x7fe12c108570 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.908+0000 7fe12b7fe700 1 -- 172.21.15.5:0/3777660855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe124004490 con 0x7fe12c108570 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 -- 172.21.15.5:0/3777660855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 msgr2=0x7fe12c108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 --2- 172.21.15.5:0/3777660855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c108990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe124009e80 tx=0x7fe12400b330 comp rx=0 tx=0).stop 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 -- 172.21.15.5:0/3777660855 shutdown_connections 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 --2- 172.21.15.5:0/3777660855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c108990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 -- 172.21.15.5:0/3777660855 >> 172.21.15.5:0/3777660855 conn(0x7fe12c103b70 msgr2=0x7fe12c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 -- 172.21.15.5:0/3777660855 shutdown_connections 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.909+0000 7fe132ec4700 1 -- 172.21.15.5:0/3777660855 wait complete. 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe132ec4700 1 Processor -- start 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe132ec4700 1 -- start start 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe132ec4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe132ec4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c10cad0 con 0x7fe12c108570 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe130c60700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe130c60700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45454/0 (socket says 172.21.15.5:45454) 2024-04-22T13:17:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe130c60700 1 -- 172.21.15.5:0/374051192 learned_addr learned my addr 172.21.15.5:0/374051192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe130c60700 1 -- 172.21.15.5:0/374051192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c19c9b0 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.910+0000 7fe130c60700 1 --2- 172.21.15.5:0/374051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c19c470 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b7b0 tx=0x7fe12400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe124020020 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe124012430 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c19cc40 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe124017040 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c19f930 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe124023020 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.911+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe12400f050 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.912+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c0640a0 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.914+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe12402a020 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.985+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe12c19fb00 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.985+0000 7fe129ffb700 1 -- 172.21.15.5:0/374051192 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fe12400b860 con 0x7fe12c108570 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.986+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 msgr2=0x7fe12c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.986+0000 7fe132ec4700 1 --2- 172.21.15.5:0/374051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c19c470 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b7b0 tx=0x7fe12400be20 comp rx=0 tx=0).stop 2024-04-22T13:17:15.600 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.986+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 shutdown_connections 2024-04-22T13:17:15.600 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.986+0000 7fe132ec4700 1 --2- 172.21.15.5:0/374051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe12c108570 0x7fe12c19c470 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:15.600 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.986+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 >> 172.21.15.5:0/374051192 conn(0x7fe12c103b70 msgr2=0x7fe12c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:15.600 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.986+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 shutdown_connections 2024-04-22T13:17:15.600 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:14.987+0000 7fe132ec4700 1 -- 172.21.15.5:0/374051192 wait complete. 2024-04-22T13:17:15.600 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-04-22T13:17:15.868 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:15 smithi005 systemd[1]: Stopping Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:17:16.200 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a[27629]: 2024-04-22T13:17:15.869+0000 7fb95ca1e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:17:16.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a[27629]: 2024-04-22T13:17:15.869+0000 7fb95ca1e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T13:17:16.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:15 smithi005 podman[27880]: 2024-04-22 13:17:15.982397207 +0000 UTC m=+0.209434871 container died 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-22T13:17:16.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 podman[27880]: 2024-04-22 13:17:16.294109262 +0000 UTC m=+0.521146934 container cleanup 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-22T13:17:16.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 bash[27880]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a 2024-04-22T13:17:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 podman[27894]: 2024-04-22 13:17:16.503675782 +0000 UTC m=+0.516006827 container remove 92feba4bfbeba400334ffc702a7d5b21a2ebe66a1e89480b8e6bc46308dbb9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-22T13:17:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-22T13:17:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 systemd[1]: Stopped Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:17:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 systemd[1]: Starting Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:17:17.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 podman[27987]: 2024-04-22 13:17:16.870948742 +0000 UTC m=+0.113917095 container create ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T13:17:17.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:16 smithi005 podman[27987]: 2024-04-22 13:17:16.775704417 +0000 UTC m=+0.018672787 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:17:17.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 podman[27987]: 2024-04-22 13:17:17.106776088 +0000 UTC m=+0.349744439 container init ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-22T13:17:17.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 podman[27987]: 2024-04-22 13:17:17.109841397 +0000 UTC m=+0.352809748 container start ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T13:17:17.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T13:17:17.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-22T13:17:17.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: pidfile_write: ignore empty --pid-file 2024-04-22T13:17:17.192 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-22T13:17:17.399 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: load: jerasure load: lrc 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: RocksDB version: 7.9.2 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Git sha 0 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: DB SUMMARY 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: DB Session ID: EX30YDUY5XY4EPK1NIGL 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: CURRENT file: CURRENT 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-22T13:17:17.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105442 ; 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.error_if_exists: 0 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.create_if_missing: 0 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.paranoid_checks: 1 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.env: 0x560e2a2a3cc0 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.info_log: 0x560e2b639be0 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.statistics: (nil) 2024-04-22T13:17:17.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.use_fsync: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_log_file_size: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.allow_fallocate: 1 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.use_direct_reads: 0 2024-04-22T13:17:17.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.db_log_dir: 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.wal_dir: 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T13:17:17.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.write_buffer_manager: 0x560e2b5cd0e0 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T13:17:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.unordered_write: 0 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.row_cache: None 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.wal_filter: None 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T13:17:17.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.two_write_queues: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.wal_compression: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.atomic_flush: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.log_readahead_size: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T13:17:17.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_background_jobs: 2 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_background_compactions: -1 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_subcompactions: 1 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T13:17:17.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_open_files: -1 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_background_flushes: -1 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Compression algorithms supported: 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kZSTD supported: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kXpressCompression supported: 0 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T13:17:17.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kZlibCompression supported: 1 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kSnappyCompression supported: 1 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kLZ4Compression supported: 1 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: kBZip2Compression supported: 0 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.merge_operator: 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_filter: None 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T13:17:17.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e2b639dc0) 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T13:17:17.411 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x560e2b5ff1f0 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-04-22T13:17:17.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-04-22T13:17:17.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression: NoCompression 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.num_levels: 7 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T13:17:17.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T13:17:17.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T13:17:17.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T13:17:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T13:17:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.inplace_update_support: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.bloom_locality: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.max_successive_merges: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T13:17:17.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.ttl: 2592000 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.enable_blob_files: false 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.min_blob_size: 0 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T13:17:17.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dc66df5d-7e4e-4999-b532-5ce19dda2998 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791837139116, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791837140131, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100527, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98421, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14272, "raw_average_key_size": 49, "raw_value_size": 90685, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713791837, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dc66df5d-7e4e-4999-b532-5ce19dda2998", "db_session_id": "EX30YDUY5XY4EPK1NIGL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791837140215, "job": 1, "event": "recovery_finished"} 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e2b714000 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: DB pointer 0x560e2b6fe000 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T13:17:17.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-22T13:17:17.422 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 100.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 100.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-22T13:17:17.423 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 31.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 31.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x560e2b5ff1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-22T13:17:17.424 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???) e1 preinit fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).mds e1 new map 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).mds e1 print_map 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T13:17:17.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:17:17.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:17:17.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:17:17.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 ceph-mon[28006]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-22T13:17:17.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 bash[27987]: ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 2024-04-22T13:17:17.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:17 smithi005 systemd[1]: Started Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:17:18.270 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: monmap epoch 1 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: last_changed 2024-04-22T13:17:09.176897+0000 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: min_mon_release 19 (squid) 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: election_strategy: 1 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: fsmap 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: mgrmap e1: no daemons active 2024-04-22T13:17:18.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:18 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3473096996' entity='client.admin' 2024-04-22T13:17:18.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.616+0000 7f27246c2700 1 Processor -- start 2024-04-22T13:17:18.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.616+0000 7f27246c2700 1 -- start start 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.616+0000 7f27246c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c10b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.616+0000 7f27246c2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10b910 con 0x7f271c108fe0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272245e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c10b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272245e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c10b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45470/0 (socket says 172.21.15.5:45470) 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272245e700 1 -- 172.21.15.5:0/908660376 learned_addr learned my addr 172.21.15.5:0/908660376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272245e700 1 -- 172.21.15.5:0/908660376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c10ba90 con 0x7f271c108fe0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272245e700 1 --2- 172.21.15.5:0/908660376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c10b3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f27180098a0 tx=0x7f2718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba10cf3b62ea25bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272145c700 1 -- 172.21.15.5:0/908660376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2718004160 con 0x7f271c108fe0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272145c700 1 -- 172.21.15.5:0/908660376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2718004300 con 0x7f271c108fe0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.617+0000 7f272145c700 1 -- 172.21.15.5:0/908660376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2718011af0 con 0x7f271c108fe0 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 -- 172.21.15.5:0/908660376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 msgr2=0x7f271c10b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 --2- 172.21.15.5:0/908660376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c10b3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f27180098a0 tx=0x7f2718009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 -- 172.21.15.5:0/908660376 shutdown_connections 2024-04-22T13:17:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 --2- 172.21.15.5:0/908660376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c10b3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 -- 172.21.15.5:0/908660376 >> 172.21.15.5:0/908660376 conn(0x7f271c1029f0 msgr2=0x7f271c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 -- 172.21.15.5:0/908660376 shutdown_connections 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 -- 172.21.15.5:0/908660376 wait complete. 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 Processor -- start 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.618+0000 7f27246c2700 1 -- start start 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27246c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27246c2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10c9a0 con 0x7f271c108fe0 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f272245e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f272245e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45474/0 (socket says 172.21.15.5:45474) 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f272245e700 1 -- 172.21.15.5:0/3473096996 learned_addr learned my addr 172.21.15.5:0/3473096996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f272245e700 1 -- 172.21.15.5:0/3473096996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c19caa0 con 0x7f271c108fe0 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f272245e700 1 --2- 172.21.15.5:0/3473096996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c19c560 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2718011ac0 tx=0x7f2718011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2718003970 con 0x7f271c108fe0 2024-04-22T13:17:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2718003b10 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c19cd30 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2718024610 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.619+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c19fa20 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.620+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2718024c20 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.620+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2718019700 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.620+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c0640a0 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.623+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2718043e50 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.695+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f271c19ffd0 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.697+0000 7f27137fe700 1 -- 172.21.15.5:0/3473096996 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2718032030 con 0x7f271c108fe0 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 msgr2=0x7f271c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 --2- 172.21.15.5:0/3473096996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c19c560 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2718011ac0 tx=0x7f2718011ba0 comp rx=0 tx=0).stop 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 shutdown_connections 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 --2- 172.21.15.5:0/3473096996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271c108fe0 0x7f271c19c560 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 >> 172.21.15.5:0/3473096996 conn(0x7f271c1029f0 msgr2=0x7f271c103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:18.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 shutdown_connections 2024-04-22T13:17:18.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:17.698+0000 7f27246c2700 1 -- 172.21.15.5:0/3473096996 wait complete. 2024-04-22T13:17:18.281 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-22T13:17:18.312 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:17:18.312 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-04-22T13:17:18.312 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-22T13:17:18.313 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-22T13:17:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x 2024-04-22T13:17:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-22T13:17:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-73e95e94-00aa-11ef-bc93-c7b262605968.target.wants/ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-73e95e94-00aa-11ef-bc93-c7b262605968@.service. 2024-04-22T13:17:18.956 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:18 smithi005 systemd[1]: Starting Ceph mgr.x for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:17:19.288 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:18 smithi005 podman[28221]: 2024-04-22 13:17:18.958695962 +0000 UTC m=+0.100312121 container create e9cd05080807eaeb3ec9f511ba4f47d354227ea9230beab6cf1e0793464ab00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:17:19.288 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:18 smithi005 podman[28221]: 2024-04-22 13:17:18.876960653 +0000 UTC m=+0.018576836 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:17:19.288 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 podman[28221]: 2024-04-22 13:17:19.135923149 +0000 UTC m=+0.277539308 container init e9cd05080807eaeb3ec9f511ba4f47d354227ea9230beab6cf1e0793464ab00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:17:19.288 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 podman[28221]: 2024-04-22 13:17:19.138553601 +0000 UTC m=+0.280169765 container start e9cd05080807eaeb3ec9f511ba4f47d354227ea9230beab6cf1e0793464ab00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-22T13:17:19.386 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:17:19.386 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-22T13:17:19.392 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:17:19.392 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-22T13:17:19.392 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-22T13:17:19.392 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T13:17:19.401 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:17:19.401 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-22T13:17:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:17:19.410 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-22T13:17:19.410 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-22T13:17:19.410 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-22T13:17:19.410 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-04-22T13:17:19.410 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-04-22T13:17:19.540 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:19.289+0000 7f5a4a58f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:17:19.540 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 bash[28221]: e9cd05080807eaeb3ec9f511ba4f47d354227ea9230beab6cf1e0793464ab00e 2024-04-22T13:17:19.540 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 systemd[1]: Started Ceph mgr.x for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:17:19.540 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:19.495+0000 7f5a4a58f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:17:19.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:19.602+0000 7f5a4a58f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:17:20.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:19 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/860354660' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:17:20.231 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:19.933+0000 7f5a4a58f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:17:20.231 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:20.027+0000 7f5a4a58f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:17:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:17:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:17:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:17:20.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:17:12.208519+0000", 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:17:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.859+0000 7f762bd59700 1 Processor -- start 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.859+0000 7f762bd59700 1 -- start start 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f762bd59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f762bd59700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7624063d20 con 0x7f7624108ff0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f7629af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f7629af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50756/0 (socket says 172.21.15.5:50756) 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f7629af5700 1 -- 172.21.15.5:0/2376764439 learned_addr learned my addr 172.21.15.5:0/2376764439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f7629af5700 1 -- 172.21.15.5:0/2376764439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762410b920 con 0x7f7624108ff0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.860+0000 7f7629af5700 1 --2- 172.21.15.5:0/2376764439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762410b3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76200098a0 tx=0x7f7620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d227b2768fcfa1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f7628af3700 1 -- 172.21.15.5:0/2376764439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7620004160 con 0x7f7624108ff0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f7628af3700 1 -- 172.21.15.5:0/2376764439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7620004300 con 0x7f7624108ff0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f7628af3700 1 -- 172.21.15.5:0/2376764439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7620011af0 con 0x7f7624108ff0 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f762bd59700 1 -- 172.21.15.5:0/2376764439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 msgr2=0x7f762410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f762bd59700 1 --2- 172.21.15.5:0/2376764439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762410b3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76200098a0 tx=0x7f7620009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f762bd59700 1 -- 172.21.15.5:0/2376764439 shutdown_connections 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f762bd59700 1 --2- 172.21.15.5:0/2376764439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762410b3e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.861+0000 7f762bd59700 1 -- 172.21.15.5:0/2376764439 >> 172.21.15.5:0/2376764439 conn(0x7f7624102a40 msgr2=0x7f7624104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f762bd59700 1 -- 172.21.15.5:0/2376764439 shutdown_connections 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f762bd59700 1 -- 172.21.15.5:0/2376764439 wait complete. 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f762bd59700 1 Processor -- start 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f762bd59700 1 -- start start 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f762bd59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f762bd59700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f762410c8e0 con 0x7f7624108ff0 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.862+0000 7f7629af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7629af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50766/0 (socket says 172.21.15.5:50766) 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7629af5700 1 -- 172.21.15.5:0/860354660 learned_addr learned my addr 172.21.15.5:0/860354660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7629af5700 1 -- 172.21.15.5:0/860354660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762419c970 con 0x7f7624108ff0 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7629af5700 1 --2- 172.21.15.5:0/860354660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762419c430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7620003f40 tx=0x7f7620011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76200038b0 con 0x7f7624108ff0 2024-04-22T13:17:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7620003a50 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762419cc00 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f762001a5c0 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762419d060 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f762001ac50 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.863+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7620019700 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.864+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76240640a0 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.866+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f76200199a0 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.940+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7624106f30 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.941+0000 7f7616ffd700 1 -- 172.21.15.5:0/860354660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7620019b80 con 0x7f7624108ff0 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 msgr2=0x7f762419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 --2- 172.21.15.5:0/860354660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762419c430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7620003f40 tx=0x7f7620011890 comp rx=0 tx=0).stop 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 shutdown_connections 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 --2- 172.21.15.5:0/860354660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7624108ff0 0x7f762419c430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 >> 172.21.15.5:0/860354660 conn(0x7f7624102a40 msgr2=0x7f7624103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 shutdown_connections 2024-04-22T13:17:20.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:19.942+0000 7f762bd59700 1 -- 172.21.15.5:0/860354660 wait complete. 2024-04-22T13:17:20.520 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-04-22T13:17:21.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:21.280+0000 7f5a4a58f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:17:21.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:21.454+0000 7f5a4a58f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:17:21.938 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:21.629+0000 7f5a4a58f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:17:21.938 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:21.940+0000 7f5a4a58f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:17:22.242 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:22.035+0000 7f5a4a58f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:17:22.242 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:22.244+0000 7f5a4a58f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:17:22.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:22.449+0000 7f5a4a58f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:17:23.284 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:23.083+0000 7f5a4a58f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:17:23.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/227667341' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:17:23.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:23.542+0000 7f5a4a58f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:17:23.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:23.642+0000 7f5a4a58f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:17:23.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:23.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:17:23.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:17:23.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-22T13:17:23.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:17:23.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:17:12.208519+0000", 2024-04-22T13:17:23.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f97cf700 1 Processor -- start 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f97cf700 1 -- start start 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f97cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f4108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f97cf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4108f60 con 0x7f50f4108570 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f4108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f4108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50782/0 (socket says 172.21.15.5:50782) 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.287+0000 7f50f2ffd700 1 -- 172.21.15.5:0/2846229193 learned_addr learned my addr 172.21.15.5:0/2846229193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.288+0000 7f50f2ffd700 1 -- 172.21.15.5:0/2846229193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f4109720 con 0x7f50f4108570 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.288+0000 7f50f2ffd700 1 --2- 172.21.15.5:0/2846229193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f4108990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f50e40098a0 tx=0x7f50e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a968bf9f92fe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:23.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.288+0000 7f50f1ffb700 1 -- 172.21.15.5:0/2846229193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4004160 con 0x7f50f4108570 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.288+0000 7f50f1ffb700 1 -- 172.21.15.5:0/2846229193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50e4004300 con 0x7f50f4108570 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.288+0000 7f50f1ffb700 1 -- 172.21.15.5:0/2846229193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4011af0 con 0x7f50f4108570 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 -- 172.21.15.5:0/2846229193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 msgr2=0x7f50f4108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 --2- 172.21.15.5:0/2846229193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f4108990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f50e40098a0 tx=0x7f50e4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 -- 172.21.15.5:0/2846229193 shutdown_connections 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 --2- 172.21.15.5:0/2846229193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f4108990 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 -- 172.21.15.5:0/2846229193 >> 172.21.15.5:0/2846229193 conn(0x7f50f4103b70 msgr2=0x7f50f4105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 -- 172.21.15.5:0/2846229193 shutdown_connections 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.289+0000 7f50f97cf700 1 -- 172.21.15.5:0/2846229193 wait complete. 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f97cf700 1 Processor -- start 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f97cf700 1 -- start start 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f97cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f97cf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50f410cad0 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50784/0 (socket says 172.21.15.5:50784) 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f2ffd700 1 -- 172.21.15.5:0/227667341 learned_addr learned my addr 172.21.15.5:0/227667341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.290+0000 7f50f2ffd700 1 -- 172.21.15.5:0/227667341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f419ca10 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50f2ffd700 1 --2- 172.21.15.5:0/227667341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f419c4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50e40117b0 tx=0x7f50e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4003af0 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50e4020070 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f419cca0 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4019630 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f419f990 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f50e4019c30 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.291+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50e400ba70 con 0x7f50f4108570 2024-04-22T13:17:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.292+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f40640a0 con 0x7f50f4108570 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.294+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f50e401a9c0 con 0x7f50f4108570 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.368+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f50f419fed0 con 0x7f50f4108570 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.368+0000 7f50dbfff700 1 -- 172.21.15.5:0/227667341 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f50e400bc70 con 0x7f50f4108570 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.369+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 msgr2=0x7f50f419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.369+0000 7f50f97cf700 1 --2- 172.21.15.5:0/227667341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f419c4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50e40117b0 tx=0x7f50e4011b70 comp rx=0 tx=0).stop 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.369+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 shutdown_connections 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.369+0000 7f50f97cf700 1 --2- 172.21.15.5:0/227667341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f4108570 0x7f50f419c4d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.369+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 >> 172.21.15.5:0/227667341 conn(0x7f50f4103b70 msgr2=0x7f50f4104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.370+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 shutdown_connections 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:23.370+0000 7f50f97cf700 1 -- 172.21.15.5:0/227667341 wait complete. 2024-04-22T13:17:23.987 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-04-22T13:17:24.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:23.851+0000 7f5a4a58f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:17:24.739 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:24.397+0000 7f5a4a58f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:17:24.739 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:24.491+0000 7f5a4a58f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:17:25.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:24.741+0000 7f5a4a58f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:17:25.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:24.879+0000 7f5a4a58f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:17:25.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:24.977+0000 7f5a4a58f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:17:25.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:25 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:25.095+0000 7f5a4a58f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:17:25.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:25 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:25.406+0000 7f5a4a58f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:17:26.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:26.033+0000 7f5a4a58f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:17:26.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:26.262+0000 7f5a4a58f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:17:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: Activating manager daemon x 2024-04-22T13:17:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: mgrmap e2: x(active, starting, since 0.00309798s) 2024-04-22T13:17:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:17:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: Manager daemon x is now available 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' 2024-04-22T13:17:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:26 smithi005 ceph-mon[28006]: from='mgr.14100 172.21.15.5:0/4079643732' entity='mgr.x' 2024-04-22T13:17:27.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:27.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:17:27.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:17:27.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:17:27.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:17:27.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:17:27.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:17:27.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:17:12.208519+0000", 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb072e14700 1 Processor -- start 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb072e14700 1 -- start start 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb072e14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb072e14700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c108f50 con 0x7fb06c108560 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb070bb0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb070bb0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50862/0 (socket says 172.21.15.5:50862) 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb070bb0700 1 -- 172.21.15.5:0/3536517643 learned_addr learned my addr 172.21.15.5:0/3536517643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb070bb0700 1 -- 172.21.15.5:0/3536517643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c109710 con 0x7fb06c108560 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.412+0000 7fb070bb0700 1 --2- 172.21.15.5:0/3536517643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c108980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0098a0 tx=0x7fb05c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1158af13e53a2b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb06b7fe700 1 -- 172.21.15.5:0/3536517643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb05c004160 con 0x7fb06c108560 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb06b7fe700 1 -- 172.21.15.5:0/3536517643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb05c004300 con 0x7fb06c108560 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb06b7fe700 1 -- 172.21.15.5:0/3536517643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb05c011af0 con 0x7fb06c108560 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 -- 172.21.15.5:0/3536517643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 msgr2=0x7fb06c108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 --2- 172.21.15.5:0/3536517643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c108980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0098a0 tx=0x7fb05c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 -- 172.21.15.5:0/3536517643 shutdown_connections 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 --2- 172.21.15.5:0/3536517643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c108980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:27.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 -- 172.21.15.5:0/3536517643 >> 172.21.15.5:0/3536517643 conn(0x7fb06c103b40 msgr2=0x7fb06c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 -- 172.21.15.5:0/3536517643 shutdown_connections 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.413+0000 7fb072e14700 1 -- 172.21.15.5:0/3536517643 wait complete. 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb072e14700 1 Processor -- start 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb072e14700 1 -- start start 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb072e14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb072e14700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c10cac0 con 0x7fb06c108560 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb070bb0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb070bb0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50866/0 (socket says 172.21.15.5:50866) 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb070bb0700 1 -- 172.21.15.5:0/1762264872 learned_addr learned my addr 172.21.15.5:0/1762264872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.414+0000 7fb070bb0700 1 -- 172.21.15.5:0/1762264872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c19ca50 con 0x7fb06c108560 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.415+0000 7fb070bb0700 1 --2- 172.21.15.5:0/1762264872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c19c510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009fb0 tx=0x7fb05c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.415+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb05c0036a0 con 0x7fb06c108560 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.415+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c19cce0 con 0x7fb06c108560 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.415+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb05c01a430 con 0x7fb06c108560 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.415+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb05c019550 con 0x7fb06c108560 2024-04-22T13:17:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.415+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c19f9d0 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.416+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fb05c01a8e0 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.416+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06c0640a0 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.416+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb05c04d4e0 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.418+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb05c020240 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.493+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb06c19fba0 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.493+0000 7fb069ffb700 1 -- 172.21.15.5:0/1762264872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb06c19fba0 con 0x7fb06c108560 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 msgr2=0x7fb06c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 --2- 172.21.15.5:0/1762264872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c19c510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009fb0 tx=0x7fb05c011940 comp rx=0 tx=0).stop 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 shutdown_connections 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 --2- 172.21.15.5:0/1762264872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06c108560 0x7fb06c19c510 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 >> 172.21.15.5:0/1762264872 conn(0x7fb06c103b40 msgr2=0x7fb06c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 shutdown_connections 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:26.494+0000 7fb072e14700 1 -- 172.21.15.5:0/1762264872 wait complete. 2024-04-22T13:17:27.100 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (3/15)... 2024-04-22T13:17:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:27 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1762264872' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:17:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:27 smithi005 ceph-mon[28006]: mgrmap e3: x(active, since 1.00738s) 2024-04-22T13:17:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:29 smithi005 ceph-mon[28006]: mgrmap e4: x(active, since 2s) 2024-04-22T13:17:32.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:32 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/65361463' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:17:37.838 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:37.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:17:37.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:17:37.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:17:37.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:17:37.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:17:37.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:17:37.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:17:37.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:37.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:17:37.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:17:37.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:17:37.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:17:37.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:17:37.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:17:12.208519+0000", 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ef240700 1 Processor -- start 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ef240700 1 -- start start 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ef240700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ef240700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e807bec0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ecfdc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ecfdc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e810b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57244/0 (socket says 172.21.15.5:57244) 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.150+0000 7fa7ecfdc700 1 -- 172.21.15.5:0/3927735295 learned_addr learned my addr 172.21.15.5:0/3927735295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.151+0000 7fa7ecfdc700 1 -- 172.21.15.5:0/3927735295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e810b8e0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.151+0000 7fa7ecfdc700 1 --2- 172.21.15.5:0/3927735295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e810b3a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0098a0 tx=0x7fa7dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b14f2d80c55250e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.151+0000 7fa7e77fe700 1 -- 172.21.15.5:0/3927735295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc004160 con 0x7fa7e8108fb0 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.151+0000 7fa7e77fe700 1 -- 172.21.15.5:0/3927735295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa7dc004300 con 0x7fa7e8108fb0 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.151+0000 7fa7e77fe700 1 -- 172.21.15.5:0/3927735295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc011af0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 -- 172.21.15.5:0/3927735295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 msgr2=0x7fa7e810b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 --2- 172.21.15.5:0/3927735295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e810b3a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0098a0 tx=0x7fa7dc009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 -- 172.21.15.5:0/3927735295 shutdown_connections 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 --2- 172.21.15.5:0/3927735295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e810b3a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 -- 172.21.15.5:0/3927735295 >> 172.21.15.5:0/3927735295 conn(0x7fa7e8102a40 msgr2=0x7fa7e8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 -- 172.21.15.5:0/3927735295 shutdown_connections 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 -- 172.21.15.5:0/3927735295 wait complete. 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 Processor -- start 2024-04-22T13:17:37.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.152+0000 7fa7ef240700 1 -- start start 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ef240700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e8108830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ef240700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e810c8b0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ecfdc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e8108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ecfdc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e8108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57250/0 (socket says 172.21.15.5:57250) 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ecfdc700 1 -- 172.21.15.5:0/65361463 learned_addr learned my addr 172.21.15.5:0/65361463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ecfdc700 1 -- 172.21.15.5:0/65361463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e8108d70 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ecfdc700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e8108830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009fb0 tx=0x7fa7dc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc0036a0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e8106f60 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa7dc01a430 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7dc019550 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.153+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81074a0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.154+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa7dc019850 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.154+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e80640a0 con 0x7fa7e8108fb0 2024-04-22T13:17:37.856 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.154+0000 7fa7e5ffb700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fa7d8039510 0x7fa7d803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.154+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7dc04da30 con 0x7fa7e8108fb0 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.155+0000 7fa7e7fff700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fa7d8039510 0x7fa7d803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.155+0000 7fa7e7fff700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fa7d8039510 0x7fa7d803b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006fd0 tx=0x7fa7d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.160+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7dc04d450 con 0x7fa7e8108fb0 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.453+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa7e8107720 con 0x7fa7e8108fb0 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.454+0000 7fa7e5ffb700 1 -- 172.21.15.5:0/65361463 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa7dc003840 con 0x7fa7e8108fb0 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fa7d8039510 msgr2=0x7fa7d803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fa7d8039510 0x7fa7d803b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006fd0 tx=0x7fa7d4006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 msgr2=0x7fa7e8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e8108830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009fb0 tx=0x7fa7dc011940 comp rx=0 tx=0).stop 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 shutdown_connections 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fa7d8039510 0x7fa7d803b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 --2- 172.21.15.5:0/65361463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e8108fb0 0x7fa7e8108830 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:37.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 >> 172.21.15.5:0/65361463 conn(0x7fa7e8102a40 msgr2=0x7fa7e81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:37.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 shutdown_connections 2024-04-22T13:17:37.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:32.456+0000 7fa7ef240700 1 -- 172.21.15.5:0/65361463 wait complete. 2024-04-22T13:17:37.858 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-04-22T13:17:38.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:38 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4217317501' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T13:17:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986faa8700 1 Processor -- start 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986faa8700 1 -- start start 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986faa8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f9868108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986faa8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9868108f50 con 0x7f9868108560 2024-04-22T13:17:39.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986d844700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f9868108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986d844700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f9868108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57254/0 (socket says 172.21.15.5:57254) 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.350+0000 7f986d844700 1 -- 172.21.15.5:0/3864784645 learned_addr learned my addr 172.21.15.5:0/3864784645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.351+0000 7f986d844700 1 -- 172.21.15.5:0/3864784645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9868109710 con 0x7f9868108560 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.351+0000 7f986d844700 1 --2- 172.21.15.5:0/3864784645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f9868108980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98640098a0 tx=0x7f9864009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6325c43cfc8f445 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.351+0000 7f986c842700 1 -- 172.21.15.5:0/3864784645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9864004160 con 0x7f9868108560 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.351+0000 7f986c842700 1 -- 172.21.15.5:0/3864784645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9864004300 con 0x7f9868108560 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.351+0000 7f986c842700 1 -- 172.21.15.5:0/3864784645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9864011af0 con 0x7f9868108560 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.351+0000 7f986faa8700 1 -- 172.21.15.5:0/3864784645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 msgr2=0x7f9868108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 --2- 172.21.15.5:0/3864784645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f9868108980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98640098a0 tx=0x7f9864009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 -- 172.21.15.5:0/3864784645 shutdown_connections 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 --2- 172.21.15.5:0/3864784645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f9868108980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 -- 172.21.15.5:0/3864784645 >> 172.21.15.5:0/3864784645 conn(0x7f9868103b40 msgr2=0x7f9868105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 -- 172.21.15.5:0/3864784645 shutdown_connections 2024-04-22T13:17:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 -- 172.21.15.5:0/3864784645 wait complete. 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 Processor -- start 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.352+0000 7f986faa8700 1 -- start start 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986faa8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f986819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986faa8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f986810cac0 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986d844700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f986819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986d844700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f986819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57256/0 (socket says 172.21.15.5:57256) 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986d844700 1 -- 172.21.15.5:0/4217317501 learned_addr learned my addr 172.21.15.5:0/4217317501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986d844700 1 -- 172.21.15.5:0/4217317501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819ca50 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986d844700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f986819c510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98640117b0 tx=0x7f9864011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9864003af0 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986819cce0 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9864020070 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986819f9d0 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.353+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9864019630 con 0x7f9868108560 2024-04-22T13:17:39.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.354+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f98640198f0 con 0x7f9868108560 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.355+0000 7f985effd700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f9854039560 0x7f985403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.355+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f986404db20 con 0x7f9868108560 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.355+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98680640a0 con 0x7f9868108560 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.355+0000 7f986d043700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f9854039560 0x7f985403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.355+0000 7f986d043700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f9854039560 0x7f985403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9858006fd0 tx=0x7f9858006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.361+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f986400e260 con 0x7f9868108560 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.567+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f98681a10a0 con 0x7f9868108560 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.568+0000 7f985effd700 1 -- 172.21.15.5:0/4217317501 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f986404d540 con 0x7f9868108560 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f9854039560 msgr2=0x7f985403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f9854039560 0x7f985403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9858006fd0 tx=0x7f9858006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 msgr2=0x7f986819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f986819c510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98640117b0 tx=0x7f9864011b70 comp rx=0 tx=0).stop 2024-04-22T13:17:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 shutdown_connections 2024-04-22T13:17:39.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f9854039560 0x7f985403ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:39.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 --2- 172.21.15.5:0/4217317501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9868108560 0x7f986819c510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:39.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 >> 172.21.15.5:0/4217317501 conn(0x7f9868103b40 msgr2=0x7f98681046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:39.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 shutdown_connections 2024-04-22T13:17:39.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:38.570+0000 7f986faa8700 1 -- 172.21.15.5:0/4217317501 wait complete. 2024-04-22T13:17:39.056 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-04-22T13:17:40.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:39 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4070976451' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-22T13:17:40.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: ignoring --setuser ceph since I am not root 2024-04-22T13:17:40.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: ignoring --setgroup ceph since I am not root 2024-04-22T13:17:40.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:40.014+0000 7f28386d5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:17:40.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:40.206+0000 7f28386d5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:17:40.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:40.312+0000 7f28386d5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:17:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4070976451' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-22T13:17:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-mon[28006]: mgrmap e5: x(active, since 13s) 2024-04-22T13:17:40.924 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:40.633+0000 7f28386d5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:17:40.924 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:40.722+0000 7f28386d5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:17:40.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb8769f5700 1 Processor -- start 2024-04-22T13:17:40.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb8769f5700 1 -- start start 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb8769f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb8769f5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb87007c900 con 0x7fb87007de80 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb86ffff700 1 -- 172.21.15.5:0/2763753379 learned_addr learned my addr 172.21.15.5:0/2763753379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.511+0000 7fb86ffff700 1 -- 172.21.15.5:0/2763753379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87007ca80 con 0x7fb87007de80 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb86ffff700 1 --2- 172.21.15.5:0/2763753379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87007c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8600098a0 tx=0x7fb860009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a15d99298a93508 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb86effd700 1 -- 172.21.15.5:0/2763753379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb860004160 con 0x7fb87007de80 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb86effd700 1 -- 172.21.15.5:0/2763753379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb860004300 con 0x7fb87007de80 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb86effd700 1 -- 172.21.15.5:0/2763753379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb860011af0 con 0x7fb87007de80 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 -- 172.21.15.5:0/2763753379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 msgr2=0x7fb87007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 --2- 172.21.15.5:0/2763753379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87007c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8600098a0 tx=0x7fb860009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 -- 172.21.15.5:0/2763753379 shutdown_connections 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 --2- 172.21.15.5:0/2763753379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87007c330 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 -- 172.21.15.5:0/2763753379 >> 172.21.15.5:0/2763753379 conn(0x7fb870103b40 msgr2=0x7fb870105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 -- 172.21.15.5:0/2763753379 shutdown_connections 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.512+0000 7fb8769f5700 1 -- 172.21.15.5:0/2763753379 wait complete. 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.513+0000 7fb8769f5700 1 Processor -- start 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.513+0000 7fb8769f5700 1 -- start start 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb8769f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb8769f5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb87010ea90 con 0x7fb87007de80 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58844/0 (socket says 172.21.15.5:58844) 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86ffff700 1 -- 172.21.15.5:0/4070976451 learned_addr learned my addr 172.21.15.5:0/4070976451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86ffff700 1 -- 172.21.15.5:0/4070976451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87019ec00 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86ffff700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87019e6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb860011ac0 tx=0x7fb860011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb860003970 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb860003b10 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87019ee90 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb860024610 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.514+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701a1b80 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.515+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fb860024890 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.515+0000 7fb86d7fa700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fb858039560 0x7fb85803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.515+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb86004d8c0 con 0x7fb87007de80 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.516+0000 7fb86f7fe700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fb858039560 0x7fb85803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:40.934 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.516+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8700640a0 con 0x7fb87007de80 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.516+0000 7fb86f7fe700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fb858039560 0x7fb85803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb864006fd0 tx=0x7fb864006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.521+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8600172b0 con 0x7fb87007de80 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.790+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb8701a2130 con 0x7fb87007de80 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.842+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb86004d2e0 con 0x7fb87007de80 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.843+0000 7fb86d7fa700 1 -- 172.21.15.5:0/4070976451 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb86002d960 con 0x7fb87007de80 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.844+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fb858039560 msgr2=0x7fb85803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.844+0000 7fb8769f5700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fb858039560 0x7fb85803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb864006fd0 tx=0x7fb864006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 msgr2=0x7fb87019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87019e6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb860011ac0 tx=0x7fb860011ba0 comp rx=0 tx=0).stop 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 shutdown_connections 2024-04-22T13:17:40.935 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7fb858039560 0x7fb85803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:40.936 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 --2- 172.21.15.5:0/4070976451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb87007de80 0x7fb87019e6c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:40.936 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 >> 172.21.15.5:0/4070976451 conn(0x7fb870103b40 msgr2=0x7fb870104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:40.936 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 shutdown_connections 2024-04-22T13:17:40.936 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:39.845+0000 7fb8769f5700 1 -- 172.21.15.5:0/4070976451 wait complete. 2024-04-22T13:17:42.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:41 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/467499592' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T13:17:42.309 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:42.008+0000 7f28386d5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:17:42.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:42.190+0000 7f28386d5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:17:42.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b33de0700 1 Processor -- start 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b33de0700 1 -- start start 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b33de0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b33de0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c107e30 con 0x7f8b2c107440 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b31b7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b31b7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58866/0 (socket says 172.21.15.5:58866) 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b31b7c700 1 -- 172.21.15.5:0/116400633 learned_addr learned my addr 172.21.15.5:0/116400633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:42.414 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.397+0000 7f8b31b7c700 1 -- 172.21.15.5:0/116400633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c108640 con 0x7f8b2c107440 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.398+0000 7f8b31b7c700 1 --2- 172.21.15.5:0/116400633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c107860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098a0 tx=0x7f8b28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d66658aa5903d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.398+0000 7f8b30b7a700 1 -- 172.21.15.5:0/116400633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b28004160 con 0x7f8b2c107440 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.398+0000 7f8b30b7a700 1 -- 172.21.15.5:0/116400633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8b28004300 con 0x7f8b2c107440 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.398+0000 7f8b30b7a700 1 -- 172.21.15.5:0/116400633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b28011af0 con 0x7f8b2c107440 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.398+0000 7f8b33de0700 1 -- 172.21.15.5:0/116400633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 msgr2=0x7f8b2c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.398+0000 7f8b33de0700 1 --2- 172.21.15.5:0/116400633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c107860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098a0 tx=0x7f8b28009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 -- 172.21.15.5:0/116400633 shutdown_connections 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 --2- 172.21.15.5:0/116400633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c107860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 -- 172.21.15.5:0/116400633 >> 172.21.15.5:0/116400633 conn(0x7f8b2c102a40 msgr2=0x7f8b2c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 -- 172.21.15.5:0/116400633 shutdown_connections 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 -- 172.21.15.5:0/116400633 wait complete. 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 Processor -- start 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 -- start start 2024-04-22T13:17:42.415 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c1925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.399+0000 7f8b33de0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c10b9f0 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b31b7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c1925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b31b7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c1925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58872/0 (socket says 172.21.15.5:58872) 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b31b7c700 1 -- 172.21.15.5:0/467499592 learned_addr learned my addr 172.21.15.5:0/467499592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b31b7c700 1 -- 172.21.15.5:0/467499592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c192af0 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b31b7c700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c1925b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8b28003f40 tx=0x7f8b28011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b280038b0 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8b28003a50 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c190ce0 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b2801a5c0 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.400+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c191220 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.401+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8b2801a870 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.401+0000 7f8b22ffd700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 0x7f8b1803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.401+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b2804da60 con 0x7f8b2c107440 2024-04-22T13:17:42.416 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.401+0000 7f8b3137b700 1 -- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 msgr2=0x7f8b1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.401+0000 7f8b3137b700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 0x7f8b1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.402+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c0640a0 con 0x7f8b2c107440 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.407+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b28019c70 con 0x7f8b2c107440 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.602+0000 7f8b3137b700 1 -- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 msgr2=0x7f8b1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.602+0000 7f8b3137b700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 0x7f8b1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.696+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8b2c1962e0 con 0x7f8b2c107440 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.696+0000 7f8b22ffd700 1 -- 172.21.15.5:0/467499592 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f8b2804d480 con 0x7f8b2c107440 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 msgr2=0x7f8b1803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 0x7f8b1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 msgr2=0x7f8b2c1925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c1925b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8b28003f40 tx=0x7f8b28011890 comp rx=0 tx=0).stop 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 shutdown_connections 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f8b180395b0 0x7f8b1803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 --2- 172.21.15.5:0/467499592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107440 0x7f8b2c1925b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:42.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.698+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 >> 172.21.15.5:0/467499592 conn(0x7f8b2c102a40 msgr2=0x7f8b2c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:42.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.699+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 shutdown_connections 2024-04-22T13:17:42.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:41.699+0000 7f8b33de0700 1 -- 172.21.15.5:0/467499592 wait complete. 2024-04-22T13:17:42.418 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-04-22T13:17:42.418 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-04-22T13:17:42.629 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:42.367+0000 7f28386d5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:17:42.986 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:42.677+0000 7f28386d5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:17:42.987 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:42.776+0000 7f28386d5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:17:42.987 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:42.987+0000 7f28386d5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:17:43.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:43.188+0000 7f28386d5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:17:44.226 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:43.813+0000 7f28386d5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:17:44.558 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:44 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:44.251+0000 7f28386d5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:17:44.558 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:44 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:44.351+0000 7f28386d5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:17:44.558 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:44 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:44.559+0000 7f28386d5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:17:45.450 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:45 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:45.103+0000 7f28386d5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:17:45.450 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:45 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:45.198+0000 7f28386d5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:17:45.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:45 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:45.451+0000 7f28386d5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:17:45.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:45 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:45.590+0000 7f28386d5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:17:45.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:45 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:45.688+0000 7f28386d5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:17:45.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:45 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:45.807+0000 7f28386d5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:17:46.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:46 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:46.119+0000 7f28386d5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:17:47.023 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:46 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:46.747+0000 7f28386d5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:17:47.023 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:46 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:17:46.971+0000 7f28386d5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: Active manager daemon x restarted 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: Activating manager daemon x 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: osdmap e2: 0 total, 0 up, 0 in 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: mgrmap e6: x(active, starting, since 0.00337827s) 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:17:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: Manager daemon x is now available 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:47 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:17:48.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:48 smithi005 ceph-mon[28006]: Found migration_current of "None". Setting to last migration. 2024-04-22T13:17:48.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:48 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:17:48.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:48 smithi005 ceph-mon[28006]: mgrmap e7: x(active, since 1.00592s) 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.820+0000 7f093882a700 1 Processor -- start 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.820+0000 7f093882a700 1 -- start start 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.820+0000 7f093882a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093882a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0934063d20 con 0x7f0934108ff0 2024-04-22T13:17:48.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58874/0 (socket says 172.21.15.5:58874) 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093259c700 1 -- 172.21.15.5:0/1994580001 learned_addr learned my addr 172.21.15.5:0/1994580001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093259c700 1 -- 172.21.15.5:0/1994580001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093410b920 con 0x7f0934108ff0 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093259c700 1 --2- 172.21.15.5:0/1994580001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093410b3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f09240098a0 tx=0x7f0924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a09d2304d6af01 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093159a700 1 -- 172.21.15.5:0/1994580001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0924004160 con 0x7f0934108ff0 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093159a700 1 -- 172.21.15.5:0/1994580001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0924004300 con 0x7f0934108ff0 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.821+0000 7f093159a700 1 -- 172.21.15.5:0/1994580001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0924011af0 con 0x7f0934108ff0 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 -- 172.21.15.5:0/1994580001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 msgr2=0x7f093410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 --2- 172.21.15.5:0/1994580001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093410b3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f09240098a0 tx=0x7f0924009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 -- 172.21.15.5:0/1994580001 shutdown_connections 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 --2- 172.21.15.5:0/1994580001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093410b3e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 -- 172.21.15.5:0/1994580001 >> 172.21.15.5:0/1994580001 conn(0x7f0934102a40 msgr2=0x7f0934104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 -- 172.21.15.5:0/1994580001 shutdown_connections 2024-04-22T13:17:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.822+0000 7f093882a700 1 -- 172.21.15.5:0/1994580001 wait complete. 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093882a700 1 Processor -- start 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093882a700 1 -- start start 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093882a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093882a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f093410c8e0 con 0x7f0934108ff0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58884/0 (socket says 172.21.15.5:58884) 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.823+0000 7f093259c700 1 -- 172.21.15.5:0/69919354 learned_addr learned my addr 172.21.15.5:0/69919354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f093259c700 1 -- 172.21.15.5:0/69919354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093419c990 con 0x7f0934108ff0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f093259c700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093419c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0924011ac0 tx=0x7f0924011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0924003b20 con 0x7f0934108ff0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093419cc20 con 0x7f0934108ff0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f092401a8c0 con 0x7f0934108ff0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09240248d0 con 0x7f0934108ff0 2024-04-22T13:17:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.824+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093419f910 con 0x7f0934108ff0 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.825+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0924022020 con 0x7f0934108ff0 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.825+0000 7f092b7fe700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.825+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 --> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f092003c1d0 con 0x7f0920039600 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.825+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f092401a430 con 0x7f0934108ff0 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.825+0000 7f0931d9b700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 msgr2=0x7f092003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:42.825+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:43.026+0000 7f0931d9b700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 msgr2=0x7f092003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:43.026+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:43.427+0000 7f0931d9b700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 msgr2=0x7f092003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:43.427+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:44.228+0000 7f0931d9b700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 msgr2=0x7f092003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:44.228+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:45.829+0000 7f0931d9b700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 msgr2=0x7f092003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3443029620 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:45.829+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:46.975+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f092402d3f0 con 0x7f0934108ff0 2024-04-22T13:17:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:46.975+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 msgr2=0x7f092003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:46.975+0000 7f092b7fe700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.976+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f092404f080 con 0x7f0934108ff0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.976+0000 7f092b7fe700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f092003cef0 0x7f092003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.976+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0924020ec0 con 0x7f092003cef0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.977+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f092003cef0 0x7f092003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.978+0000 7f0931d9b700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f092003cef0 0x7f092003f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f091c003a10 tx=0x7f091c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.978+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0924020ec0 con 0x7f092003cef0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.984+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0934069ba0 con 0x7f092003cef0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f092b7fe700 1 -- 172.21.15.5:0/69919354 <== mgr.14120 v2:172.21.15.5:6800/4095818300 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0934069ba0 con 0x7f092003cef0 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f092003cef0 msgr2=0x7f092003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f092003cef0 0x7f092003f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f091c003a10 tx=0x7f091c0092b0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 msgr2=0x7f093419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:48.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093419c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0924011ac0 tx=0x7f0924011ba0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 shutdown_connections 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f092003cef0 0x7f092003f2e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0934108ff0 0x7f093419c450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 --2- 172.21.15.5:0/69919354 >> [v2:172.21.15.5:6800/3443029620,v1:172.21.15.5:6801/3443029620] conn(0x7f0920039600 0x7f092003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 >> 172.21.15.5:0/69919354 conn(0x7f0934102a40 msgr2=0x7f0934103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 shutdown_connections 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:47.985+0000 7f093882a700 1 -- 172.21.15.5:0/69919354 wait complete. 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-04-22T13:17:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: [22/Apr/2024:13:17:48] ENGINE Bus STARTING 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: [22/Apr/2024:13:17:48] ENGINE Serving on https://172.21.15.5:7150 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: [22/Apr/2024:13:17:48] ENGINE Serving on http://172.21.15.5:8765 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: [22/Apr/2024:13:17:48] ENGINE Bus STARTED 2024-04-22T13:17:49.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:49 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:49.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.986+0000 7f42f229c700 1 Processor -- start 2024-04-22T13:17:49.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.986+0000 7f42f229c700 1 -- start start 2024-04-22T13:17:49.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.986+0000 7f42f229c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec10b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.986+0000 7f42f229c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec07be40 con 0x7f42ec109040 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec10b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37098/0 (socket says 172.21.15.5:37098) 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42ebfff700 1 -- 172.21.15.5:0/102641040 learned_addr learned my addr 172.21.15.5:0/102641040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42ebfff700 1 -- 172.21.15.5:0/102641040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec10b970 con 0x7f42ec109040 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42ebfff700 1 --2- 172.21.15.5:0/102641040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec10b430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0098a0 tx=0x7f42dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ae0e1ef38dc32e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42eaffd700 1 -- 172.21.15.5:0/102641040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc004160 con 0x7f42ec109040 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42eaffd700 1 -- 172.21.15.5:0/102641040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42dc004300 con 0x7f42ec109040 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.987+0000 7f42eaffd700 1 -- 172.21.15.5:0/102641040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc011af0 con 0x7f42ec109040 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 -- 172.21.15.5:0/102641040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 msgr2=0x7f42ec10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 --2- 172.21.15.5:0/102641040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec10b430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0098a0 tx=0x7f42dc009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 -- 172.21.15.5:0/102641040 shutdown_connections 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 --2- 172.21.15.5:0/102641040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec10b430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 -- 172.21.15.5:0/102641040 >> 172.21.15.5:0/102641040 conn(0x7f42ec102a50 msgr2=0x7f42ec104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 -- 172.21.15.5:0/102641040 shutdown_connections 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 -- 172.21.15.5:0/102641040 wait complete. 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 Processor -- start 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.988+0000 7f42f229c700 1 -- start start 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42f229c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42f229c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec10c8d0 con 0x7f42ec109040 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37112/0 (socket says 172.21.15.5:37112) 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42ebfff700 1 -- 172.21.15.5:0/2789979390 learned_addr learned my addr 172.21.15.5:0/2789979390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42ebfff700 1 -- 172.21.15.5:0/2789979390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec19c9d0 con 0x7f42ec109040 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42ebfff700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec19c490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f42dc011ac0 tx=0x7f42dc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.989+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc003970 con 0x7f42ec109040 2024-04-22T13:17:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.990+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42dc003b10 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.990+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec19cc60 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.990+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc024610 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.990+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec19f950 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.990+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f42dc024840 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.991+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ec0640a0 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.991+0000 7f42e97fa700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f42d40394e0 0x7f42d403b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.991+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f42dc04d710 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.991+0000 7f42eb7fe700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f42d40394e0 0x7f42d403b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.991+0000 7f42eb7fe700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f42d40394e0 0x7f42d403b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006fd0 tx=0x7f42e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:48.997+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42dc0172b0 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.100+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f42dc02d910 con 0x7f42ec109040 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.225+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f42ec103c50 con 0x7f42d40394e0 2024-04-22T13:17:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.228+0000 7f42e97fa700 1 -- 172.21.15.5:0/2789979390 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42ec103c50 con 0x7f42d40394e0 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f42d40394e0 msgr2=0x7f42d403b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f42d40394e0 0x7f42d403b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006fd0 tx=0x7f42e0006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 msgr2=0x7f42ec19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec19c490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f42dc011ac0 tx=0x7f42dc011ba0 comp rx=0 tx=0).stop 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 shutdown_connections 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f42d40394e0 0x7f42d403b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 --2- 172.21.15.5:0/2789979390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ec109040 0x7f42ec19c490 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 >> 172.21.15.5:0/2789979390 conn(0x7f42ec102a50 msgr2=0x7f42ec103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 shutdown_connections 2024-04-22T13:17:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:49.232+0000 7f42f229c700 1 -- 172.21.15.5:0/2789979390 wait complete. 2024-04-22T13:17:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:50 smithi005 ceph-mon[28006]: mgrmap e8: x(active, since 2s) 2024-04-22T13:17:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:50 smithi005 ceph-mon[28006]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:50 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:50 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f1a0596d700 1 Processor -- start 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f1a0596d700 1 -- start start 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f1a0596d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a00108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f1a0596d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00108f50 con 0x7f1a00108560 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f19feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a00108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f19feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a00108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37128/0 (socket says 172.21.15.5:37128) 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f19feffd700 1 -- 172.21.15.5:0/2904452034 learned_addr learned my addr 172.21.15.5:0/2904452034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.185+0000 7f19feffd700 1 -- 172.21.15.5:0/2904452034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a00109710 con 0x7f1a00108560 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f19feffd700 1 --2- 172.21.15.5:0/2904452034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a00108980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098a0 tx=0x7f19f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e611578d627da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f19fdffb700 1 -- 172.21.15.5:0/2904452034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0004160 con 0x7f1a00108560 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f19fdffb700 1 -- 172.21.15.5:0/2904452034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19f0004300 con 0x7f1a00108560 2024-04-22T13:17:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f19fdffb700 1 -- 172.21.15.5:0/2904452034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0011af0 con 0x7f1a00108560 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 -- 172.21.15.5:0/2904452034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 msgr2=0x7f1a00108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 --2- 172.21.15.5:0/2904452034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a00108980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098a0 tx=0x7f19f0009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 -- 172.21.15.5:0/2904452034 shutdown_connections 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 --2- 172.21.15.5:0/2904452034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a00108980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 -- 172.21.15.5:0/2904452034 >> 172.21.15.5:0/2904452034 conn(0x7f1a00103b40 msgr2=0x7f1a00105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 -- 172.21.15.5:0/2904452034 shutdown_connections 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.186+0000 7f1a0596d700 1 -- 172.21.15.5:0/2904452034 wait complete. 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.187+0000 7f1a0596d700 1 Processor -- start 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.187+0000 7f1a0596d700 1 -- start start 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.187+0000 7f1a0596d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a0019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.187+0000 7f1a0596d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0010cac0 con 0x7f1a00108560 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f19feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a0019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f19feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a0019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37140/0 (socket says 172.21.15.5:37140) 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f19feffd700 1 -- 172.21.15.5:0/1945076932 learned_addr learned my addr 172.21.15.5:0/1945076932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:50.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f19feffd700 1 -- 172.21.15.5:0/1945076932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0019ca30 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f19feffd700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a0019c4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19f00117b0 tx=0x7f19f0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0003af0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19f0020070 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0019630 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0019ccc0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.188+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0019f9b0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.189+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f19f00198f0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.189+0000 7f1a0496b700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f19e80395b0 0x7f19e803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.189+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f19f004dba0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.189+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a00069ba0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.189+0000 7f19fe7fc700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f19e80395b0 0x7f19e803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.190+0000 7f19fe7fc700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f19e80395b0 0x7f19e803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006fd0 tx=0x7f19f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.195+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19f00042a0 con 0x7f1a00108560 2024-04-22T13:17:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.404+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1a00104e70 con 0x7f19e80395b0 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.405+0000 7f1a0496b700 1 -- 172.21.15.5:0/1945076932 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1a00104e70 con 0x7f19e80395b0 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f19e80395b0 msgr2=0x7f19e803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f19e80395b0 0x7f19e803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006fd0 tx=0x7f19f4006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 msgr2=0x7f1a0019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a0019c4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19f00117b0 tx=0x7f19f0011b70 comp rx=0 tx=0).stop 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 shutdown_connections 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f19e80395b0 0x7f19e803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 --2- 172.21.15.5:0/1945076932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a00108560 0x7f1a0019c4f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 >> 172.21.15.5:0/1945076932 conn(0x7f1a00103b40 msgr2=0x7f1a00104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 shutdown_connections 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:50.407+0000 7f1a0596d700 1 -- 172.21.15.5:0/1945076932 wait complete. 2024-04-22T13:17:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-04-22T13:17:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-mon[28006]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:52.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: Generating public/private rsa key pair. 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: Your identification has been saved in /tmp/tmpxdv_0mmc/key. 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: Your public key has been saved in /tmp/tmpxdv_0mmc/key.pub. 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: The key fingerprint is: 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: SHA256:c2WcMbhtm7k2BImEzXqjy+lYlL+51PmG5UC/7ujc/pU ceph-73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: The key's randomart image is: 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: +---[RSA 3072]----+ 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | + .o | 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | . + .. + | 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | o . += | 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | ..+ =oo | 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | ooSo.+ + | 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | ... +..B .| 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | ..oo o* o E.| 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | o+. +.+B . | 2024-04-22T13:17:52.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: | ... +o+**o. | 2024-04-22T13:17:52.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:17:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: +----[SHA256]-----+ 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.421+0000 7f0facecf700 1 Processor -- start 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0facecf700 1 -- start start 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0facecf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0facecf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa807c8e0 con 0x7f0fa807de60 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37148/0 (socket says 172.21.15.5:37148) 2024-04-22T13:17:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0fa659c700 1 -- 172.21.15.5:0/1404268825 learned_addr learned my addr 172.21.15.5:0/1404268825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:52.379 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.422+0000 7f0fa659c700 1 -- 172.21.15.5:0/1404268825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa807ca60 con 0x7f0fa807de60 2024-04-22T13:17:52.379 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0fa659c700 1 --2- 172.21.15.5:0/1404268825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa807c310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009e80 tx=0x7f0fa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fcb9920e04a897 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:52.379 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0fa559a700 1 -- 172.21.15.5:0/1404268825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0017070 con 0x7f0fa807de60 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0fa559a700 1 -- 172.21.15.5:0/1404268825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0fa000b8c0 con 0x7f0fa807de60 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0fa559a700 1 -- 172.21.15.5:0/1404268825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0004490 con 0x7f0fa807de60 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0facecf700 1 -- 172.21.15.5:0/1404268825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 msgr2=0x7f0fa807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0facecf700 1 --2- 172.21.15.5:0/1404268825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa807c310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009e80 tx=0x7f0fa000b330 comp rx=0 tx=0).stop 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0facecf700 1 -- 172.21.15.5:0/1404268825 shutdown_connections 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0facecf700 1 --2- 172.21.15.5:0/1404268825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa807c310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.383 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.423+0000 7f0facecf700 1 -- 172.21.15.5:0/1404268825 >> 172.21.15.5:0/1404268825 conn(0x7f0fa8103b40 msgr2=0x7f0fa8105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.424+0000 7f0facecf700 1 -- 172.21.15.5:0/1404268825 shutdown_connections 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.424+0000 7f0facecf700 1 -- 172.21.15.5:0/1404268825 wait complete. 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.424+0000 7f0facecf700 1 Processor -- start 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.424+0000 7f0facecf700 1 -- start start 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0facecf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0facecf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa810eab0 con 0x7f0fa807de60 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37152/0 (socket says 172.21.15.5:37152) 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0fa659c700 1 -- 172.21.15.5:0/4039376378 learned_addr learned my addr 172.21.15.5:0/4039376378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0fa659c700 1 -- 172.21.15.5:0/4039376378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa819ebb0 con 0x7f0fa807de60 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0fa659c700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa819e670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000b7b0 tx=0x7f0fa000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0020020 con 0x7f0fa807de60 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0fa0003d10 con 0x7f0fa807de60 2024-04-22T13:17:52.384 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa819ee40 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0017040 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.425+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81a1b30 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.426+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0fa0023020 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.426+0000 7f0f9f7fe700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f0f940395b0 0x7f0f9403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.426+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0fa000f050 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.426+0000 7f0fa5d9b700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f0f940395b0 0x7f0f9403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.426+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa80640a0 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.427+0000 7f0fa5d9b700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f0f940395b0 0x7f0f9403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f90006fd0 tx=0x7f0f90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.432+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fa002a020 con 0x7f0fa807de60 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.641+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0fa8104dc0 con 0x7f0f940395b0 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.823+0000 7f0f9f7fe700 1 -- 172.21.15.5:0/4039376378 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0fa8104dc0 con 0x7f0f940395b0 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f0f940395b0 msgr2=0x7f0f9403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f0f940395b0 0x7f0f9403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f90006fd0 tx=0x7f0f90006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 msgr2=0x7f0fa819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:52.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa819e670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000b7b0 tx=0x7f0fa000be90 comp rx=0 tx=0).stop 2024-04-22T13:17:52.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 shutdown_connections 2024-04-22T13:17:52.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f0f940395b0 0x7f0f9403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 --2- 172.21.15.5:0/4039376378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa807de60 0x7f0fa819e670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 >> 172.21.15.5:0/4039376378 conn(0x7f0fa8103b40 msgr2=0x7f0fa81046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:52.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 shutdown_connections 2024-04-22T13:17:52.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:51.825+0000 7f0facecf700 1 -- 172.21.15.5:0/4039376378 wait complete. 2024-04-22T13:17:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:52 smithi005 ceph-mon[28006]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:52 smithi005 ceph-mon[28006]: Generating ssh key... 2024-04-22T13:17:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:52 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:52 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.774+0000 7f8c085e1700 1 Processor -- start 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.774+0000 7f8c085e1700 1 -- start start 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.775+0000 7f8c085e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.775+0000 7f8c085e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0007c900 con 0x7f8c0007de80 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.775+0000 7f8c0637d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.775+0000 7f8c0637d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37160/0 (socket says 172.21.15.5:37160) 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.775+0000 7f8c0637d700 1 -- 172.21.15.5:0/2307310894 learned_addr learned my addr 172.21.15.5:0/2307310894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:53.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.775+0000 7f8c0637d700 1 -- 172.21.15.5:0/2307310894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0007ca80 con 0x7f8c0007de80 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.776+0000 7f8c0637d700 1 --2- 172.21.15.5:0/2307310894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0007c330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098a0 tx=0x7f8bf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcfba3b5659ac79 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.776+0000 7f8c0537b700 1 -- 172.21.15.5:0/2307310894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf4004160 con 0x7f8c0007de80 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.776+0000 7f8c0537b700 1 -- 172.21.15.5:0/2307310894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8bf4004300 con 0x7f8c0007de80 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.776+0000 7f8c0537b700 1 -- 172.21.15.5:0/2307310894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf4011af0 con 0x7f8c0007de80 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.776+0000 7f8c085e1700 1 -- 172.21.15.5:0/2307310894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 msgr2=0x7f8c0007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.776+0000 7f8c085e1700 1 --2- 172.21.15.5:0/2307310894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0007c330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098a0 tx=0x7f8bf4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 -- 172.21.15.5:0/2307310894 shutdown_connections 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 --2- 172.21.15.5:0/2307310894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0007c330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 -- 172.21.15.5:0/2307310894 >> 172.21.15.5:0/2307310894 conn(0x7f8c00103b40 msgr2=0x7f8c00105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 -- 172.21.15.5:0/2307310894 shutdown_connections 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 -- 172.21.15.5:0/2307310894 wait complete. 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 Processor -- start 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 -- start start 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:53.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.777+0000 7f8c085e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0010ea90 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c0637d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c0637d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37176/0 (socket says 172.21.15.5:37176) 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c0637d700 1 -- 172.21.15.5:0/2573038022 learned_addr learned my addr 172.21.15.5:0/2573038022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c0637d700 1 -- 172.21.15.5:0/2573038022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0019ec10 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c0637d700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0019e6d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fb0 tx=0x7f8bf4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf40036a0 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8bf401a430 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0019eea0 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf4019550 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.778+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c001a1b90 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.779+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8bf401a8e0 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.779+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c000640a0 con 0x7f8c0007de80 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.780+0000 7f8bf37fe700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f8bec0395b0 0x7f8bec03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:53.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.780+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8bf404d8d0 con 0x7f8c0007de80 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.780+0000 7f8c05b7c700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f8bec0395b0 0x7f8bec03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.780+0000 7f8c05b7c700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f8bec0395b0 0x7f8bec03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc006fd0 tx=0x7f8bfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.785+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bf400b3e0 con 0x7f8c0007de80 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.996+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8c00104e20 con 0x7f8bec0395b0 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.997+0000 7f8bf37fe700 1 -- 172.21.15.5:0/2573038022 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8c00104e20 con 0x7f8bec0395b0 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f8bec0395b0 msgr2=0x7f8bec03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f8bec0395b0 0x7f8bec03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc006fd0 tx=0x7f8bfc006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 msgr2=0x7f8c0019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0019e6d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fb0 tx=0x7f8bf4011940 comp rx=0 tx=0).stop 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 shutdown_connections 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f8bec0395b0 0x7f8bec03ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 --2- 172.21.15.5:0/2573038022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0007de80 0x7f8c0019e6d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 >> 172.21.15.5:0/2573038022 conn(0x7f8c00103b40 msgr2=0x7f8c00104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 shutdown_connections 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:52.999+0000 7f8c085e1700 1 -- 172.21.15.5:0/2573038022 wait complete. 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:17:53.551 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-04-22T13:17:53.552 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-04-22T13:17:54.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:53 smithi005 ceph-mon[28006]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:55.051 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:54 smithi005 ceph-mon[28006]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:55.051 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:54 smithi005 ceph-mon[28006]: Deploying cephadm binary to smithi005 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f7964648700 1 Processor -- start 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f7964648700 1 -- start start 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f7964648700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f7964648700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f795c07c8e0 con 0x7f795c07de60 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f79623e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:58.995 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f79623e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37182/0 (socket says 172.21.15.5:37182) 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f79623e4700 1 -- 172.21.15.5:0/1577743171 learned_addr learned my addr 172.21.15.5:0/1577743171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.940+0000 7f79623e4700 1 -- 172.21.15.5:0/1577743171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c07ca60 con 0x7f795c07de60 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f79623e4700 1 --2- 172.21.15.5:0/1577743171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c07c310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f79580098a0 tx=0x7f7958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a558baafb3307c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f79613e2700 1 -- 172.21.15.5:0/1577743171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958004160 con 0x7f795c07de60 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f79613e2700 1 -- 172.21.15.5:0/1577743171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7958004300 con 0x7f795c07de60 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f79613e2700 1 -- 172.21.15.5:0/1577743171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958011af0 con 0x7f795c07de60 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f7964648700 1 -- 172.21.15.5:0/1577743171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 msgr2=0x7f795c07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f7964648700 1 --2- 172.21.15.5:0/1577743171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c07c310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f79580098a0 tx=0x7f7958009bb0 comp rx=0 tx=0).stop 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f7964648700 1 -- 172.21.15.5:0/1577743171 shutdown_connections 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f7964648700 1 --2- 172.21.15.5:0/1577743171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c07c310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f7964648700 1 -- 172.21.15.5:0/1577743171 >> 172.21.15.5:0/1577743171 conn(0x7f795c103b40 msgr2=0x7f795c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.941+0000 7f7964648700 1 -- 172.21.15.5:0/1577743171 shutdown_connections 2024-04-22T13:17:58.996 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f7964648700 1 -- 172.21.15.5:0/1577743171 wait complete. 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f7964648700 1 Processor -- start 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f7964648700 1 -- start start 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f7964648700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f7964648700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10eab0 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f79623e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f79623e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.942+0000 7f79623e4700 1 -- 172.21.15.5:0/1568497514 learned_addr learned my addr 172.21.15.5:0/1568497514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f79623e4700 1 -- 172.21.15.5:0/1568497514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c19ec00 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f79623e4700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c19e6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7958009fb0 tx=0x7f7958011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79580036a0 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c19ee90 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f795801a430 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958019550 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.943+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1a1b80 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.944+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f795801a8e0 con 0x7f795c07de60 2024-04-22T13:17:58.997 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.944+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c0640a0 con 0x7f795c07de60 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.944+0000 7f79537fe700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f7948039560 0x7f794803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.945+0000 7f7961be3700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f7948039560 0x7f794803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.945+0000 7f7961be3700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f7948039560 0x7f794803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f794c006fd0 tx=0x7f794c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.946+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f795800fda0 con 0x7f795c07de60 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:53.951+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f795800b3e0 con 0x7f795c07de60 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:54.166+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7f795c104e50 con 0x7f7948039560 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.436+0000 7f79537fe700 1 -- 172.21.15.5:0/1568497514 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f795c104e50 con 0x7f7948039560 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f7948039560 msgr2=0x7f794803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f7948039560 0x7f794803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f794c006fd0 tx=0x7f794c006e40 comp rx=0 tx=0).stop 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 msgr2=0x7f795c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c19e6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7958009fb0 tx=0x7f7958011940 comp rx=0 tx=0).stop 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 shutdown_connections 2024-04-22T13:17:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f7948039560 0x7f794803ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:58.999 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 --2- 172.21.15.5:0/1568497514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f795c07de60 0x7f795c19e6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:58.999 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.439+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 >> 172.21.15.5:0/1568497514 conn(0x7f795c103b40 msgr2=0x7f795c104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:58.999 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.440+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 shutdown_connections 2024-04-22T13:17:58.999 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:58.440+0000 7f7964648700 1 -- 172.21.15.5:0/1568497514 wait complete. 2024-04-22T13:17:58.999 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-04-22T13:17:59.610 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:59 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:17:59.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:59 smithi005 ceph-mon[28006]: Added host smithi005 2024-04-22T13:17:59.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:17:59 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:00.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:00 smithi005 ceph-mon[28006]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:00.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:00 smithi005 ceph-mon[28006]: Saving service mon spec with placement count:5 2024-04-22T13:18:00.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:00 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:18:00.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:00 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.456+0000 7fe670847700 1 Processor -- start 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.456+0000 7fe670847700 1 -- start start 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe670847700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe670847700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c107e30 con 0x7fe66c107440 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe66a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:01.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe66a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35044/0 (socket says 172.21.15.5:35044) 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe66a59c700 1 -- 172.21.15.5:0/1593230175 learned_addr learned my addr 172.21.15.5:0/1593230175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe66a59c700 1 -- 172.21.15.5:0/1593230175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c108640 con 0x7fe66c107440 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.457+0000 7fe66a59c700 1 --2- 172.21.15.5:0/1593230175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c107860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0098a0 tx=0x7fe65c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375918779746f51d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe66959a700 1 -- 172.21.15.5:0/1593230175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65c004160 con 0x7fe66c107440 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe66959a700 1 -- 172.21.15.5:0/1593230175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe65c004300 con 0x7fe66c107440 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe66959a700 1 -- 172.21.15.5:0/1593230175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65c011af0 con 0x7fe66c107440 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 -- 172.21.15.5:0/1593230175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 msgr2=0x7fe66c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 --2- 172.21.15.5:0/1593230175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c107860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0098a0 tx=0x7fe65c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 -- 172.21.15.5:0/1593230175 shutdown_connections 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 --2- 172.21.15.5:0/1593230175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c107860 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 -- 172.21.15.5:0/1593230175 >> 172.21.15.5:0/1593230175 conn(0x7fe66c102a40 msgr2=0x7fe66c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 -- 172.21.15.5:0/1593230175 shutdown_connections 2024-04-22T13:18:01.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.458+0000 7fe670847700 1 -- 172.21.15.5:0/1593230175 wait complete. 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe670847700 1 Processor -- start 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe670847700 1 -- start start 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe670847700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe670847700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c10b9f0 con 0x7fe66c107440 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe66a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe66a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35060/0 (socket says 172.21.15.5:35060) 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.459+0000 7fe66a59c700 1 -- 172.21.15.5:0/286151476 learned_addr learned my addr 172.21.15.5:0/286151476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe66a59c700 1 -- 172.21.15.5:0/286151476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c19ca60 con 0x7fe66c107440 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe66a59c700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c19c520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009fb0 tx=0x7fe65c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65c0036a0 con 0x7fe66c107440 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c19ccf0 con 0x7fe66c107440 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe65c01a430 con 0x7fe66c107440 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65c019550 con 0x7fe66c107440 2024-04-22T13:18:01.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.460+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c19f9e0 con 0x7fe66c107440 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.461+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe65c019850 con 0x7fe66c107440 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.461+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c0640a0 con 0x7fe66c107440 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.461+0000 7fe65b7fe700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fe654039560 0x7fe65403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.461+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe65c04d920 con 0x7fe66c107440 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.461+0000 7fe669d9b700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fe654039560 0x7fe65403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.462+0000 7fe669d9b700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fe654039560 0x7fe65403ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe660006fd0 tx=0x7fe660006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.466+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe65c00fda0 con 0x7fe66c107440 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.692+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe66c0008d0 con 0x7fe654039560 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.697+0000 7fe65b7fe700 1 -- 172.21.15.5:0/286151476 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe66c0008d0 con 0x7fe654039560 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fe654039560 msgr2=0x7fe65403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fe654039560 0x7fe65403ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe660006fd0 tx=0x7fe660006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 msgr2=0x7fe66c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:01.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c19c520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009fb0 tx=0x7fe65c011940 comp rx=0 tx=0).stop 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 shutdown_connections 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fe654039560 0x7fe65403ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 --2- 172.21.15.5:0/286151476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe66c107440 0x7fe66c19c520 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 >> 172.21.15.5:0/286151476 conn(0x7fe66c102a40 msgr2=0x7fe66c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 shutdown_connections 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:17:59.699+0000 7fe670847700 1 -- 172.21.15.5:0/286151476 wait complete. 2024-04-22T13:18:01.215 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-04-22T13:18:02.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:02 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:18:02.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:02 smithi005 ceph-mon[28006]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:02.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:02 smithi005 ceph-mon[28006]: Saving service mgr spec with placement count:2 2024-04-22T13:18:02.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:02 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:18:02.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-22T13:18:02.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.629+0000 7efcdcbde700 1 Processor -- start 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcdcbde700 1 -- start start 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcdcbde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd8106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcdcbde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8107230 con 0x7efcd8106840 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd8106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd8106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35072/0 (socket says 172.21.15.5:35072) 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcd659c700 1 -- 172.21.15.5:0/3554204493 learned_addr learned my addr 172.21.15.5:0/3554204493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.630+0000 7efcd659c700 1 -- 172.21.15.5:0/3554204493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd81073b0 con 0x7efcd8106840 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcd659c700 1 --2- 172.21.15.5:0/3554204493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd8106c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efcc00098a0 tx=0x7efcc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498ff2b1ecd09cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcd559a700 1 -- 172.21.15.5:0/3554204493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc0004160 con 0x7efcd8106840 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcd559a700 1 -- 172.21.15.5:0/3554204493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc0004300 con 0x7efcd8106840 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcd559a700 1 -- 172.21.15.5:0/3554204493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc0011a20 con 0x7efcd8106840 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcdcbde700 1 -- 172.21.15.5:0/3554204493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 msgr2=0x7efcd8106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcdcbde700 1 --2- 172.21.15.5:0/3554204493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd8106c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efcc00098a0 tx=0x7efcc0009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:02.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcdcbde700 1 -- 172.21.15.5:0/3554204493 shutdown_connections 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcdcbde700 1 --2- 172.21.15.5:0/3554204493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd8106c60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcdcbde700 1 -- 172.21.15.5:0/3554204493 >> 172.21.15.5:0/3554204493 conn(0x7efcd81021a0 msgr2=0x7efcd81045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.631+0000 7efcdcbde700 1 -- 172.21.15.5:0/3554204493 shutdown_connections 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.632+0000 7efcdcbde700 1 -- 172.21.15.5:0/3554204493 wait complete. 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.632+0000 7efcdcbde700 1 Processor -- start 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.632+0000 7efcdcbde700 1 -- start start 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.632+0000 7efcdcbde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.632+0000 7efcdcbde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcd810a750 con 0x7efcd8106840 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35082/0 (socket says 172.21.15.5:35082) 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcd659c700 1 -- 172.21.15.5:0/2271776643 learned_addr learned my addr 172.21.15.5:0/2271776643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcd659c700 1 -- 172.21.15.5:0/2271776643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd819eb10 con 0x7efcd8106840 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcd659c700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd819e5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009520 tx=0x7efcc0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc00039e0 con 0x7efcd8106840 2024-04-22T13:18:02.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd819ee00 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc001b8f0 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc0019770 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.633+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81a1a90 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.634+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efcc0019a70 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.634+0000 7efccf7fe700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7efcc40395b0 0x7efcc403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.634+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd80640a0 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.635+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efcc004de90 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.635+0000 7efcd5d9b700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7efcc40395b0 0x7efcc403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.635+0000 7efcd5d9b700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7efcc40395b0 0x7efcc403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006fd0 tx=0x7efcc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.640+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efcc001f030 con 0x7efcd8106840 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.871+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efcd81a1c60 con 0x7efcc40395b0 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.875+0000 7efccf7fe700 1 -- 172.21.15.5:0/2271776643 <== mgr.14120 v2:172.21.15.5:6800/4095818300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efcd81a1c60 con 0x7efcc40395b0 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.877+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7efcc40395b0 msgr2=0x7efcc403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:02.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7efcc40395b0 0x7efcc403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006fd0 tx=0x7efcc8006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 msgr2=0x7efcd819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd819e5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009520 tx=0x7efcc0011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 shutdown_connections 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7efcc40395b0 0x7efcc403ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 --2- 172.21.15.5:0/2271776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd8106840 0x7efcd819e5d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 >> 172.21.15.5:0/2271776643 conn(0x7efcd81021a0 msgr2=0x7efcd8102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 shutdown_connections 2024-04-22T13:18:02.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:01.878+0000 7efcdcbde700 1 -- 172.21.15.5:0/2271776643 wait complete. 2024-04-22T13:18:04.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:04 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3804949691' entity='client.admin' 2024-04-22T13:18:04.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:04 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.856+0000 7f5cf3597700 1 Processor -- start 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.856+0000 7f5cf3597700 1 -- start start 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.856+0000 7f5cf3597700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec107230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.856+0000 7f5cf3597700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec07be40 con 0x7f5cec106e10 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.856+0000 7f5cf1333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5cf1333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35088/0 (socket says 172.21.15.5:35088) 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5cf1333700 1 -- 172.21.15.5:0/309865212 learned_addr learned my addr 172.21.15.5:0/309865212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5cf1333700 1 -- 172.21.15.5:0/309865212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec107770 con 0x7f5cec106e10 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5cf1333700 1 --2- 172.21.15.5:0/309865212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec107230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0098a0 tx=0x7f5cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d12f8697c9b20a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5ce3fff700 1 -- 172.21.15.5:0/309865212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cdc004160 con 0x7f5cec106e10 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5ce3fff700 1 -- 172.21.15.5:0/309865212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc004300 con 0x7f5cec106e10 2024-04-22T13:18:04.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.857+0000 7f5ce3fff700 1 -- 172.21.15.5:0/309865212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cdc011a20 con 0x7f5cec106e10 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 -- 172.21.15.5:0/309865212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 msgr2=0x7f5cec107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 --2- 172.21.15.5:0/309865212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec107230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0098a0 tx=0x7f5cdc009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 -- 172.21.15.5:0/309865212 shutdown_connections 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 --2- 172.21.15.5:0/309865212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec107230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 -- 172.21.15.5:0/309865212 >> 172.21.15.5:0/309865212 conn(0x7f5cec102a50 msgr2=0x7f5cec104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 -- 172.21.15.5:0/309865212 shutdown_connections 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.858+0000 7f5cf3597700 1 -- 172.21.15.5:0/309865212 wait complete. 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf3597700 1 Processor -- start 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf3597700 1 -- start start 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf3597700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf3597700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1087e0 con 0x7f5cec106e10 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf1333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf1333700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35094/0 (socket says 172.21.15.5:35094) 2024-04-22T13:18:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf1333700 1 -- 172.21.15.5:0/3804949691 learned_addr learned my addr 172.21.15.5:0/3804949691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf1333700 1 -- 172.21.15.5:0/3804949691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec19eb80 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.859+0000 7f5cf1333700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec19e640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0116e0 tx=0x7f5cdc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.860+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cdc003b20 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.860+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc01b8f0 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.860+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec19ee10 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.860+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cdc0238c0 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.860+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec1a1b00 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.861+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5cdc023b70 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.861+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec0640a0 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.861+0000 7f5ce27fc700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f5cd8039560 0x7f5cd803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.861+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cdc04e040 con 0x7f5cec106e10 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.861+0000 7f5cf0b32700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f5cd8039560 0x7f5cd803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.861+0000 7f5cf0b32700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f5cd8039560 0x7f5cd803ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006fd0 tx=0x7f5ce8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:04.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:02.866+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cdc01f030 con 0x7f5cec106e10 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.077+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f5cec02e1d0 con 0x7f5cec106e10 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.080+0000 7f5ce27fc700 1 -- 172.21.15.5:0/3804949691 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f5cdc020d70 con 0x7f5cec106e10 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f5cd8039560 msgr2=0x7f5cd803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f5cd8039560 0x7f5cd803ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006fd0 tx=0x7f5ce8006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 msgr2=0x7f5cec19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec19e640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0116e0 tx=0x7f5cdc011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 shutdown_connections 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f5cd8039560 0x7f5cd803ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 --2- 172.21.15.5:0/3804949691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec106e10 0x7f5cec19e640 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0116e0 tx=0x7f5cdc011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.082+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 >> 172.21.15.5:0/3804949691 conn(0x7f5cec102a50 msgr2=0x7f5cec195280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.083+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 shutdown_connections 2024-04-22T13:18:04.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:03.083+0000 7f5cf3597700 1 -- 172.21.15.5:0/3804949691 wait complete. 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.546+0000 7f3b39e87700 1 Processor -- start 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.546+0000 7f3b39e87700 1 -- start start 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.546+0000 7f3b39e87700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b34108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.546+0000 7f3b39e87700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34108f50 con 0x7f3b34108560 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b34108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b34108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35108/0 (socket says 172.21.15.5:35108) 2024-04-22T13:18:05.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b337fe700 1 -- 172.21.15.5:0/3204549795 learned_addr learned my addr 172.21.15.5:0/3204549795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b337fe700 1 -- 172.21.15.5:0/3204549795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b34109710 con 0x7f3b34108560 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b337fe700 1 --2- 172.21.15.5:0/3204549795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b34108980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009e80 tx=0x7f3b2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6b5f76a8e3dc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b327fc700 1 -- 172.21.15.5:0/3204549795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b24017070 con 0x7f3b34108560 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b327fc700 1 -- 172.21.15.5:0/3204549795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2400b8c0 con 0x7f3b34108560 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.547+0000 7f3b327fc700 1 -- 172.21.15.5:0/3204549795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b24004490 con 0x7f3b34108560 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 -- 172.21.15.5:0/3204549795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 msgr2=0x7f3b34108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 --2- 172.21.15.5:0/3204549795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b34108980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009e80 tx=0x7f3b2400b330 comp rx=0 tx=0).stop 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 -- 172.21.15.5:0/3204549795 shutdown_connections 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 --2- 172.21.15.5:0/3204549795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b34108980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 -- 172.21.15.5:0/3204549795 >> 172.21.15.5:0/3204549795 conn(0x7f3b34103b40 msgr2=0x7f3b34105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 -- 172.21.15.5:0/3204549795 shutdown_connections 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 -- 172.21.15.5:0/3204549795 wait complete. 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.548+0000 7f3b39e87700 1 Processor -- start 2024-04-22T13:18:05.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b39e87700 1 -- start start 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b39e87700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b3407db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b39e87700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410cac0 con 0x7f3b34108560 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b3407db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b3407db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35110/0 (socket says 172.21.15.5:35110) 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b337fe700 1 -- 172.21.15.5:0/1733815474 learned_addr learned my addr 172.21.15.5:0/1733815474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b337fe700 1 -- 172.21.15.5:0/1733815474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3407e0c0 con 0x7f3b34108560 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.549+0000 7f3b337fe700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b3407db80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400bca0 tx=0x7f3b2400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.550+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b24020020 con 0x7f3b34108560 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.550+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b24003b50 con 0x7f3b34108560 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.550+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3407c2b0 con 0x7f3b34108560 2024-04-22T13:18:05.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.550+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b24017040 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.550+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3407c7f0 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.550+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3b24023020 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.551+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b340640a0 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.551+0000 7f3b30ff9700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f3b20039600 0x7f3b2003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.551+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b2400f050 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.551+0000 7f3b32ffd700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f3b20039600 0x7f3b2003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.551+0000 7f3b32ffd700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f3b20039600 0x7f3b2003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006fd0 tx=0x7f3b1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.556+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b2402a020 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.762+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3b3407cb30 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.765+0000 7f3b30ff9700 1 -- 172.21.15.5:0/1733815474 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3b3407cb30 con 0x7f3b34108560 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f3b20039600 msgr2=0x7f3b2003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f3b20039600 0x7f3b2003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006fd0 tx=0x7f3b1c006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 msgr2=0x7f3b3407db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:05.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b3407db80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400bca0 tx=0x7f3b2400bd80 comp rx=0 tx=0).stop 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 shutdown_connections 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f3b20039600 0x7f3b2003bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 --2- 172.21.15.5:0/1733815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b34108560 0x7f3b3407db80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 >> 172.21.15.5:0/1733815474 conn(0x7f3b34103b40 msgr2=0x7f3b341046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.767+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 shutdown_connections 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:04.768+0000 7f3b39e87700 1 -- 172.21.15.5:0/1733815474 wait complete. 2024-04-22T13:18:05.413 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-04-22T13:18:06.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:05 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1733815474' entity='client.admin' 2024-04-22T13:18:07.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:06 smithi005 ceph-mon[28006]: from='mgr.14120 172.21.15.5:0/3202998420' entity='mgr.x' 2024-04-22T13:18:07.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:06 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3545199862' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-22T13:18:07.133 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: ignoring --setuser ceph since I am not root 2024-04-22T13:18:07.133 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: ignoring --setgroup ceph since I am not root 2024-04-22T13:18:07.133 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:07.135+0000 7f324f2f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:18:07.427 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:07.327+0000 7f324f2f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:18:07.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:07.434+0000 7f324f2f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:18:07.954 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:07.755+0000 7f324f2f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:18:07.954 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:07.843+0000 7f324f2f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:18:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2893eb6700 1 Processor -- start 2024-04-22T13:18:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2893eb6700 1 -- start start 2024-04-22T13:18:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2893eb6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2893eb6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f288c107e30 con 0x7f288c107440 2024-04-22T13:18:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2891c52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2891c52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35118/0 (socket says 172.21.15.5:35118) 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.826+0000 7f2891c52700 1 -- 172.21.15.5:0/1318721549 learned_addr learned my addr 172.21.15.5:0/1318721549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.827+0000 7f2891c52700 1 -- 172.21.15.5:0/1318721549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c108640 con 0x7f288c107440 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.827+0000 7f2891c52700 1 --2- 172.21.15.5:0/1318721549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c107860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28880098a0 tx=0x7f2888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604b80e3f8c95747 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.827+0000 7f2890c50700 1 -- 172.21.15.5:0/1318721549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2888004160 con 0x7f288c107440 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.827+0000 7f2890c50700 1 -- 172.21.15.5:0/1318721549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2888004300 con 0x7f288c107440 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.827+0000 7f2890c50700 1 -- 172.21.15.5:0/1318721549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2888011a20 con 0x7f288c107440 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 -- 172.21.15.5:0/1318721549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 msgr2=0x7f288c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 --2- 172.21.15.5:0/1318721549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c107860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28880098a0 tx=0x7f2888009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 -- 172.21.15.5:0/1318721549 shutdown_connections 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 --2- 172.21.15.5:0/1318721549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c107860 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 -- 172.21.15.5:0/1318721549 >> 172.21.15.5:0/1318721549 conn(0x7f288c102a40 msgr2=0x7f288c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 -- 172.21.15.5:0/1318721549 shutdown_connections 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 -- 172.21.15.5:0/1318721549 wait complete. 2024-04-22T13:18:08.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 Processor -- start 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.828+0000 7f2893eb6700 1 -- start start 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2893eb6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c1925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2893eb6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f288c10b9f0 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2891c52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c1925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2891c52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c1925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35128/0 (socket says 172.21.15.5:35128) 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2891c52700 1 -- 172.21.15.5:0/3545199862 learned_addr learned my addr 172.21.15.5:0/3545199862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2891c52700 1 -- 172.21.15.5:0/3545199862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c192af0 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2891c52700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c1925b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f2888011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2888003aa0 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288801b8f0 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c190ce0 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2888023890 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.829+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c191220 con 0x7f288c107440 2024-04-22T13:18:08.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.830+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f288801b440 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.830+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c0640a0 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.830+0000 7f2882ffd700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f28780395b0 0x7f287803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.830+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f288804de60 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.831+0000 7f2891451700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f28780395b0 0x7f287803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.831+0000 7f2891451700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f28780395b0 0x7f287803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f287c006fd0 tx=0x7f287c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:05.836+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f288801f030 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.107+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f288c1962e0 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.952+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f28880292a0 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.954+0000 7f2882ffd700 1 -- 172.21.15.5:0/3545199862 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f2888051030 con 0x7f288c107440 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f28780395b0 msgr2=0x7f287803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f28780395b0 0x7f287803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f287c006fd0 tx=0x7f287c006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 msgr2=0x7f288c1925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c1925b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f2888011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:08.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 shutdown_connections 2024-04-22T13:18:08.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7f28780395b0 0x7f287803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:08.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 --2- 172.21.15.5:0/3545199862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f288c107440 0x7f288c1925b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:08.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 >> 172.21.15.5:0/3545199862 conn(0x7f288c102a40 msgr2=0x7f288c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:08.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 shutdown_connections 2024-04-22T13:18:08.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:06.955+0000 7f2893eb6700 1 -- 172.21.15.5:0/3545199862 wait complete. 2024-04-22T13:18:08.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3545199862' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-22T13:18:08.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:07 smithi005 ceph-mon[28006]: mgrmap e9: x(active, since 19s) 2024-04-22T13:18:09.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:09 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3498645492' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T13:18:09.288 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:09.111+0000 7f324f2f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:18:09.288 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:09.289+0000 7f324f2f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:18:09.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:09.467+0000 7f324f2f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:18:09.603 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.723+0000 7fc902d4e700 1 Processor -- start 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.723+0000 7fc902d4e700 1 -- start start 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.723+0000 7fc902d4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.723+0000 7fc902d4e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc063cc0 con 0x7fc8fc106e40 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc900aea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc900aea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35158/0 (socket says 172.21.15.5:35158) 2024-04-22T13:18:09.604 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc900aea700 1 -- 172.21.15.5:0/1730784311 learned_addr learned my addr 172.21.15.5:0/1730784311 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc900aea700 1 -- 172.21.15.5:0/1730784311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1077a0 con 0x7fc8fc106e40 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc900aea700 1 --2- 172.21.15.5:0/1730784311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc107260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00098a0 tx=0x7fc8f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e53d6f94b3f8a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc8fb7fe700 1 -- 172.21.15.5:0/1730784311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8f0004160 con 0x7fc8fc106e40 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc8fb7fe700 1 -- 172.21.15.5:0/1730784311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f0004300 con 0x7fc8fc106e40 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.724+0000 7fc8fb7fe700 1 -- 172.21.15.5:0/1730784311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8f0011a20 con 0x7fc8fc106e40 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 -- 172.21.15.5:0/1730784311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 msgr2=0x7fc8fc107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 --2- 172.21.15.5:0/1730784311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc107260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00098a0 tx=0x7fc8f0009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 -- 172.21.15.5:0/1730784311 shutdown_connections 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 --2- 172.21.15.5:0/1730784311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc107260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 -- 172.21.15.5:0/1730784311 >> 172.21.15.5:0/1730784311 conn(0x7fc8fc102a40 msgr2=0x7fc8fc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 -- 172.21.15.5:0/1730784311 shutdown_connections 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.725+0000 7fc902d4e700 1 -- 172.21.15.5:0/1730784311 wait complete. 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc902d4e700 1 Processor -- start 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc902d4e700 1 -- start start 2024-04-22T13:18:09.605 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc902d4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc902d4e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc108810 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc900aea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc900aea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35162/0 (socket says 172.21.15.5:35162) 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.726+0000 7fc900aea700 1 -- 172.21.15.5:0/3498645492 learned_addr learned my addr 172.21.15.5:0/3498645492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc900aea700 1 -- 172.21.15.5:0/3498645492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc19eb00 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc900aea700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc19e5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00116e0 tx=0x7fc8f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8f0003a60 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f001b8f0 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8f0019860 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc19ed90 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.727+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1a1a80 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.728+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fc8f0019b10 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.728+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc0640a0 con 0x7fc8fc106e40 2024-04-22T13:18:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.728+0000 7fc8f9ffb700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 0x7fc8ec03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.729+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc8f004de30 con 0x7fc8fc106e40 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.729+0000 7fc8fbfff700 1 -- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 msgr2=0x7fc8ec03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.729+0000 7fc8fbfff700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 0x7fc8ec03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.734+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8f001f030 con 0x7fc8fc106e40 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.930+0000 7fc8fbfff700 1 -- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 msgr2=0x7fc8ec03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:08.930+0000 7fc8fbfff700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 0x7fc8ec03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.004+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc8fc02e1d0 con 0x7fc8fc106e40 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.005+0000 7fc8f9ffb700 1 -- 172.21.15.5:0/3498645492 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc8f0019e00 con 0x7fc8fc106e40 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 msgr2=0x7fc8ec03bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 0x7fc8ec03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 msgr2=0x7fc8fc19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc19e5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00116e0 tx=0x7fc8f0011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 shutdown_connections 2024-04-22T13:18:09.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fc8ec039610 0x7fc8ec03bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:09.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 --2- 172.21.15.5:0/3498645492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8fc106e40 0x7fc8fc19e5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:09.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 >> 172.21.15.5:0/3498645492 conn(0x7fc8fc102a40 msgr2=0x7fc8fc195260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:09.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 shutdown_connections 2024-04-22T13:18:09.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:09.007+0000 7fc902d4e700 1 -- 172.21.15.5:0/3498645492 wait complete. 2024-04-22T13:18:09.608 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-04-22T13:18:09.608 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-04-22T13:18:09.870 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:09.774+0000 7f324f2f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:18:09.870 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:09.872+0000 7f324f2f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:18:10.289 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:10.088+0000 7f324f2f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:18:10.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:10.292+0000 7f324f2f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:18:11.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:10.917+0000 7f324f2f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:18:11.666 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:11.358+0000 7f324f2f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:18:11.666 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:11.458+0000 7f324f2f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:18:11.666 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:11.668+0000 7f324f2f3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:18:12.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:12.212+0000 7f324f2f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:18:12.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:12.308+0000 7f324f2f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:18:12.921 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:12.564+0000 7f324f2f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:18:12.921 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:12.702+0000 7f324f2f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:18:12.921 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:12.802+0000 7f324f2f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:18:12.921 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:12.923+0000 7f324f2f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:18:13.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:13.235+0000 7f324f2f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:18:14.147 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:13.866+0000 7f324f2f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:18:14.148 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:18:14.096+0000 7f324f2f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: Active manager daemon x restarted 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: Activating manager daemon x 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: osdmap e3: 0 total, 0 up, 0 in 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: mgrmap e10: x(active, starting, since 0.0033305s) 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: Manager daemon x is now available 2024-04-22T13:18:14.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:18:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:18:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:15 smithi005 ceph-mon[28006]: mgrmap e11: x(active, since 1.00635s) 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcad12d0700 1 Processor -- start 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcad12d0700 1 -- start start 2024-04-22T13:18:15.890 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcad12d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc10b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcad12d0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc063ce0 con 0x7fcacc109030 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc10b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40446/0 (socket says 172.21.15.5:40446) 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.096+0000 7fcacaffd700 1 -- 172.21.15.5:0/3636461872 learned_addr learned my addr 172.21.15.5:0/3636461872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.097+0000 7fcacaffd700 1 -- 172.21.15.5:0/3636461872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc10b960 con 0x7fcacc109030 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.097+0000 7fcacaffd700 1 --2- 172.21.15.5:0/3636461872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc10b420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcac00098a0 tx=0x7fcac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6bdd35c961a6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.097+0000 7fcac9ffb700 1 -- 172.21.15.5:0/3636461872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0004160 con 0x7fcacc109030 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.097+0000 7fcac9ffb700 1 -- 172.21.15.5:0/3636461872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac0004300 con 0x7fcacc109030 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.097+0000 7fcac9ffb700 1 -- 172.21.15.5:0/3636461872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0011a20 con 0x7fcacc109030 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 -- 172.21.15.5:0/3636461872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 msgr2=0x7fcacc10b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3636461872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc10b420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcac00098a0 tx=0x7fcac0009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 -- 172.21.15.5:0/3636461872 shutdown_connections 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3636461872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc10b420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 -- 172.21.15.5:0/3636461872 >> 172.21.15.5:0/3636461872 conn(0x7fcacc102a40 msgr2=0x7fcacc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 -- 172.21.15.5:0/3636461872 shutdown_connections 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.098+0000 7fcad12d0700 1 -- 172.21.15.5:0/3636461872 wait complete. 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcad12d0700 1 Processor -- start 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcad12d0700 1 -- start start 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcad12d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcad12d0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc10c900 con 0x7fcacc109030 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40460/0 (socket says 172.21.15.5:40460) 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcacaffd700 1 -- 172.21.15.5:0/3802666685 learned_addr learned my addr 172.21.15.5:0/3802666685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.099+0000 7fcacaffd700 1 -- 172.21.15.5:0/3802666685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc19eb70 con 0x7fcacc109030 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.100+0000 7fcacaffd700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc19e630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcac00116e0 tx=0x7fcac0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.100+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0003a60 con 0x7fcacc109030 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.100+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac001b8f0 con 0x7fcacc109030 2024-04-22T13:18:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.100+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc19ee00 con 0x7fcacc109030 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.100+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac0019860 con 0x7fcacc109030 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.100+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1a1af0 con 0x7fcacc109030 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.101+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcac0019b10 con 0x7fcacc109030 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.101+0000 7fcab3fff700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.101+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 --> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcab403c1e0 con 0x7fcab4039610 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.101+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcac004dd10 con 0x7fcacc109030 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.101+0000 7fcaca7fc700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 msgr2=0x7fcab403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.101+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.302+0000 7fcaca7fc700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 msgr2=0x7fcab403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.302+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.703+0000 7fcaca7fc700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 msgr2=0x7fcab403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:10.703+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:11.504+0000 7fcaca7fc700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 msgr2=0x7fcab403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:11.504+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T13:18:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:13.105+0000 7fcaca7fc700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 msgr2=0x7fcab403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4095818300 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:13.105+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:14.100+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fcac001a8f0 con 0x7fcacc109030 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:14.100+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 msgr2=0x7fcab403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:14.100+0000 7fcab3fff700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.102+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fcac004fa40 con 0x7fcacc109030 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.102+0000 7fcab3fff700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab403cff0 0x7fcab403f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.102+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcab403c1e0 con 0x7fcab403cff0 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.103+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab403cff0 0x7fcab403f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.103+0000 7fcaca7fc700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab403cff0 0x7fcab403f3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcabc003a10 tx=0x7fcabc009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.104+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fcab403c1e0 con 0x7fcab403cff0 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.110+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcacc069ba0 con 0x7fcab403cff0 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcab3fff700 1 -- 172.21.15.5:0/3802666685 <== mgr.14152 v2:172.21.15.5:6800/2674774949 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fcacc069ba0 con 0x7fcab403cff0 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab403cff0 msgr2=0x7fcab403f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:15.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab403cff0 0x7fcab403f3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcabc003a10 tx=0x7fcabc009270 comp rx=0 tx=0).stop 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 msgr2=0x7fcacc19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc19e630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcac00116e0 tx=0x7fcac0011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 shutdown_connections 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab403cff0 0x7fcab403f3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcacc109030 0x7fcacc19e630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 --2- 172.21.15.5:0/3802666685 >> [v2:172.21.15.5:6800/4095818300,v1:172.21.15.5:6801/4095818300] conn(0x7fcab4039610 0x7fcab403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 >> 172.21.15.5:0/3802666685 conn(0x7fcacc102a40 msgr2=0x7fcacc103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 shutdown_connections 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:15.111+0000 7fcad12d0700 1 -- 172.21.15.5:0/3802666685 wait complete. 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-04-22T13:18:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.346+0000 7efd345db700 1 Processor -- start 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd345db700 1 -- start start 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd345db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c107200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd345db700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c063d20 con 0x7efd2c106de0 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd32377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd32377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c107200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40556/0 (socket says 172.21.15.5:40556) 2024-04-22T13:18:17.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd32377700 1 -- 172.21.15.5:0/445409735 learned_addr learned my addr 172.21.15.5:0/445409735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.347+0000 7efd32377700 1 -- 172.21.15.5:0/445409735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c107740 con 0x7efd2c106de0 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.348+0000 7efd32377700 1 --2- 172.21.15.5:0/445409735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c107200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd280098a0 tx=0x7efd28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645539779d0c7234 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.348+0000 7efd31375700 1 -- 172.21.15.5:0/445409735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd28004160 con 0x7efd2c106de0 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.348+0000 7efd31375700 1 -- 172.21.15.5:0/445409735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd28004300 con 0x7efd2c106de0 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.348+0000 7efd31375700 1 -- 172.21.15.5:0/445409735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd28011a20 con 0x7efd2c106de0 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.348+0000 7efd345db700 1 -- 172.21.15.5:0/445409735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 msgr2=0x7efd2c107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.348+0000 7efd345db700 1 --2- 172.21.15.5:0/445409735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c107200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd280098a0 tx=0x7efd28009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 -- 172.21.15.5:0/445409735 shutdown_connections 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 --2- 172.21.15.5:0/445409735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c107200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 -- 172.21.15.5:0/445409735 >> 172.21.15.5:0/445409735 conn(0x7efd2c102a40 msgr2=0x7efd2c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 -- 172.21.15.5:0/445409735 shutdown_connections 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 -- 172.21.15.5:0/445409735 wait complete. 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 Processor -- start 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 -- start start 2024-04-22T13:18:17.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.349+0000 7efd345db700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1087b0 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd32377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd32377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40570/0 (socket says 172.21.15.5:40570) 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd32377700 1 -- 172.21.15.5:0/3795518749 learned_addr learned my addr 172.21.15.5:0/3795518749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd32377700 1 -- 172.21.15.5:0/3795518749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c19eb20 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd32377700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c19e5e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd28000c00 tx=0x7efd28011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd28003aa0 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2801b8f0 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c19edb0 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd28023870 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.350+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c1a1aa0 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.351+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7efd28023b20 con 0x7efd2c106de0 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.351+0000 7efd237fe700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efd18039550 0x7efd1803ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:17.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.351+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd2804def0 con 0x7efd2c106de0 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.351+0000 7efd31b76700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efd18039550 0x7efd1803ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.351+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c0640a0 con 0x7efd2c106de0 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.352+0000 7efd31b76700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efd18039550 0x7efd1803ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006fd0 tx=0x7efd1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.357+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd2801f030 con 0x7efd2c106de0 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.579+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7efd2c02e140 con 0x7efd18039550 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.704+0000 7efd237fe700 1 -- 172.21.15.5:0/3795518749 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7efd2c02e140 con 0x7efd18039550 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efd18039550 msgr2=0x7efd1803ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efd18039550 0x7efd1803ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006fd0 tx=0x7efd1c006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 msgr2=0x7efd2c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c19e5e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd28000c00 tx=0x7efd28011aa0 comp rx=0 tx=0).stop 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 shutdown_connections 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efd18039550 0x7efd1803ba10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 --2- 172.21.15.5:0/3795518749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2c106de0 0x7efd2c19e5e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:17.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.706+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 >> 172.21.15.5:0/3795518749 conn(0x7efd2c102a40 msgr2=0x7efd2c195250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:17.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.707+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 shutdown_connections 2024-04-22T13:18:17.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:16.707+0000 7efd345db700 1 -- 172.21.15.5:0/3795518749 wait complete. 2024-04-22T13:18:17.174 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-04-22T13:18:17.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: [22/Apr/2024:13:18:15] ENGINE Bus STARTING 2024-04-22T13:18:17.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T13:18:17.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T13:18:17.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: [22/Apr/2024:13:18:15] ENGINE Serving on https://172.21.15.5:7150 2024-04-22T13:18:17.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: [22/Apr/2024:13:18:15] ENGINE Serving on http://172.21.15.5:8765 2024-04-22T13:18:17.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: [22/Apr/2024:13:18:15] ENGINE Bus STARTED 2024-04-22T13:18:17.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:17.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:17.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:16 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:18.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:17 smithi005 ceph-mon[28006]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:18.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:17 smithi005 ceph-mon[28006]: mgrmap e12: x(active, since 2s) 2024-04-22T13:18:18.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:19.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:18 smithi005 ceph-mon[28006]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:19.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:18 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:19.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9OHgbc75nLl2u.hHDzZRdux9/lCuG1aSBKJJAd2KJoTOfPwEWlI5K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713791898, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-22T13:18:19.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.657+0000 7f13f88e3700 1 Processor -- start 2024-04-22T13:18:19.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.657+0000 7f13f88e3700 1 -- start start 2024-04-22T13:18:19.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.657+0000 7f13f88e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f4107870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:19.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.657+0000 7f13f88e3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4107e40 con 0x7f13f4107450 2024-04-22T13:18:19.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f4107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f4107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40584/0 (socket says 172.21.15.5:40584) 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f259c700 1 -- 172.21.15.5:0/722196043 learned_addr learned my addr 172.21.15.5:0/722196043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f259c700 1 -- 172.21.15.5:0/722196043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4108650 con 0x7f13f4107450 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f259c700 1 --2- 172.21.15.5:0/722196043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f4107870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13e80098a0 tx=0x7f13e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ea929448d2d32f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f159a700 1 -- 172.21.15.5:0/722196043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e8004160 con 0x7f13f4107450 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f159a700 1 -- 172.21.15.5:0/722196043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e8004300 con 0x7f13f4107450 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.658+0000 7f13f159a700 1 -- 172.21.15.5:0/722196043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e8011a20 con 0x7f13f4107450 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 -- 172.21.15.5:0/722196043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 msgr2=0x7f13f4107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 --2- 172.21.15.5:0/722196043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f4107870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13e80098a0 tx=0x7f13e8009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 -- 172.21.15.5:0/722196043 shutdown_connections 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 --2- 172.21.15.5:0/722196043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f4107870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 -- 172.21.15.5:0/722196043 >> 172.21.15.5:0/722196043 conn(0x7f13f4102a50 msgr2=0x7f13f4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 -- 172.21.15.5:0/722196043 shutdown_connections 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.659+0000 7f13f88e3700 1 -- 172.21.15.5:0/722196043 wait complete. 2024-04-22T13:18:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.660+0000 7f13f88e3700 1 Processor -- start 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.660+0000 7f13f88e3700 1 -- start start 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f88e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f88e3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13f407afe0 con 0x7f13f4107450 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40588/0 (socket says 172.21.15.5:40588) 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f259c700 1 -- 172.21.15.5:0/2583293660 learned_addr learned my addr 172.21.15.5:0/2583293660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f259c700 1 -- 172.21.15.5:0/2583293660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f419c9f0 con 0x7f13f4107450 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f259c700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f419c4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13e80119f0 tx=0x7f13e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e80039e0 con 0x7f13f4107450 2024-04-22T13:18:19.310 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e801b8f0 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f419cc80 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e80237a0 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.661+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f419f970 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.662+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f13e8023aa0 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.662+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f40640a0 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.663+0000 7f13e37fe700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13dc0395b0 0x7f13dc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.663+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f13e804dfd0 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.663+0000 7f13f1d9b700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13dc0395b0 0x7f13dc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.663+0000 7f13f1d9b700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13dc0395b0 0x7f13dc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.668+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13e801f030 con 0x7f13f4107450 2024-04-22T13:18:19.311 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:17.906+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f13f4195f50 con 0x7f13dc0395b0 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.152+0000 7f13e37fe700 1 -- 172.21.15.5:0/2583293660 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f13f4195f50 con 0x7f13dc0395b0 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13dc0395b0 msgr2=0x7f13dc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13dc0395b0 0x7f13dc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 msgr2=0x7f13f419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f419c4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13e80119f0 tx=0x7f13e8011ad0 comp rx=0 tx=0).stop 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 shutdown_connections 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13dc0395b0 0x7f13dc03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 --2- 172.21.15.5:0/2583293660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4107450 0x7f13f419c4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 >> 172.21.15.5:0/2583293660 conn(0x7f13f4102a50 msgr2=0x7f13f4103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 shutdown_connections 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:18.154+0000 7f13f88e3700 1 -- 172.21.15.5:0/2583293660 wait complete. 2024-04-22T13:18:19.312 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-04-22T13:18:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:20 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3136860118' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-22T13:18:20.547 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a42b7c700 1 Processor -- start 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a42b7c700 1 -- start start 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a42b7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a42b7c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c108f50 con 0x7f5a3c108560 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a40918700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a40918700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36234/0 (socket says 172.21.15.5:36234) 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a40918700 1 -- 172.21.15.5:0/1281643367 learned_addr learned my addr 172.21.15.5:0/1281643367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a40918700 1 -- 172.21.15.5:0/1281643367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c109710 con 0x7f5a3c108560 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.821+0000 7f5a40918700 1 --2- 172.21.15.5:0/1281643367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c108980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a300098a0 tx=0x7f5a30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def59d27226e191c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a3b7fe700 1 -- 172.21.15.5:0/1281643367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a30004160 con 0x7f5a3c108560 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a3b7fe700 1 -- 172.21.15.5:0/1281643367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a30004300 con 0x7f5a3c108560 2024-04-22T13:18:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a3b7fe700 1 -- 172.21.15.5:0/1281643367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a30011a20 con 0x7f5a3c108560 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 -- 172.21.15.5:0/1281643367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 msgr2=0x7f5a3c108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 --2- 172.21.15.5:0/1281643367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c108980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a300098a0 tx=0x7f5a30009bb0 comp rx=0 tx=0).stop 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 -- 172.21.15.5:0/1281643367 shutdown_connections 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 --2- 172.21.15.5:0/1281643367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c108980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 -- 172.21.15.5:0/1281643367 >> 172.21.15.5:0/1281643367 conn(0x7f5a3c103b40 msgr2=0x7f5a3c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 -- 172.21.15.5:0/1281643367 shutdown_connections 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.822+0000 7f5a42b7c700 1 -- 172.21.15.5:0/1281643367 wait complete. 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a42b7c700 1 Processor -- start 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a42b7c700 1 -- start start 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a42b7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a42b7c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c10cac0 con 0x7f5a3c108560 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a40918700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a40918700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36238/0 (socket says 172.21.15.5:36238) 2024-04-22T13:18:20.549 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a40918700 1 -- 172.21.15.5:0/3136860118 learned_addr learned my addr 172.21.15.5:0/3136860118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a40918700 1 -- 172.21.15.5:0/3136860118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c19c9d0 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.823+0000 7f5a40918700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c19c490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a300119f0 tx=0x7f5a30011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.824+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a300039e0 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.824+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a3001b8f0 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.824+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c19cc60 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.824+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a300237a0 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.824+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c19f950 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.825+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f5a3001b440 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.825+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3c0640a0 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.825+0000 7f5a39ffb700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5a2c039610 0x7f5a2c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.825+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a3004df50 con 0x7f5a3c108560 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.825+0000 7f5a3bfff700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5a2c039610 0x7f5a2c03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.825+0000 7f5a3bfff700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5a2c039610 0x7f5a2c03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a28006fd0 tx=0x7f5a28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:20.550 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:19.830+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a3001f030 con 0x7f5a3c108560 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.038+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5a3c19ff00 con 0x7f5a3c108560 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.038+0000 7f5a39ffb700 1 -- 172.21.15.5:0/3136860118 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5a300480b0 con 0x7f5a3c108560 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.040+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5a2c039610 msgr2=0x7f5a2c03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5a2c039610 0x7f5a2c03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a28006fd0 tx=0x7f5a28006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 msgr2=0x7f5a3c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c19c490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a300119f0 tx=0x7f5a30011ad0 comp rx=0 tx=0).stop 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 shutdown_connections 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5a2c039610 0x7f5a2c03bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 --2- 172.21.15.5:0/3136860118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a3c108560 0x7f5a3c19c490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 >> 172.21.15.5:0/3136860118 conn(0x7f5a3c103b40 msgr2=0x7f5a3c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 shutdown_connections 2024-04-22T13:18:20.551 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:20.041+0000 7f5a42b7c700 1 -- 172.21.15.5:0/3136860118 wait complete. 2024-04-22T13:18:20.555 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:18:20.556 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-22T13:18:20.563 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:18:20.563 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-22T13:18:20.563 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-22T13:18:20.563 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-22T13:18:20.565 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-04-22T13:18:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005:8443/ 2024-04-22T13:18:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-04-22T13:18:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: Password: 3gkzj10m56 2024-04-22T13:18:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:20.566 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config directory 2024-04-22T13:18:20.566 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling autotune for osd_memory_target 2024-04-22T13:18:21.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7cc13700 1 Processor -- start 2024-04-22T13:18:21.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7cc13700 1 -- start start 2024-04-22T13:18:21.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7cc13700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f78108f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7cc13700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f78107060 con 0x7f1f78108b20 2024-04-22T13:18:21.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f78108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f78108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36254/0 (socket says 172.21.15.5:36254) 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7659c700 1 -- 172.21.15.5:0/2887499375 learned_addr learned my addr 172.21.15.5:0/2887499375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.187+0000 7f1f7659c700 1 -- 172.21.15.5:0/2887499375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f781071e0 con 0x7f1f78108b20 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7659c700 1 --2- 172.21.15.5:0/2887499375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f78108f40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bb40 tx=0x7f1f6800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e40f7a51d988a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f75d9b700 1 -- 172.21.15.5:0/2887499375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f6800dbe0 con 0x7f1f78108b20 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f75d9b700 1 -- 172.21.15.5:0/2887499375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f680044e0 con 0x7f1f78108b20 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f75d9b700 1 -- 172.21.15.5:0/2887499375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f68010470 con 0x7f1f78108b20 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7cc13700 1 -- 172.21.15.5:0/2887499375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 msgr2=0x7f1f78108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7cc13700 1 --2- 172.21.15.5:0/2887499375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f78108f40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bb40 tx=0x7f1f6800be50 comp rx=0 tx=0).stop 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7cc13700 1 -- 172.21.15.5:0/2887499375 shutdown_connections 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7cc13700 1 --2- 172.21.15.5:0/2887499375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f78108f40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7cc13700 1 -- 172.21.15.5:0/2887499375 >> 172.21.15.5:0/2887499375 conn(0x7f1f78102a60 msgr2=0x7f1f78104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.188+0000 7f1f7cc13700 1 -- 172.21.15.5:0/2887499375 shutdown_connections 2024-04-22T13:18:21.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.189+0000 7f1f7cc13700 1 -- 172.21.15.5:0/2887499375 wait complete. 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.189+0000 7f1f7cc13700 1 Processor -- start 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.189+0000 7f1f7cc13700 1 -- start start 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.189+0000 7f1f7cc13700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f7819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.189+0000 7f1f7cc13700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7807b160 con 0x7f1f78108b20 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f7819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f7819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36262/0 (socket says 172.21.15.5:36262) 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7659c700 1 -- 172.21.15.5:0/586635693 learned_addr learned my addr 172.21.15.5:0/586635693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7659c700 1 -- 172.21.15.5:0/586635693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7819ecf0 con 0x7f1f78108b20 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7659c700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f7819e7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f68003c30 tx=0x7f1f68003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f68010720 con 0x7f1f78108b20 2024-04-22T13:18:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7819ef80 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f6801d070 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f68021aa0 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.190+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781a1c70 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.191+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f1f6801f030 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.191+0000 7f1f6f7fe700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1f640396a0 0x7f1f6403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.191+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f6804dc40 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.191+0000 7f1f6ffff700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1f640396a0 0x7f1f6403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.192+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f780640a0 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.192+0000 7f1f6ffff700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1f640396a0 0x7f1f6403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f60006fd0 tx=0x7f1f60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.197+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f68015070 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.407+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f1f78193900 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.407+0000 7f1f6f7fe700 1 -- 172.21.15.5:0/586635693 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f1f68021da0 con 0x7f1f78108b20 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1f640396a0 msgr2=0x7f1f6403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:21.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1f640396a0 0x7f1f6403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f60006fd0 tx=0x7f1f60006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 msgr2=0x7f1f7819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f7819e7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f68003c30 tx=0x7f1f68003ff0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 shutdown_connections 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1f640396a0 0x7f1f6403bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 --2- 172.21.15.5:0/586635693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f78108b20 0x7f1f7819e7b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 >> 172.21.15.5:0/586635693 conn(0x7f1f78102a60 msgr2=0x7f1f781909c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 shutdown_connections 2024-04-22T13:18:21.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:21.409+0000 7f1f7cc13700 1 -- 172.21.15.5:0/586635693 wait complete. 2024-04-22T13:18:22.190 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:22 smithi005 ceph-mon[28006]: mgrmap e13: x(active, since 6s) 2024-04-22T13:18:23.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.399+0000 7f1d0f515700 1 Processor -- start 2024-04-22T13:18:23.231 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.399+0000 7f1d0f515700 1 -- start start 2024-04-22T13:18:23.231 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.399+0000 7f1d0f515700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d08108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.399+0000 7f1d0f515700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d08108f50 con 0x7f1d08108560 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d0d2b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d08108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d0d2b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d08108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36272/0 (socket says 172.21.15.5:36272) 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d0d2b1700 1 -- 172.21.15.5:0/1280287533 learned_addr learned my addr 172.21.15.5:0/1280287533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d0d2b1700 1 -- 172.21.15.5:0/1280287533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d08109710 con 0x7f1d08108560 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d0d2b1700 1 --2- 172.21.15.5:0/1280287533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d08108980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009e80 tx=0x7f1d0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdebd2cf930e3021 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d07fff700 1 -- 172.21.15.5:0/1280287533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d00017070 con 0x7f1d08108560 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d07fff700 1 -- 172.21.15.5:0/1280287533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0000b8c0 con 0x7f1d08108560 2024-04-22T13:18:23.232 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.400+0000 7f1d07fff700 1 -- 172.21.15.5:0/1280287533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d00004490 con 0x7f1d08108560 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 -- 172.21.15.5:0/1280287533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 msgr2=0x7f1d08108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 --2- 172.21.15.5:0/1280287533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d08108980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009e80 tx=0x7f1d0000b330 comp rx=0 tx=0).stop 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 -- 172.21.15.5:0/1280287533 shutdown_connections 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 --2- 172.21.15.5:0/1280287533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d08108980 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 -- 172.21.15.5:0/1280287533 >> 172.21.15.5:0/1280287533 conn(0x7f1d08103b40 msgr2=0x7f1d08105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 -- 172.21.15.5:0/1280287533 shutdown_connections 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.401+0000 7f1d0f515700 1 -- 172.21.15.5:0/1280287533 wait complete. 2024-04-22T13:18:23.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0f515700 1 Processor -- start 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0f515700 1 -- start start 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0f515700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d0819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0f515700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0810cac0 con 0x7f1d08108560 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0d2b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d0819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0d2b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d0819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36282/0 (socket says 172.21.15.5:36282) 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0d2b1700 1 -- 172.21.15.5:0/2783728779 learned_addr learned my addr 172.21.15.5:0/2783728779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.402+0000 7f1d0d2b1700 1 -- 172.21.15.5:0/2783728779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0819c9e0 con 0x7f1d08108560 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.403+0000 7f1d0d2b1700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d0819c4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000bca0 tx=0x7f1d0000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.403+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d00020020 con 0x7f1d08108560 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.403+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d00003b50 con 0x7f1d08108560 2024-04-22T13:18:23.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.403+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0819cc70 con 0x7f1d08108560 2024-04-22T13:18:23.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.403+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d00017040 con 0x7f1d08108560 2024-04-22T13:18:23.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.403+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0819f960 con 0x7f1d08108560 2024-04-22T13:18:23.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.404+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f1d00023020 con 0x7f1d08108560 2024-04-22T13:18:23.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.404+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d080640a0 con 0x7f1d08108560 2024-04-22T13:18:23.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.404+0000 7f1d067fc700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1cf00396a0 0x7f1cf003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:23.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.404+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1d0000f050 con 0x7f1d08108560 2024-04-22T13:18:23.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.404+0000 7f1d0cab0700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1cf00396a0 0x7f1cf003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:23.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.405+0000 7f1d0cab0700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1cf00396a0 0x7f1cf003bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006fd0 tx=0x7f1cfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:23.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.409+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d0002a020 con 0x7f1d08108560 2024-04-22T13:18:23.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.732+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1d0819fb30 con 0x7f1d08108560 2024-04-22T13:18:23.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.734+0000 7f1d067fc700 1 -- 172.21.15.5:0/2783728779 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f1d0819fb30 con 0x7f1d08108560 2024-04-22T13:18:23.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1cf00396a0 msgr2=0x7f1cf003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:23.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1cf00396a0 0x7f1cf003bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006fd0 tx=0x7f1cfc006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:23.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 msgr2=0x7f1d0819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d0819c4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000bca0 tx=0x7f1d0000bd80 comp rx=0 tx=0).stop 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 shutdown_connections 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1cf00396a0 0x7f1cf003bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 --2- 172.21.15.5:0/2783728779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08108560 0x7f1d0819c4a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 >> 172.21.15.5:0/2783728779 conn(0x7f1d08103b40 msgr2=0x7f1d081046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 shutdown_connections 2024-04-22T13:18:23.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-22T13:18:22.736+0000 7f1d0f515700 1 -- 172.21.15.5:0/2783728779 wait complete. 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-22T13:18:23.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:23.242 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-04-22T13:18:23.270 INFO:tasks.cephadm:Fetching config... 2024-04-22T13:18:23.271 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:18:23.271 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-22T13:18:23.288 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-22T13:18:23.288 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:18:23.288 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-22T13:18:23.345 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-22T13:18:23.346 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:18:23.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-22T13:18:23.411 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-22T13:18:23.411 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:18:23.411 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-22T13:18:23.467 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-22T13:18:23.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T13:18:23.541 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:18:23.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T13:18:23.594 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:18:23.606 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T13:18:23.648 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQapJBiHORp+CcEmZ8niDqnRz/Z6zjp75PujNNZEhlt3XYLPsug0FrZMfT4qB5bLHkzi/ApyA2Gh7QiNBRJHmZk+4XxNlw4q7bWlJgxMOAfTyrPHPx0SDNVmz6bDrV89Sw5+//9s92O/b9bNdXOKjSaz+O8jbXQ54OCM7r88Ww1FxaP9r30XfxJinWpVsp21GjHobuU8e3H7/3KzLkJTOfm5xdUEbFP4Zq4sS92zp4RUBzNfPvbr4GnEMh2T4j7L6ojYMOC8eMXz0BZelLoeCFejE20qiZW2GZSpr7loxv68m9w8vkEAr6+FCQ/EVa5nlMYEIQ53AgRUaWPH9AWzuyxRfYd3uy/6c89Lq2Hxa2bwwqhDvxqZQq95nWaybS5tqkn/qtAxErYbo9YvJbdDeUcMHOxq8nMFHeEB6jck1w9L8+kxg6X1i6upYKlJwH+ohkmvOpzG3GgrowaZxz7p7Ir1oRb/tLjDf8Uo+K2SdDgX3HXgt1RKIpwugE580j7dM= ceph-73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:18:23.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-22T13:18:23.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2783728779' entity='client.admin' 2024-04-22T13:18:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 -- 172.21.15.5:0/3894407659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 msgr2=0x7fc01c1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 --2- 172.21.15.5:0/3894407659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c1017c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc018009910 tx=0x7fc018009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 -- 172.21.15.5:0/3894407659 shutdown_connections 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 --2- 172.21.15.5:0/3894407659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c1017c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 -- 172.21.15.5:0/3894407659 >> 172.21.15.5:0/3894407659 conn(0x7fc01c0fc940 msgr2=0x7fc01c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 -- 172.21.15.5:0/3894407659 shutdown_connections 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 -- 172.21.15.5:0/3894407659 wait complete. 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.106+0000 7fc0245af700 1 Processor -- start 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc0245af700 1 -- start start 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc0245af700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc0245af700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1072e0 con 0x7fc01c1013a0 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc02234b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc02234b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36304/0 (socket says 172.21.15.5:36304) 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc02234b700 1 -- 172.21.15.5:0/3014783928 learned_addr learned my addr 172.21.15.5:0/3014783928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc02234b700 1 -- 172.21.15.5:0/3014783928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c198010 con 0x7fc01c1013a0 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc02234b700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c197ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc018005800 tx=0x7fc018009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc018027070 con 0x7fc01c1013a0 2024-04-22T13:18:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.107+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01800be10 con 0x7fc01c1013a0 2024-04-22T13:18:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.108+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c1982a0 con 0x7fc01c1013a0 2024-04-22T13:18:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.108+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc01800f460 con 0x7fc01c1013a0 2024-04-22T13:18:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.108+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c19af90 con 0x7fc01c1013a0 2024-04-22T13:18:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.108+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fc01802a070 con 0x7fc01c1013a0 2024-04-22T13:18:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.109+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c107a60 con 0x7fc01c1013a0 2024-04-22T13:18:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.109+0000 7fc0137fe700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc008039650 0x7fc00803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.109+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc01801e030 con 0x7fc01c1013a0 2024-04-22T13:18:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.109+0000 7fc021b4a700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc008039650 0x7fc00803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.109+0000 7fc021b4a700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc008039650 0x7fc00803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc00c006fd0 tx=0x7fc00c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.114+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc018013070 con 0x7fc01c1013a0 2024-04-22T13:18:26.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.322+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc01c02e090 con 0x7fc01c1013a0 2024-04-22T13:18:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.324+0000 7fc0137fe700 1 -- 172.21.15.5:0/3014783928 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc018031020 con 0x7fc01c1013a0 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc008039650 msgr2=0x7fc00803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc008039650 0x7fc00803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc00c006fd0 tx=0x7fc00c006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 msgr2=0x7fc01c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c197ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc018005800 tx=0x7fc018009f60 comp rx=0 tx=0).stop 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 shutdown_connections 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc008039650 0x7fc00803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 --2- 172.21.15.5:0/3014783928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c1013a0 0x7fc01c197ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 >> 172.21.15.5:0/3014783928 conn(0x7fc01c0fc940 msgr2=0x7fc01c06a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 shutdown_connections 2024-04-22T13:18:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:26.327+0000 7fc0245af700 1 -- 172.21.15.5:0/3014783928 wait complete. 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3014783928' entity='client.admin' 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:26.974 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-22T13:18:26.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-22T13:18:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 -- 172.21.15.5:0/578054430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 msgr2=0x7f799c1017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 --2- 172.21.15.5:0/578054430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c1017e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7990009910 tx=0x7f7990009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 -- 172.21.15.5:0/578054430 shutdown_connections 2024-04-22T13:18:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 --2- 172.21.15.5:0/578054430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c1017e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 -- 172.21.15.5:0/578054430 >> 172.21.15.5:0/578054430 conn(0x7f799c0fc980 msgr2=0x7f799c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 -- 172.21.15.5:0/578054430 shutdown_connections 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.214+0000 7f79a2ecb700 1 -- 172.21.15.5:0/578054430 wait complete. 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a2ecb700 1 Processor -- start 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a2ecb700 1 -- start start 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a2ecb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a2ecb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f799c107300 con 0x7f799c1013c0 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a0c67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a0c67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c06b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36558/0 (socket says 172.21.15.5:36558) 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.215+0000 7f79a0c67700 1 -- 172.21.15.5:0/2679934554 learned_addr learned my addr 172.21.15.5:0/2679934554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f79a0c67700 1 -- 172.21.15.5:0/2679934554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c19a800 con 0x7f799c1013c0 2024-04-22T13:18:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f79a0c67700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c06b410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7990005800 tx=0x7f7990009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990027070 con 0x7f799c1013c0 2024-04-22T13:18:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799000be10 con 0x7f799c1013c0 2024-04-22T13:18:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c19a9d0 con 0x7f799c1013c0 2024-04-22T13:18:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f799000f460 con 0x7f799c1013c0 2024-04-22T13:18:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.216+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c19adf0 con 0x7f799c1013c0 2024-04-22T13:18:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.217+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f799002a070 con 0x7f799c1013c0 2024-04-22T13:18:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.217+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c107a80 con 0x7f799c1013c0 2024-04-22T13:18:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.217+0000 7f7999ffb700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f798c039700 0x7f798c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.217+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f799001e030 con 0x7f799c1013c0 2024-04-22T13:18:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.217+0000 7f799bfff700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f798c039700 0x7f798c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.218+0000 7f799bfff700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f798c039700 0x7f798c03bbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7988006fd0 tx=0x7f7988006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:29.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.222+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7990013070 con 0x7f799c1013c0 2024-04-22T13:18:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.432+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f799c02e140 con 0x7f798c039700 2024-04-22T13:18:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.436+0000 7f7999ffb700 1 -- 172.21.15.5:0/2679934554 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f799c02e140 con 0x7f798c039700 2024-04-22T13:18:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f798c039700 msgr2=0x7f798c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f798c039700 0x7f798c03bbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7988006fd0 tx=0x7f7988006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 msgr2=0x7f799c06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c06b410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7990005800 tx=0x7f7990009f60 comp rx=0 tx=0).stop 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 shutdown_connections 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f798c039700 0x7f798c03bbc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 --2- 172.21.15.5:0/2679934554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f799c1013c0 0x7f799c06b410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 >> 172.21.15.5:0/2679934554 conn(0x7f799c0fc980 msgr2=0x7f799c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 shutdown_connections 2024-04-22T13:18:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:29.438+0000 7f79a2ecb700 1 -- 172.21.15.5:0/2679934554 wait complete. 2024-04-22T13:18:30.173 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi049 2024-04-22T13:18:30.174 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:18:30.174 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/etc/ceph/ceph.conf 2024-04-22T13:18:30.194 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:18:30.194 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:30.252 INFO:tasks.cephadm:Adding host smithi049 to orchestrator... 2024-04-22T13:18:30.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch host add smithi049 2024-04-22T13:18:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:30.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:30.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:30.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:30.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:30.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:30.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:18:30.719 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:18:30.719 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:30.719 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:30.719 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:30.719 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:31.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:31 smithi005 ceph-mon[28006]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:18:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 -- 172.21.15.5:0/3170465016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 msgr2=0x7f06881088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 --2- 172.21.15.5:0/3170465016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f06881088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0684009910 tx=0x7f0684009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:31.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 -- 172.21.15.5:0/3170465016 shutdown_connections 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 --2- 172.21.15.5:0/3170465016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f06881088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 -- 172.21.15.5:0/3170465016 >> 172.21.15.5:0/3170465016 conn(0x7f06880fdf80 msgr2=0x7f06881003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 -- 172.21.15.5:0/3170465016 shutdown_connections 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 -- 172.21.15.5:0/3170465016 wait complete. 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 Processor -- start 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.887+0000 7f068f65d700 1 -- start start 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068f65d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f0688074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068f65d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f068810c940 con 0x7f06881084e0 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068d3f9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f0688074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068d3f9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f0688074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36576/0 (socket says 172.21.15.5:36576) 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068d3f9700 1 -- 172.21.15.5:0/1360482265 learned_addr learned my addr 172.21.15.5:0/1360482265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068d3f9700 1 -- 172.21.15.5:0/1360482265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0688076c30 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068d3f9700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f0688074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06840056d0 tx=0x7f06840057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684026070 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f068400bc00 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06880752e0 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684003920 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.888+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06880757a0 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.889+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f0684029070 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.889+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688103b90 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.889+0000 7f067e7fc700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0674039650 0x7f067403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.889+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f068401e070 con 0x7f06881084e0 2024-04-22T13:18:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.890+0000 7f068cbf8700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0674039650 0x7f067403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:31.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.890+0000 7f068cbf8700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0674039650 0x7f067403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:31.894+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0684013070 con 0x7f06881084e0 2024-04-22T13:18:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:32.105+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "target": ["mon-mgr", ""]}) -- 0x7f06880762e0 con 0x7f0674039650 2024-04-22T13:18:33.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:33 smithi005 ceph-mon[28006]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:33.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:33 smithi005 ceph-mon[28006]: Deploying cephadm binary to smithi049 2024-04-22T13:18:35.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:35 smithi005 ceph-mon[28006]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.378+0000 7f067e7fc700 1 -- 172.21.15.5:0/1360482265 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f06880762e0 con 0x7f0674039650 2024-04-22T13:18:36.376 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi049' with addr '172.21.15.49' 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0674039650 msgr2=0x7f067403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0674039650 0x7f067403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 msgr2=0x7f0688074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f0688074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06840056d0 tx=0x7f06840057b0 comp rx=0 tx=0).stop 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 shutdown_connections 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0674039650 0x7f067403bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 --2- 172.21.15.5:0/1360482265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06881084e0 0x7f0688074d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 >> 172.21.15.5:0/1360482265 conn(0x7f06880fdf80 msgr2=0x7f0688109ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 shutdown_connections 2024-04-22T13:18:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:36.381+0000 7f068f65d700 1 -- 172.21.15.5:0/1360482265 wait complete. 2024-04-22T13:18:36.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T13:18:37.112 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:37.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:37 smithi005 ceph-mon[28006]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:37.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:37 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:37.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:37 smithi005 ceph-mon[28006]: Added host smithi049 2024-04-22T13:18:37.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:37 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:37.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:37 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/63792779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 msgr2=0x7fd5a0108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 --2- 172.21.15.5:0/63792779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0108930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009910 tx=0x7fd59c009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/63792779 shutdown_connections 2024-04-22T13:18:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 --2- 172.21.15.5:0/63792779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0108930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/63792779 >> 172.21.15.5:0/63792779 conn(0x7fd5a00fdf70 msgr2=0x7fd5a0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/63792779 shutdown_connections 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.612+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/63792779 wait complete. 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a7c4c700 1 Processor -- start 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a7c4c700 1 -- start start 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a7c4c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a7c4c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a010c9c0 con 0x7fd5a0108550 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a59e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a59e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36596/0 (socket says 172.21.15.5:36596) 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.613+0000 7fd5a59e8700 1 -- 172.21.15.5:0/2138747082 learned_addr learned my addr 172.21.15.5:0/2138747082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd5a59e8700 1 -- 172.21.15.5:0/2138747082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a0198400 con 0x7fd5a0108550 2024-04-22T13:18:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd5a59e8700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0197ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0056d0 tx=0x7fd59c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c026070 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59c00bc00 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a0198690 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c003920 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.614+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a0198af0 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.615+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fd59c029070 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.615+0000 7fd596ffd700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd58c0396b0 0x7fd58c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.615+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd59c01e070 con 0x7fd5a0108550 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.615+0000 7fd5a51e7700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd58c0396b0 0x7fd58c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.615+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a0103bc0 con 0x7fd5a0108550 2024-04-22T13:18:38.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.616+0000 7fd5a51e7700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd58c0396b0 0x7fd58c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd590006fd0 tx=0x7fd590006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:38.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.621+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd59c030070 con 0x7fd5a0108550 2024-04-22T13:18:38.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:38 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:38.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.834+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5a0199290 con 0x7fd58c0396b0 2024-04-22T13:18:38.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.835+0000 7fd596ffd700 1 -- 172.21.15.5:0/2138747082 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7fd5a0199290 con 0x7fd58c0396b0 2024-04-22T13:18:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:38.833 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}] 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd58c0396b0 msgr2=0x7fd58c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd58c0396b0 0x7fd58c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd590006fd0 tx=0x7fd590006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 msgr2=0x7fd5a0197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0197ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0056d0 tx=0x7fd59c0057b0 comp rx=0 tx=0).stop 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 shutdown_connections 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd58c0396b0 0x7fd58c03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 --2- 172.21.15.5:0/2138747082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a0108550 0x7fd5a0197ec0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 >> 172.21.15.5:0/2138747082 conn(0x7fd5a00fdf70 msgr2=0x7fd5a00fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 shutdown_connections 2024-04-22T13:18:38.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:38.837+0000 7fd5a7c4c700 1 -- 172.21.15.5:0/2138747082 wait complete. 2024-04-22T13:18:39.435 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi107 2024-04-22T13:18:39.435 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:18:39.435 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.conf 2024-04-22T13:18:39.453 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:18:39.453 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:39.511 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2024-04-22T13:18:39.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch host add smithi107 2024-04-22T13:18:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:39 smithi005 ceph-mon[28006]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:39.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:39 smithi005 ceph-mon[28006]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:18:39.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 -- 172.21.15.5:0/3014238470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 msgr2=0x7f2030101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 --2- 172.21.15.5:0/3014238470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030101990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2024009910 tx=0x7f2024009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 -- 172.21.15.5:0/3014238470 shutdown_connections 2024-04-22T13:18:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 --2- 172.21.15.5:0/3014238470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030101990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 -- 172.21.15.5:0/3014238470 >> 172.21.15.5:0/3014238470 conn(0x7f20300fce40 msgr2=0x7f20300ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 -- 172.21.15.5:0/3014238470 shutdown_connections 2024-04-22T13:18:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.133+0000 7f20350cf700 1 -- 172.21.15.5:0/3014238470 wait complete. 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.134+0000 7f20350cf700 1 Processor -- start 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.134+0000 7f20350cf700 1 -- start start 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.134+0000 7f20350cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.134+0000 7f20350cf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f203010b830 con 0x7f20301015b0 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f202ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f202ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41100/0 (socket says 172.21.15.5:41100) 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f202ed9d700 1 -- 172.21.15.5:0/2116034734 learned_addr learned my addr 172.21.15.5:0/2116034734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f202ed9d700 1 -- 172.21.15.5:0/2116034734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030104f40 con 0x7f20301015b0 2024-04-22T13:18:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f202ed9d700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030103050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2024006010 tx=0x7f2024005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2024026070 con 0x7f20301015b0 2024-04-22T13:18:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202400bc00 con 0x7f20301015b0 2024-04-22T13:18:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20301035f0 con 0x7f20301015b0 2024-04-22T13:18:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2024003910 con 0x7f20301015b0 2024-04-22T13:18:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.135+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2030103ab0 con 0x7f20301015b0 2024-04-22T13:18:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.136+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f2024029070 con 0x7f20301015b0 2024-04-22T13:18:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.136+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2030102a90 con 0x7f20301015b0 2024-04-22T13:18:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.136+0000 7f2017fff700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2018039650 0x7f201803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.136+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f202401e070 con 0x7f20301015b0 2024-04-22T13:18:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.137+0000 7f202e59c700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2018039650 0x7f201803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.137+0000 7f202e59c700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2018039650 0x7f201803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2020006fd0 tx=0x7f2020006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.141+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2024013070 con 0x7f20301015b0 2024-04-22T13:18:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:41.354+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}) -- 0x7f2030104580 con 0x7f2018039650 2024-04-22T13:18:41.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:41 smithi005 ceph-mon[28006]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:42.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:43 smithi005 ceph-mon[28006]: Deploying cephadm binary to smithi107 2024-04-22T13:18:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:43 smithi005 ceph-mon[28006]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:43 smithi005 ceph-mon[28006]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:18:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:43 smithi005 ceph-mon[28006]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:18:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.791+0000 7f2017fff700 1 -- 172.21.15.5:0/2116034734 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2030104580 con 0x7f2018039650 2024-04-22T13:18:45.789 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi107' with addr '172.21.15.107' 2024-04-22T13:18:45.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:45 smithi005 ceph-mon[28006]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2018039650 msgr2=0x7f201803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2018039650 0x7f201803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2020006fd0 tx=0x7f2020006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 msgr2=0x7f2030103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030103050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2024006010 tx=0x7f2024005780 comp rx=0 tx=0).stop 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 shutdown_connections 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2018039650 0x7f201803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 --2- 172.21.15.5:0/2116034734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301015b0 0x7f2030103050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 >> 172.21.15.5:0/2116034734 conn(0x7f20300fce40 msgr2=0x7f203006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 shutdown_connections 2024-04-22T13:18:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:45.794+0000 7f20350cf700 1 -- 172.21.15.5:0/2116034734 wait complete. 2024-04-22T13:18:46.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T13:18:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:46.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: Added host smithi107 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:46.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:46 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:47.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:47 smithi005 ceph-mon[28006]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:47.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 -- 172.21.15.5:0/209737078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 msgr2=0x7f81ec1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 --2- 172.21.15.5:0/209737078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec1019d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009910 tx=0x7f81dc009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 -- 172.21.15.5:0/209737078 shutdown_connections 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 --2- 172.21.15.5:0/209737078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec1019d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 -- 172.21.15.5:0/209737078 >> 172.21.15.5:0/209737078 conn(0x7f81ec0fcea0 msgr2=0x7f81ec0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 -- 172.21.15.5:0/209737078 shutdown_connections 2024-04-22T13:18:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.216+0000 7f81f12ed700 1 -- 172.21.15.5:0/209737078 wait complete. 2024-04-22T13:18:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81f12ed700 1 Processor -- start 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81f12ed700 1 -- start start 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81f12ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81f12ed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec10b870 con 0x7f81ec1015f0 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41110/0 (socket says 172.21.15.5:41110) 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.217+0000 7f81eaffd700 1 -- 172.21.15.5:0/4004633698 learned_addr learned my addr 172.21.15.5:0/4004633698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81eaffd700 1 -- 172.21.15.5:0/4004633698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec198420 con 0x7f81ec1015f0 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81eaffd700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec197ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0056d0 tx=0x7f81dc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81dc026070 con 0x7f81ec1015f0 2024-04-22T13:18:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec198710 con 0x7f81ec1015f0 2024-04-22T13:18:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81dc00bba0 con 0x7f81ec1015f0 2024-04-22T13:18:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81dc003920 con 0x7f81ec1015f0 2024-04-22T13:18:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.218+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec19c7b0 con 0x7f81ec1015f0 2024-04-22T13:18:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.219+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f81dc029070 con 0x7f81ec1015f0 2024-04-22T13:18:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.219+0000 7f81d3fff700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81d4039650 0x7f81d403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.219+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ec102ad0 con 0x7f81ec1015f0 2024-04-22T13:18:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.219+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f81dc01e070 con 0x7f81ec1015f0 2024-04-22T13:18:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.219+0000 7f81ea7fc700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81d4039650 0x7f81d403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.219+0000 7f81ea7fc700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81d4039650 0x7f81d403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81e0006fd0 tx=0x7f81e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.224+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81dc013070 con 0x7f81ec1015f0 2024-04-22T13:18:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.438+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81ec199240 con 0x7f81d4039650 2024-04-22T13:18:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.439+0000 7f81d3fff700 1 -- 172.21.15.5:0/4004633698 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f81ec199240 con 0x7f81d4039650 2024-04-22T13:18:48.436 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:18:48.437 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}] 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81d4039650 msgr2=0x7f81d403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81d4039650 0x7f81d403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81e0006fd0 tx=0x7f81e0006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 msgr2=0x7f81ec197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec197ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0056d0 tx=0x7f81dc0057b0 comp rx=0 tx=0).stop 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 shutdown_connections 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81d4039650 0x7f81d403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 --2- 172.21.15.5:0/4004633698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81ec1015f0 0x7f81ec197ee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 >> 172.21.15.5:0/4004633698 conn(0x7f81ec0fcea0 msgr2=0x7f81ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 shutdown_connections 2024-04-22T13:18:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:48.441+0000 7f81f12ed700 1 -- 172.21.15.5:0/4004633698 wait complete. 2024-04-22T13:18:49.033 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-22T13:18:49.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-22T13:18:49.278 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:49.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:49 smithi005 ceph-mon[28006]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:49.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:49.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:49 smithi005 ceph-mon[28006]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:18:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 -- 172.21.15.5:0/523014698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 msgr2=0x7fb1b81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 --2- 172.21.15.5:0/523014698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b81088c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009910 tx=0x7fb1a8009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 -- 172.21.15.5:0/523014698 shutdown_connections 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 --2- 172.21.15.5:0/523014698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b81088c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 -- 172.21.15.5:0/523014698 >> 172.21.15.5:0/523014698 conn(0x7fb1b80fdf80 msgr2=0x7fb1b81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 -- 172.21.15.5:0/523014698 shutdown_connections 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.816+0000 7fb1be5e8700 1 -- 172.21.15.5:0/523014698 wait complete. 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1be5e8700 1 Processor -- start 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1be5e8700 1 -- start start 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1be5e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1be5e8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810c940 con 0x7fb1b81084e0 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42308/0 (socket says 172.21.15.5:42308) 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1b7fff700 1 -- 172.21.15.5:0/1228701698 learned_addr learned my addr 172.21.15.5:0/1228701698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.817+0000 7fb1b7fff700 1 -- 172.21.15.5:0/1228701698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b8076c30 con 0x7fb1b81084e0 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.818+0000 7fb1b7fff700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b8074d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80056d0 tx=0x7fb1a80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.818+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1a8026070 con 0x7fb1b81084e0 2024-04-22T13:18:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.818+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a800bc00 con 0x7fb1b81084e0 2024-04-22T13:18:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.818+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b80752e0 con 0x7fb1b81084e0 2024-04-22T13:18:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.818+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1a8003920 con 0x7fb1b81084e0 2024-04-22T13:18:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.818+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b80757a0 con 0x7fb1b81084e0 2024-04-22T13:18:50.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.819+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fb1a8029070 con 0x7fb1b81084e0 2024-04-22T13:18:50.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.819+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8103b90 con 0x7fb1b81084e0 2024-04-22T13:18:50.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.819+0000 7fb1b57fa700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb1a4039650 0x7fb1a403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:50.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.819+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb1a801e070 con 0x7fb1b81084e0 2024-04-22T13:18:50.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.819+0000 7fb1b77fe700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb1a4039650 0x7fb1a403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:50.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.820+0000 7fb1b77fe700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb1a4039650 0x7fb1a403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0006fd0 tx=0x7fb1a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:50.824+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1a8013070 con 0x7fb1b81084e0 2024-04-22T13:18:51.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.038+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb1b8076270 con 0x7fb1b81084e0 2024-04-22T13:18:51.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.352+0000 7fb1b57fa700 1 -- 172.21.15.5:0/1228701698 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb1a8030090 con 0x7fb1b81084e0 2024-04-22T13:18:51.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb1a4039650 msgr2=0x7fb1a403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:51.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb1a4039650 0x7fb1a403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0006fd0 tx=0x7fb1a0006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:51.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 msgr2=0x7fb1b8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b8074d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80056d0 tx=0x7fb1a80057b0 comp rx=0 tx=0).stop 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 shutdown_connections 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb1a4039650 0x7fb1a403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 --2- 172.21.15.5:0/1228701698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b81084e0 0x7fb1b8074d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 >> 172.21.15.5:0/1228701698 conn(0x7fb1b80fdf80 msgr2=0x7fb1b810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 shutdown_connections 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:18:51.356+0000 7fb1be5e8700 1 -- 172.21.15.5:0/1228701698 wait complete. 2024-04-22T13:18:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-04-22T13:18:51.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:51 smithi005 ceph-mon[28006]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:51.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:51 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1228701698' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T13:18:51.957 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-04-22T13:18:51.957 INFO:tasks.cephadm:Adding mon.b on smithi049 2024-04-22T13:18:51.957 INFO:tasks.cephadm:Adding mon.c on smithi107 2024-04-22T13:18:51.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c' 2024-04-22T13:18:52.195 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:18:52.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1228701698' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:18:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:52 smithi005 ceph-mon[28006]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:18:53.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 -- 172.21.15.107:0/1657298428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 msgr2=0x7fe2001088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 --2- 172.21.15.107:0/1657298428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe2001088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009910 tx=0x7fe1f0009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 -- 172.21.15.107:0/1657298428 shutdown_connections 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 --2- 172.21.15.107:0/1657298428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe2001088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 -- 172.21.15.107:0/1657298428 >> 172.21.15.107:0/1657298428 conn(0x7fe2000fdf80 msgr2=0x7fe2001003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 -- 172.21.15.107:0/1657298428 shutdown_connections 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.814+0000 7fe2057a7700 1 -- 172.21.15.107:0/1657298428 wait complete. 2024-04-22T13:18:53.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe2057a7700 1 Processor -- start 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe2057a7700 1 -- start start 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe2057a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe200074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe2057a7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe20010c940 con 0x7fe2001084e0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe1feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe200074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe1feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe200074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.107:40942/0 (socket says 172.21.15.107:40942) 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.815+0000 7fe1feffd700 1 -- 172.21.15.107:0/2996697704 learned_addr learned my addr 172.21.15.107:0/2996697704 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe1feffd700 1 -- 172.21.15.107:0/2996697704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe200076c30 con 0x7fe2001084e0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe1feffd700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe200074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00056d0 tx=0x7fe1f00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f0026070 con 0x7fe2001084e0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2000752e0 con 0x7fe2001084e0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f000bc00 con 0x7fe2001084e0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f0003920 con 0x7fe2001084e0 2024-04-22T13:18:53.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.816+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2000757a0 con 0x7fe2001084e0 2024-04-22T13:18:53.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.817+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe200103b90 con 0x7fe2001084e0 2024-04-22T13:18:53.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.817+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fe1f0029070 con 0x7fe2001084e0 2024-04-22T13:18:53.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.817+0000 7fe1fcff9700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1ec039650 0x7fe1ec03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:53.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.817+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe1f001e070 con 0x7fe2001084e0 2024-04-22T13:18:53.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.818+0000 7fe1f67fc700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1ec039650 0x7fe1ec03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:53.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.818+0000 7fe1f67fc700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1ec039650 0x7fe1ec03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:53.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:53.823+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1f0013070 con 0x7fe2001084e0 2024-04-22T13:18:54.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.031+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c", "target": ["mon-mgr", ""]}) -- 0x7fe200076270 con 0x7fe1ec039650 2024-04-22T13:18:54.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.038+0000 7fe1fcff9700 1 -- 172.21.15.107:0/2996697704 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe200076270 con 0x7fe1ec039650 2024-04-22T13:18:54.035 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled mon update... 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1ec039650 msgr2=0x7fe1ec03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1ec039650 0x7fe1ec03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 msgr2=0x7fe200074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe200074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00056d0 tx=0x7fe1f00057b0 comp rx=0 tx=0).stop 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 shutdown_connections 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1ec039650 0x7fe1ec03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 --2- 172.21.15.107:0/2996697704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2001084e0 0x7fe200074d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 >> 172.21.15.107:0/2996697704 conn(0x7fe2000fdf80 msgr2=0x7fe20010a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 shutdown_connections 2024-04-22T13:18:54.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:54.040+0000 7fe2057a7700 1 -- 172.21.15.107:0/2996697704 wait complete. 2024-04-22T13:18:54.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:53 smithi005 ceph-mon[28006]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:53 smithi005 ceph-mon[28006]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:53 smithi005 ceph-mon[28006]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:18:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:54.636 DEBUG:teuthology.orchestra.run.smithi049:mon.b> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.b.service 2024-04-22T13:18:54.639 DEBUG:teuthology.orchestra.run.smithi107:mon.c> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.c.service 2024-04-22T13:18:54.641 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T13:18:54.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c;count:3 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: Deploying daemon mon.c on smithi107 2024-04-22T13:18:55.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:55 smithi005 ceph-mon[28006]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:56.735 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:18:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:18:57 smithi005 ceph-mon[28006]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 -- 172.21.15.107:0/3494752291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 msgr2=0x7f5b68103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 --2- 172.21.15.107:0/3494752291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68103af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009910 tx=0x7f5b58009c20 comp rx=0 tx=0).stop 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 -- 172.21.15.107:0/3494752291 shutdown_connections 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 --2- 172.21.15.107:0/3494752291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68103af0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 -- 172.21.15.107:0/3494752291 >> 172.21.15.107:0/3494752291 conn(0x7f5b680fcf80 msgr2=0x7f5b680ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 -- 172.21.15.107:0/3494752291 shutdown_connections 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 -- 172.21.15.107:0/3494752291 wait complete. 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.590+0000 7f5b6daed700 1 Processor -- start 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b6daed700 1 -- start start 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b6daed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:58.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b6daed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810cab0 con 0x7f5b68103710 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.107:59898/0 (socket says 172.21.15.107:59898) 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b677fe700 1 -- 172.21.15.107:0/2693827131 learned_addr learned my addr 172.21.15.107:0/2693827131 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.591+0000 7f5b677fe700 1 -- 172.21.15.107:0/2693827131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b68198580 con 0x7f5b68103710 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.592+0000 7f5b677fe700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68198040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5b580056d0 tx=0x7f5b580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.592+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b58026070 con 0x7f5b68103710 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.592+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b68198810 con 0x7f5b68103710 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.592+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b58004370 con 0x7f5b68103710 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.592+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b580039b0 con 0x7f5b68103710 2024-04-22T13:18:58.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.592+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b68198c70 con 0x7f5b68103710 2024-04-22T13:18:58.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.593+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f5b58029070 con 0x7f5b68103710 2024-04-22T13:18:58.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.593+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6806b530 con 0x7f5b68103710 2024-04-22T13:18:58.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.593+0000 7f5b64ff9700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5b50039650 0x7f5b5003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:58.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.593+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b5801e070 con 0x7f5b68103710 2024-04-22T13:18:58.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.593+0000 7f5b66ffd700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5b50039650 0x7f5b5003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:58.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.594+0000 7f5b66ffd700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5b50039650 0x7f5b5003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006fd0 tx=0x7f5b5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:58.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.599+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b58013070 con 0x7f5b68103710 2024-04-22T13:18:58.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.889+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5b681993a0 con 0x7f5b68103710 2024-04-22T13:18:58.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.890+0000 7f5b64ff9700 1 -- 172.21.15.107:0/2693827131 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f5b58030090 con 0x7f5b68103710 2024-04-22T13:18:58.888 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:18:58.888 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","modified":"2024-04-22T13:17:09.176897Z","created":"2024-04-22T13:17:09.176897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5b50039650 msgr2=0x7f5b5003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5b50039650 0x7f5b5003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006fd0 tx=0x7f5b5c006e40 comp rx=0 tx=0).stop 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 msgr2=0x7f5b68198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68198040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5b580056d0 tx=0x7f5b580057b0 comp rx=0 tx=0).stop 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 shutdown_connections 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5b50039650 0x7f5b5003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:58.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 --2- 172.21.15.107:0/2693827131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68103710 0x7f5b68198040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 >> 172.21.15.107:0/2693827131 conn(0x7f5b680fcf80 msgr2=0x7f5b681064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 shutdown_connections 2024-04-22T13:18:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:18:58.892+0000 7f5b6daed700 1 -- 172.21.15.107:0/2693827131 wait complete. 2024-04-22T13:18:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 1 2024-04-22T13:18:59.028 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 podman[31829]: 2024-04-22 13:18:59.007435002 +0000 UTC m=+0.361231145 container start 47e7084a938f70fdcbad07f9bbb6d6cc4e0fae233dcd42ab497a1183ec714822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-22T13:18:59.029 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T13:18:59.029 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-22T13:18:59.029 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pidfile_write: ignore empty --pid-file 2024-04-22T13:18:59.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: load: jerasure load: lrc 2024-04-22T13:18:59.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: RocksDB version: 7.9.2 2024-04-22T13:18:59.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Git sha 0 2024-04-22T13:18:59.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: DB SUMMARY 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: DB Session ID: A492BYEM1S8133TDJMOI 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: CURRENT file: CURRENT 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.error_if_exists: 0 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.create_if_missing: 0 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.paranoid_checks: 1 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T13:18:59.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.env: 0x55c108893cc0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.info_log: 0x55c10a075be0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.statistics: (nil) 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.use_fsync: 0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_log_file_size: 0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.allow_fallocate: 1 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T13:18:59.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.use_direct_reads: 0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.db_log_dir: 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.wal_dir: 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.write_buffer_manager: 0x55c10a0090e0 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T13:18:59.289 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.unordered_write: 0 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.row_cache: None 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.wal_filter: None 2024-04-22T13:18:59.290 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.two_write_queues: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.wal_compression: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.atomic_flush: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.log_readahead_size: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T13:18:59.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T13:18:59.292 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_background_jobs: 2 2024-04-22T13:18:59.292 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_background_compactions: -1 2024-04-22T13:18:59.292 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_subcompactions: 1 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T13:18:59.293 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_open_files: -1 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_background_flushes: -1 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Compression algorithms supported: 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kZSTD supported: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kXpressCompression supported: 0 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kZlibCompression supported: 1 2024-04-22T13:18:59.294 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kSnappyCompression supported: 1 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kLZ4Compression supported: 1 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: kBZip2Compression supported: 0 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.merge_operator: 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_filter: None 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c10a075dc0) 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-04-22T13:18:59.295 INFO:journalctl@ceph.mon.c.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_type: 0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: data_block_index_type: 0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_shortening: 1 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: checksum: 4 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: no_block_cache: 0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache: 0x55c10a03b1f0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_options: 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: capacity : 536870912 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: num_shard_bits : 4 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: strict_capacity_limit : 0 2024-04-22T13:18:59.296 INFO:journalctl@ceph.mon.c.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_compressed: (nil) 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: persistent_cache: (nil) 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_size: 4096 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_size_deviation: 10 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_restart_interval: 16 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_block_restart_interval: 1 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: metadata_block_size: 4096 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: partition_filters: 0 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: use_delta_encoding: 1 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: filter_policy: bloomfilter 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: whole_key_filtering: 1 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: verify_compression: 0 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: format_version: 5 2024-04-22T13:18:59.297 INFO:journalctl@ceph.mon.c.smithi107.stdout: enable_index_compression: 1 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_align: 0 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout: max_auto_readahead_size: 262144 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout: prepopulate_block_cache: 0 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout: initial_auto_readahead_size: 8192 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression: NoCompression 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.num_levels: 7 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T13:18:59.298 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T13:18:59.299 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T13:18:59.300 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T13:18:59.301 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.inplace_update_support: 0 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T13:18:59.302 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.bloom_locality: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.max_successive_merges: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.ttl: 2592000 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.enable_blob_files: false 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.min_blob_size: 0 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T13:18:59.303 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 67078faa-00d9-4c86-974b-16dc5369bd79 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791939038148, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791939038714, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713791939, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "67078faa-00d9-4c86-974b-16dc5369bd79", "db_session_id": "A492BYEM1S8133TDJMOI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791939038818, "job": 1, "event": "recovery_finished"} 2024-04-22T13:18:59.304 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c10a150000 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: DB pointer 0x55c10a13a000 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** DB Stats ** 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-04-22T13:18:59.305 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:18:59.306 INFO:journalctl@ceph.mon.c.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: Block cache BinnedLRUCache@0x55c10a03b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-22T13:18:59.307 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: using public_addr v2:172.21.15.107:0/0 -> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: starting mon.c rank -1 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(???) e0 preinit fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).mds e1 new map 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout: e1 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout: legacy client fscid: -1 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout: No filesystems configured 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-22T13:18:59.308 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Added host smithi107 2024-04-22T13:18:59.309 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1228701698' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1228701698' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.310 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c;count:3 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:59.311 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: Deploying daemon mon.c on smithi107 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 ceph-mon[31859]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 bash[31829]: 47e7084a938f70fdcbad07f9bbb6d6cc4e0fae233dcd42ab497a1183ec714822 2024-04-22T13:18:59.312 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:18:59 smithi107 systemd[1]: Started Ceph mon.c for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:19:00.718 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T13:19:00.718 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T13:19:00.961 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:19:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:19:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: mon.a calling monitor election 2024-04-22T13:19:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: mon.c calling monitor election 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: monmap epoch 2 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: last_changed 2024-04-22T13:18:59.064184+0000 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: min_mon_release 19 (squid) 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: election_strategy: 1 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:04.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:04.538 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: fsmap 2024-04-22T13:19:04.538 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:04.538 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: mgrmap e13: x(active, since 49s) 2024-04-22T13:19:04.538 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: overall HEALTH_OK 2024-04-22T13:19:04.538 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:04.538 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: mon.a calling monitor election 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: mon.c calling monitor election 2024-04-22T13:19:04.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: monmap epoch 2 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: last_changed 2024-04-22T13:18:59.064184+0000 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: min_mon_release 19 (squid) 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: election_strategy: 1 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: fsmap 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: mgrmap e13: x(active, since 49s) 2024-04-22T13:19:04.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: overall HEALTH_OK 2024-04-22T13:19:04.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:04.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.319+0000 7f915aec5700 1 -- 172.21.15.107:0/2967527650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 msgr2=0x7f9144005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.319+0000 7f915aec5700 1 --2- 172.21.15.107:0/2967527650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9144005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9154102490 tx=0x7f914800feb0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.320+0000 7f915aec5700 1 -- 172.21.15.107:0/2967527650 shutdown_connections 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.320+0000 7f915aec5700 1 --2- 172.21.15.107:0/2967527650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9144005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.320+0000 7f915aec5700 1 -- 172.21.15.107:0/2967527650 >> 172.21.15.107:0/2967527650 conn(0x7f91540fe050 msgr2=0x7f9154100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.320+0000 7f915aec5700 1 -- 172.21.15.107:0/2967527650 shutdown_connections 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.320+0000 7f915aec5700 1 -- 172.21.15.107:0/2967527650 wait complete. 2024-04-22T13:19:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f915aec5700 1 Processor -- start 2024-04-22T13:19:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f915aec5700 1 -- start start 2024-04-22T13:19:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f915aec5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9154197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f915aec5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f915aec5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f915410a4b0 con 0x7f91440035c0 2024-04-22T13:19:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f915aec5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f915410a330 con 0x7f9154198500 2024-04-22T13:19:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9153fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9158c61700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9154197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9158c61700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9154197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.107:59970/0 (socket says 172.21.15.107:59970) 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9158c61700 1 -- 172.21.15.107:0/3938707989 learned_addr learned my addr 172.21.15.107:0/3938707989 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9153fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48956/0 (socket says 172.21.15.107:48956) 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9153fff700 1 -- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 msgr2=0x7f915419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9153fff700 1 -- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 msgr2=0x7f915419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9153fff700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9153fff700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9158c61700 1 -- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 msgr2=0x7f915419c7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9158c61700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.321+0000 7f9158c61700 1 -- 172.21.15.107:0/3938707989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9154198b40 con 0x7f91440035c0 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.322+0000 7f9158c61700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9154197fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9148001eb0 tx=0x7f91480050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.322+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9148004ac0 con 0x7f91440035c0 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.322+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9148004030 con 0x7f91440035c0 2024-04-22T13:19:05.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.322+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915419cd30 con 0x7f91440035c0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.322+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9148011b80 con 0x7f91440035c0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.322+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915419d310 con 0x7f91440035c0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.323+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f914802c070 con 0x7f91440035c0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.323+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154103ca0 con 0x7f91440035c0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.324+0000 7f9151ffb700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91380393f0 0x7f913803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.324+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f914802a070 con 0x7f91440035c0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.324+0000 7f9153fff700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91380393f0 0x7f913803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:05.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.324+0000 7f9153fff700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91380393f0 0x7f913803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9140009850 tx=0x7f9140008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:05.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.329+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f914801ac90 con 0x7f91440035c0 2024-04-22T13:19:05.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:05 smithi107 ceph-mon[31859]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:05.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:05.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:05.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:05.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:05 smithi107 ceph-mon[31859]: Deploying daemon mon.b on smithi049 2024-04-22T13:19:05.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:05.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:05 smithi005 ceph-mon[28006]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:05.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:05.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:05.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:05.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:05 smithi005 ceph-mon[28006]: Deploying daemon mon.b on smithi049 2024-04-22T13:19:05.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:05.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.615+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f915419d5c0 con 0x7f91440035c0 2024-04-22T13:19:05.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.615+0000 7f9151ffb700 1 -- 172.21.15.107:0/3938707989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f91480044e0 con 0x7f91440035c0 2024-04-22T13:19:05.613 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:19:05.613 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","modified":"2024-04-22T13:18:59.064184Z","created":"2024-04-22T13:17:09.176897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T13:19:05.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.617+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91380393f0 msgr2=0x7f913803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:05.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.617+0000 7f915aec5700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91380393f0 0x7f913803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9140009850 tx=0x7f9140008040 comp rx=0 tx=0).stop 2024-04-22T13:19:05.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 msgr2=0x7f9154197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:05.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9154197fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9148001eb0 tx=0x7f91480050d0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 shutdown_connections 2024-04-22T13:19:05.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91380393f0 0x7f913803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440035c0 0x7f9154197fc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 --2- 172.21.15.107:0/3938707989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154198500 0x7f915419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:05.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 >> 172.21.15.107:0/3938707989 conn(0x7f91540fe050 msgr2=0x7f91540fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:05.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 shutdown_connections 2024-04-22T13:19:05.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:05.618+0000 7f915aec5700 1 -- 172.21.15.107:0/3938707989 wait complete. 2024-04-22T13:19:05.620 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 2 2024-04-22T13:19:06.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:06 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/3938707989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:19:06.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:06 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/3938707989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:19:07.190 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T13:19:07.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T13:19:07.434 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:19:07.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:07 smithi107 ceph-mon[31859]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:07.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:07 smithi005 ceph-mon[28006]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:07.766 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 podman[31360]: 2024-04-22 13:19:07.566417781 +0000 UTC m=+0.018682201 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:19:07.766 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 podman[31360]: 2024-04-22 13:19:07.769486947 +0000 UTC m=+0.221751385 container create b37e1c5be6ecbdb9bb8ad023ab83784313f5e9b4db85a191494f49244c8cff54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:19:08.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 podman[31360]: 2024-04-22 13:19:07.938700715 +0000 UTC m=+0.390965118 container init b37e1c5be6ecbdb9bb8ad023ab83784313f5e9b4db85a191494f49244c8cff54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 podman[31360]: 2024-04-22 13:19:07.94286177 +0000 UTC m=+0.395126165 container start b37e1c5be6ecbdb9bb8ad023ab83784313f5e9b4db85a191494f49244c8cff54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pidfile_write: ignore empty --pid-file 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: load: jerasure load: lrc 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: RocksDB version: 7.9.2 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Git sha 0 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: DB SUMMARY 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: DB Session ID: NVM38ZH748ATGGIZ366H 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: CURRENT file: CURRENT 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T13:19:08.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.error_if_exists: 0 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.create_if_missing: 0 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.paranoid_checks: 1 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.env: 0x55e411d2fcc0 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.info_log: 0x55e414313be0 2024-04-22T13:19:08.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.statistics: (nil) 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.use_fsync: 0 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_log_file_size: 0 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T13:19:08.026 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.allow_fallocate: 1 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.use_direct_reads: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.db_log_dir: 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.wal_dir: 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T13:19:08.027 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.write_buffer_manager: 0x55e4142a70e0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.unordered_write: 0 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T13:19:08.028 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.row_cache: None 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.wal_filter: None 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.two_write_queues: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.wal_compression: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.atomic_flush: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.log_readahead_size: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T13:19:08.029 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_background_jobs: 2 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_background_compactions: -1 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_subcompactions: 1 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T13:19:08.030 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_open_files: -1 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_background_flushes: -1 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Compression algorithms supported: 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kZSTD supported: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kXpressCompression supported: 0 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kZlibCompression supported: 1 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kSnappyCompression supported: 1 2024-04-22T13:19:08.031 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kLZ4Compression supported: 1 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: kBZip2Compression supported: 0 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.merge_operator: 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_filter: None 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e414313dc0) 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-04-22T13:19:08.032 INFO:journalctl@ceph.mon.b.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: index_type: 0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: data_block_index_type: 0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: index_shortening: 1 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: checksum: 4 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: no_block_cache: 0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_cache: 0x55e4142d91f0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_cache_options: 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: capacity : 536870912 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: num_shard_bits : 4 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: strict_capacity_limit : 0 2024-04-22T13:19:08.033 INFO:journalctl@ceph.mon.b.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_cache_compressed: (nil) 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: persistent_cache: (nil) 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_size: 4096 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_size_deviation: 10 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_restart_interval: 16 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: index_block_restart_interval: 1 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: metadata_block_size: 4096 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: partition_filters: 0 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: use_delta_encoding: 1 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: filter_policy: bloomfilter 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: whole_key_filtering: 1 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: verify_compression: 0 2024-04-22T13:19:08.034 INFO:journalctl@ceph.mon.b.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: format_version: 5 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: enable_index_compression: 1 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: block_align: 0 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: max_auto_readahead_size: 262144 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: prepopulate_block_cache: 0 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: initial_auto_readahead_size: 8192 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression: NoCompression 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.num_levels: 7 2024-04-22T13:19:08.035 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T13:19:08.036 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T13:19:08.037 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T13:19:08.038 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.inplace_update_support: 0 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.bloom_locality: 0 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.max_successive_merges: 0 2024-04-22T13:19:08.039 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.ttl: 2592000 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.enable_blob_files: false 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.min_blob_size: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T13:19:08.040 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 295a3a3c-b6df-480a-8bc9-0f9ccfcf3f20 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791947975480, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791947976104, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713791947, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "295a3a3c-b6df-480a-8bc9-0f9ccfcf3f20", "db_session_id": "NVM38ZH748ATGGIZ366H", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791947976208, "job": 1, "event": "recovery_finished"} 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e4143ee000 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: DB pointer 0x55e4143d8000 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T13:19:08.041 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: ** DB Stats ** 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: ** Compaction Stats [default] ** 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:19:08.042 INFO:journalctl@ceph.mon.b.smithi049.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: ** Compaction Stats [default] ** 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T13:19:08.043 INFO:journalctl@ceph.mon.b.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: Block cache BinnedLRUCache@0x55e4142d91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-22T13:19:08.044 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: using public_addr v2:172.21.15.49:0/0 -> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: starting mon.b rank -1 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(???) e0 preinit fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).mds e1 new map 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout: e1 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout: legacy client fscid: -1 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout: 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout: No filesystems configured 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-22T13:19:08.045 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.046 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Added host smithi107 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1228701698' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1228701698' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.047 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c;count:3 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:08.048 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Deploying daemon mon.c on smithi107 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.a calling monitor election 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.049 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.c calling monitor election 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: monmap epoch 2 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: last_changed 2024-04-22T13:18:59.064184+0000 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: min_mon_release 19 (squid) 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: election_strategy: 1 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: fsmap 2024-04-22T13:19:08.050 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mgrmap e13: x(active, since 49s) 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: overall HEALTH_OK 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: Deploying daemon mon.b on smithi049 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/3938707989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:07 smithi049 ceph-mon[31379]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:08 smithi049 bash[31360]: b37e1c5be6ecbdb9bb8ad023ab83784313f5e9b4db85a191494f49244c8cff54 2024-04-22T13:19:08.051 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:08 smithi049 systemd[1]: Started Ceph mon.b for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:19:13.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.237+0000 7fd43d04e700 1 -- 172.21.15.107:0/1607284824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 msgr2=0x7fd420005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.237+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1607284824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd420005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd438078c80 tx=0x7fd42c010e70 comp rx=0 tx=0).stop 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.238+0000 7fd43d04e700 1 -- 172.21.15.107:0/1607284824 shutdown_connections 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.238+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1607284824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd420005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.238+0000 7fd43d04e700 1 -- 172.21.15.107:0/1607284824 >> 172.21.15.107:0/1607284824 conn(0x7fd438077920 msgr2=0x7fd438077d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.238+0000 7fd43d04e700 1 -- 172.21.15.107:0/1607284824 shutdown_connections 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.238+0000 7fd43d04e700 1 -- 172.21.15.107:0/1607284824 wait complete. 2024-04-22T13:19:13.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.239+0000 7fd43d04e700 1 Processor -- start 2024-04-22T13:19:13.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.239+0000 7fd43d04e700 1 -- start start 2024-04-22T13:19:13.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.239+0000 7fd43d04e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd438198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:13.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.239+0000 7fd43d04e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:13.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.239+0000 7fd43d04e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 0x7fd43819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:13.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43d04e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd438108240 con 0x7fd4200035c0 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43d04e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381080c0 con 0x7fd43819cf30 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43d04e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381083c0 con 0x7fd438198660 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd438198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48366/0 (socket says 172.21.15.107:48366) 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 -- 172.21.15.107:0/1335905522 learned_addr learned my addr 172.21.15.107:0/1335905522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43759e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 0x7fd43819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 msgr2=0x7fd43819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 msgr2=0x7fd43819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43659c700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43759e700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 msgr2=0x7fd43819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-22T13:19:13.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43759e700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 msgr2=0x7fd43819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43759e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 0x7fd43819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd43759e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 0x7fd43819d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 msgr2=0x7fd43819c9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 msgr2=0x7fd43819d3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 0x7fd43819d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 -- 172.21.15.107:0/1335905522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4381a4ca0 con 0x7fd4200035c0 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.240+0000 7fd436d9d700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd438198120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd42c011010 tx=0x7fd42c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.241+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c02a020 con 0x7fd4200035c0 2024-04-22T13:19:13.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.241+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42c0056a0 con 0x7fd4200035c0 2024-04-22T13:19:13.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.241+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381a4e70 con 0x7fd4200035c0 2024-04-22T13:19:13.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.241+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c022040 con 0x7fd4200035c0 2024-04-22T13:19:13.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.241+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a5320 con 0x7fd4200035c0 2024-04-22T13:19:13.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.242+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fd42c02d020 con 0x7fd4200035c0 2024-04-22T13:19:13.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.242+0000 7fd41ffff700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd424039ae0 0x7fd42403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:13.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.242+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4380640a0 con 0x7fd4200035c0 2024-04-22T13:19:13.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.242+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd42c012b90 con 0x7fd4200035c0 2024-04-22T13:19:13.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.242+0000 7fd43659c700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd424039ae0 0x7fd42403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:13.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.243+0000 7fd43659c700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd424039ae0 0x7fd42403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd428009180 tx=0x7fd428008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:13.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.248+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd42c034020 con 0x7fd4200035c0 2024-04-22T13:19:13.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.533+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd43802df80 con 0x7fd4200035c0 2024-04-22T13:19:13.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.534+0000 7fd41ffff700 1 -- 172.21.15.107:0/1335905522 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fd42c02c2c0 con 0x7fd4200035c0 2024-04-22T13:19:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T13:19:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","modified":"2024-04-22T13:19:08.002976Z","created":"2024-04-22T13:17:09.176897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T13:19:13.532 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:19:13.532 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: mon.c calling monitor election 2024-04-22T13:19:13.532 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: mon.a calling monitor election 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: monmap epoch 3 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: last_changed 2024-04-22T13:19:08.002976+0000 2024-04-22T13:19:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: min_mon_release 19 (squid) 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: election_strategy: 1 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: fsmap 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: mgrmap e13: x(active, since 58s) 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: overall HEALTH_OK 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd424039ae0 msgr2=0x7fd42403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:13.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd424039ae0 0x7fd42403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd428009180 tx=0x7fd428008040 comp rx=0 tx=0).stop 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 msgr2=0x7fd438198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd438198120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd42c011010 tx=0x7fd42c003d00 comp rx=0 tx=0).stop 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 shutdown_connections 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd424039ae0 0x7fd42403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd43819cf30 0x7fd43819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4200035c0 0x7fd438198120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 --2- 172.21.15.107:0/1335905522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd438198660 0x7fd43819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 >> 172.21.15.107:0/1335905522 conn(0x7fd438077920 msgr2=0x7fd4381050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 shutdown_connections 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:13.536+0000 7fd43d04e700 1 -- 172.21.15.107:0/1335905522 wait complete. 2024-04-22T13:19:13.536 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 3 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: mon.c calling monitor election 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: mon.a calling monitor election 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: monmap epoch 3 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: last_changed 2024-04-22T13:19:08.002976+0000 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: min_mon_release 19 (squid) 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: election_strategy: 1 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: fsmap 2024-04-22T13:19:13.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:13.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: mgrmap e13: x(active, since 58s) 2024-04-22T13:19:13.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: overall HEALTH_OK 2024-04-22T13:19:13.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:13.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:14.061 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-22T13:19:14.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-22T13:19:14.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:14.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/1335905522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:19:14.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:14.319 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:19:14.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:14.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/1335905522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:19:14.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.c calling monitor election 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.a calling monitor election 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:19:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: monmap epoch 3 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: last_changed 2024-04-22T13:19:08.002976+0000 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: min_mon_release 19 (squid) 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: election_strategy: 1 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: fsmap 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mgrmap e13: x(active, since 58s) 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: overall HEALTH_OK 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:15.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/1335905522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:19:15.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: mon.b calling monitor election 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: mon.b calling monitor election 2024-04-22T13:19:15.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: mon.a calling monitor election 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: mon.c calling monitor election 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: monmap epoch 3 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: last_changed 2024-04-22T13:19:08.002976+0000 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: min_mon_release 19 (squid) 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: election_strategy: 1 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: fsmap 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: mgrmap e13: x(active, since 60s) 2024-04-22T13:19:15.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: overall HEALTH_OK 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:15.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 -- 172.21.15.5:0/443934433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381015f0 msgr2=0x7ffa381019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/443934433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381015f0 0x7ffa381019d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009910 tx=0x7ffa28009c20 comp rx=0 tx=0).stop 2024-04-22T13:19:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 -- 172.21.15.5:0/443934433 shutdown_connections 2024-04-22T13:19:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/443934433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381015f0 0x7ffa381019d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 -- 172.21.15.5:0/443934433 >> 172.21.15.5:0/443934433 conn(0x7ffa38076ea0 msgr2=0x7ffa380772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 -- 172.21.15.5:0/443934433 shutdown_connections 2024-04-22T13:19:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.993+0000 7ffa3dedc700 1 -- 172.21.15.5:0/443934433 wait complete. 2024-04-22T13:19:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 Processor -- start 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 -- start start 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa381015f0 0x7ffa38103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa38103580 0x7ffa38103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 0x7ffa381a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3810b820 con 0x7ffa381a43b0 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3810b6a0 con 0x7ffa38103580 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.994+0000 7ffa3dedc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3810b9a0 con 0x7ffa381015f0 2024-04-22T13:19:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 0x7ffa381a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 0x7ffa381a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 -- 172.21.15.5:0/3654931018 learned_addr learned my addr 172.21.15.5:0/3654931018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa36ffd700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa38103580 0x7ffa38103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa377fe700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa381015f0 0x7ffa38103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 -- 172.21.15.5:0/3654931018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa381015f0 msgr2=0x7ffa38103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa381015f0 0x7ffa38103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 -- 172.21.15.5:0/3654931018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa38103580 msgr2=0x7ffa38103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa38103580 0x7ffa38103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 -- 172.21.15.5:0/3654931018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa381a6e70 con 0x7ffa381a43b0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa37fff700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 0x7ffa381a67a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000ba50 tx=0x7ffa3000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30019070 con 0x7ffa381a43b0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa30014440 con 0x7ffa381a43b0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa381a7160 con 0x7ffa381a43b0 2024-04-22T13:19:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000ce60 con 0x7ffa381a43b0 2024-04-22T13:19:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.995+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa381a76a0 con 0x7ffa381a43b0 2024-04-22T13:19:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.996+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7ffa3001c070 con 0x7ffa381a43b0 2024-04-22T13:19:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.996+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa380640a0 con 0x7ffa381a43b0 2024-04-22T13:19:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.997+0000 7ffa34ff9700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ffa20039ae0 0x7ffa2003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.997+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffa3004be00 con 0x7ffa381a43b0 2024-04-22T13:19:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.997+0000 7ffa377fe700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ffa20039ae0 0x7ffa2003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:15.997+0000 7ffa377fe700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ffa20039ae0 0x7ffa2003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009750 tx=0x7ffa2801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.002+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa30014710 con 0x7ffa381a43b0 2024-04-22T13:19:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.223+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ffa381a7960 con 0x7ffa381a43b0 2024-04-22T13:19:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.224+0000 7ffa34ff9700 1 -- 172.21.15.5:0/3654931018 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7ffa30014910 con 0x7ffa381a43b0 2024-04-22T13:19:16.222 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:16.223 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-22T13:19:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] 2024-04-22T13:19:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ffa20039ae0 msgr2=0x7ffa2003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ffa20039ae0 0x7ffa2003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009750 tx=0x7ffa2801a040 comp rx=0 tx=0).stop 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 msgr2=0x7ffa381a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 0x7ffa381a67a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000ba50 tx=0x7ffa3000bd60 comp rx=0 tx=0).stop 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 shutdown_connections 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa381a43b0 0x7ffa381a67a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa381015f0 0x7ffa38103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ffa20039ae0 0x7ffa2003bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 --2- 172.21.15.5:0/3654931018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa38103580 0x7ffa38103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 >> 172.21.15.5:0/3654931018 conn(0x7ffa38076ea0 msgr2=0x7ffa38069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 shutdown_connections 2024-04-22T13:19:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:16.226+0000 7ffa3dedc700 1 -- 172.21.15.5:0/3654931018 wait complete. 2024-04-22T13:19:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.b calling monitor election 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.b calling monitor election 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.a calling monitor election 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.c calling monitor election 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: monmap epoch 3 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: last_changed 2024-04-22T13:19:08.002976+0000 2024-04-22T13:19:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: min_mon_release 19 (squid) 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: election_strategy: 1 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: fsmap 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: mgrmap e13: x(active, since 60s) 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: overall HEALTH_OK 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:16.275 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: mon.b calling monitor election 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: mon.b calling monitor election 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: mon.a calling monitor election 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: mon.c calling monitor election 2024-04-22T13:19:16.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: monmap epoch 3 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: last_changed 2024-04-22T13:19:08.002976+0000 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: created 2024-04-22T13:17:09.176897+0000 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: min_mon_release 19 (squid) 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: election_strategy: 1 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: 1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.c 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: 2: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.b 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: fsmap 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: mgrmap e13: x(active, since 60s) 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: overall HEALTH_OK 2024-04-22T13:19:16.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:16.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:17.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T13:19:17.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: Reconfiguring daemon mon.a on smithi005 2024-04-22T13:19:17.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:17.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3654931018' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T13:19:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: Reconfiguring daemon mon.a on smithi005 2024-04-22T13:19:17.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:17.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3654931018' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:19:17.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T13:19:17.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: Reconfiguring daemon mon.a on smithi005 2024-04-22T13:19:17.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:19:17.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3654931018' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:17.705 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-22T13:19:17.705 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:19:17.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T13:19:17.733 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:19:17.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:19:17.797 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:19:17.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T13:19:17.825 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:19:17.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:19:17.890 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:19:17.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T13:19:17.921 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:19:17.921 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:19:17.989 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-04-22T13:19:17.990 INFO:tasks.cephadm:Adding mgr.y on smithi049 2024-04-22T13:19:17.990 INFO:tasks.cephadm:Adding mgr.z on smithi107 2024-04-22T13:19:17.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi049=y;smithi107=z' 2024-04-22T13:19:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:17 smithi005 ceph-mon[28006]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:18.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:18.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:18.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:18.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:17 smithi049 ceph-mon[31379]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:18.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:18.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:18.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:18.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:18.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:18.264 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:19:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:17 smithi107 ceph-mon[31859]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:19.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:18 smithi107 ceph-mon[31859]: Reconfiguring mon.b (monmap changed)... 2024-04-22T13:19:19.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:18 smithi107 ceph-mon[31859]: Reconfiguring daemon mon.b on smithi049 2024-04-22T13:19:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:18 smithi049 ceph-mon[31379]: Reconfiguring mon.b (monmap changed)... 2024-04-22T13:19:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:18 smithi049 ceph-mon[31379]: Reconfiguring daemon mon.b on smithi049 2024-04-22T13:19:19.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:18 smithi005 ceph-mon[28006]: Reconfiguring mon.b (monmap changed)... 2024-04-22T13:19:19.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:18 smithi005 ceph-mon[28006]: Reconfiguring daemon mon.b on smithi049 2024-04-22T13:19:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: Reconfiguring mon.c (monmap changed)... 2024-04-22T13:19:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:20.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:20.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:20.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:19 smithi049 ceph-mon[31379]: Reconfiguring daemon mon.c on smithi107 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: Reconfiguring mon.c (monmap changed)... 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:20.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:19 smithi107 ceph-mon[31859]: Reconfiguring daemon mon.c on smithi107 2024-04-22T13:19:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: Reconfiguring mon.c (monmap changed)... 2024-04-22T13:19:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:19:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:19:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:19 smithi005 ceph-mon[28006]: Reconfiguring daemon mon.c on smithi107 2024-04-22T13:19:20.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.502+0000 7f7e16cf2700 1 -- 172.21.15.107:0/159573161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e10103b90 msgr2=0x7f7e10103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:20.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.502+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/159573161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e10103b90 0x7f7e10103fb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009910 tx=0x7f7e04009c20 comp rx=0 tx=0).stop 2024-04-22T13:19:20.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.502+0000 7f7e16cf2700 1 -- 172.21.15.107:0/159573161 shutdown_connections 2024-04-22T13:19:20.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.502+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/159573161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e10103b90 0x7f7e10103fb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.502+0000 7f7e16cf2700 1 -- 172.21.15.107:0/159573161 >> 172.21.15.107:0/159573161 conn(0x7f7e100fc1b0 msgr2=0x7f7e100fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:20.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 -- 172.21.15.107:0/159573161 shutdown_connections 2024-04-22T13:19:20.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 -- 172.21.15.107:0/159573161 wait complete. 2024-04-22T13:19:20.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 Processor -- start 2024-04-22T13:19:20.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 -- start start 2024-04-22T13:19:20.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 0x7f7e10197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e10198240 0x7f7e1019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e101986c0 0x7f7e1019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10108470 con 0x7f7e101986c0 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e101082f0 con 0x7f7e10198240 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.503+0000 7f7e16cf2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e101085f0 con 0x7f7e10103b90 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 0x7f7e10197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 0x7f7e10197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57940/0 (socket says 172.21.15.107:57940) 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 -- 172.21.15.107:0/2912400501 learned_addr learned my addr 172.21.15.107:0/2912400501 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e1528f700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e101986c0 0x7f7e1019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e0ffff700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e10198240 0x7f7e1019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 -- 172.21.15.107:0/2912400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e10198240 msgr2=0x7f7e1019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:20.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e10198240 0x7f7e1019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 -- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e101986c0 msgr2=0x7f7e1019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e101986c0 0x7f7e1019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 -- 172.21.15.107:0/2912400501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e101a0120 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e1528f700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e101986c0 0x7f7e1019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e14a8e700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 0x7f7e10197d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e04005e00 tx=0x7f7e04005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04026070 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e101a0410 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0400b6f0 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.504+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e040038e0 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.505+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e101a09d0 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.505+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f7e04029070 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.505+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e100640a0 con 0x7f7e10103b90 2024-04-22T13:19:20.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.506+0000 7f7e0dffb700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e00039ae0 0x7f7e0003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:20.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.506+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e0404d5d0 con 0x7f7e10103b90 2024-04-22T13:19:20.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.506+0000 7f7e0ffff700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e00039ae0 0x7f7e0003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:20.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.506+0000 7f7e0ffff700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e00039ae0 0x7f7e0003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009ad0 tx=0x7f7dfc009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:20.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.511+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e04013070 con 0x7f7e10103b90 2024-04-22T13:19:20.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.723+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi107=z", "target": ["mon-mgr", ""]}) -- 0x7f7e10062e80 con 0x7f7e00039ae0 2024-04-22T13:19:20.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.729+0000 7f7e0dffb700 1 -- 172.21.15.107:0/2912400501 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7e10062e80 con 0x7f7e00039ae0 2024-04-22T13:19:20.727 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled mgr update... 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.731+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e00039ae0 msgr2=0x7f7e0003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.731+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e00039ae0 0x7f7e0003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009ad0 tx=0x7f7dfc009210 comp rx=0 tx=0).stop 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 msgr2=0x7f7e10197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 0x7f7e10197d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e04005e00 tx=0x7f7e04005020 comp rx=0 tx=0).stop 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 shutdown_connections 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e101986c0 0x7f7e1019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e10103b90 0x7f7e10197d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e00039ae0 0x7f7e0003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 --2- 172.21.15.107:0/2912400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e10198240 0x7f7e1019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:20.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 >> 172.21.15.107:0/2912400501 conn(0x7f7e100fc1b0 msgr2=0x7f7e100fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:20.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 shutdown_connections 2024-04-22T13:19:20.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:19:20.732+0000 7f7e16cf2700 1 -- 172.21.15.107:0/2912400501 wait complete. 2024-04-22T13:19:21.520 DEBUG:teuthology.orchestra.run.smithi049:mgr.y> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.y.service 2024-04-22T13:19:21.523 DEBUG:teuthology.orchestra.run.smithi107:mgr.z> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.z.service 2024-04-22T13:19:21.526 INFO:tasks.cephadm:Deploying OSDs... 2024-04-22T13:19:21.526 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:19:21.526 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T13:19:21.543 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T13:19:21.543 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-22 13:18:26.717930895 +0000 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-22 13:12:42.053868668 +0000 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-22 13:12:42.053868668 +0000 2024-04-22T13:19:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-22 13:12:42.053868668 +0000 2024-04-22T13:19:21.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T13:19:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-22T13:19:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-22T13:19:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000107607 s, 4.8 MB/s 2024-04-22T13:19:21.664 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T13:19:21.720 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-04-22T13:19:21.775 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T13:19:21.775 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:21.775 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T13:19:21.776 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:21.776 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:21.776 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-22 13:18:26.752930139 +0000 2024-04-22T13:19:21.776 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-22 13:12:42.395860966 +0000 2024-04-22T13:19:21.776 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-22 13:12:42.395860966 +0000 2024-04-22T13:19:21.776 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-22 13:12:42.395860966 +0000 2024-04-22T13:19:21.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T13:19:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-22T13:19:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-22T13:19:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000108282 s, 4.7 MB/s 2024-04-22T13:19:21.839 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T13:19:21.895 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-04-22T13:19:21.951 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-22 13:18:26.787929384 +0000 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-22 13:12:42.723853579 +0000 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-22 13:12:42.723853579 +0000 2024-04-22T13:19:21.952 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-22 13:12:42.723853579 +0000 2024-04-22T13:19:21.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi107=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: Saving service mgr spec with placement smithi005=x;smithi049=y;smithi107=z;count:3 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:22.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:22.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:19:22.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:22.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-22T13:19:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-22T13:19:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 9.9898e-05 s, 5.1 MB/s 2024-04-22T13:19:22.016 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T13:19:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi107=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: Saving service mgr spec with placement smithi005=x;smithi049=y;smithi107=z;count:3 2024-04-22T13:19:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi049=y;smithi107=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: Saving service mgr spec with placement smithi005=x;smithi049=y;smithi107=z;count:3 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:22.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.072 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-04-22T13:19:22.127 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-22 13:18:26.821928650 +0000 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-22 13:12:43.059846012 +0000 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-22 13:12:43.059846012 +0000 2024-04-22T13:19:22.128 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-22 13:12:43.059846012 +0000 2024-04-22T13:19:22.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T13:19:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-22T13:19:22.191 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-22T13:19:22.191 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000108187 s, 4.7 MB/s 2024-04-22T13:19:22.192 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T13:19:22.248 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-22T13:19:22.248 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T13:19:22.268 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T13:19:22.268 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-22 13:19:07.358585238 +0000 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-22 13:13:26.496868914 +0000 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-22 13:13:26.496868914 +0000 2024-04-22T13:19:22.327 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-22 13:13:26.496868914 +0000 2024-04-22T13:19:22.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T13:19:22.394 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-22T13:19:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-22T13:19:22.395 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000127851 s, 4.0 MB/s 2024-04-22T13:19:22.396 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T13:19:22.455 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-04-22T13:19:22.512 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-22 13:19:07.358585238 +0000 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-22 13:13:26.847858958 +0000 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-22 13:13:26.847858958 +0000 2024-04-22T13:19:22.513 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-22 13:13:26.847858958 +0000 2024-04-22T13:19:22.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T13:19:22.578 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-22T13:19:22.578 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-22T13:19:22.579 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000120911 s, 4.2 MB/s 2024-04-22T13:19:22.580 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T13:19:22.639 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-22 13:19:07.359585211 +0000 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-22 13:13:27.182849458 +0000 2024-04-22T13:19:22.697 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-22 13:13:27.182849458 +0000 2024-04-22T13:19:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-22 13:13:27.182849458 +0000 2024-04-22T13:19:22.698 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T13:19:22.763 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-22T13:19:22.763 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-22T13:19:22.763 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000128118 s, 4.0 MB/s 2024-04-22T13:19:22.764 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T13:19:22.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:22 smithi107 ceph-mon[31859]: Deploying daemon mgr.z on smithi107 2024-04-22T13:19:22.821 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-04-22T13:19:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T13:19:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:22.877 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T13:19:22.877 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:22.878 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:22.878 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-22 13:19:07.359585211 +0000 2024-04-22T13:19:22.878 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-22 13:13:27.524839758 +0000 2024-04-22T13:19:22.878 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-22 13:13:27.524839758 +0000 2024-04-22T13:19:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-04-22 13:13:27.524839758 +0000 2024-04-22T13:19:22.878 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T13:19:22.942 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-22T13:19:22.942 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-22T13:19:22.942 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.7854e-05 s, 5.2 MB/s 2024-04-22T13:19:22.943 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T13:19:23.001 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T13:19:23.001 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T13:19:23.018 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T13:19:23.018 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-04-22T13:19:23.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:22 smithi049 ceph-mon[31379]: Deploying daemon mgr.z on smithi107 2024-04-22T13:19:23.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:22 smithi005 ceph-mon[28006]: Deploying daemon mgr.z on smithi107 2024-04-22T13:19:23.072 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T13:19:23.072 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:23.072 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-22T13:19:23.072 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:23.073 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:23.073 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 13:18:57.748719700 +0000 2024-04-22T13:19:23.073 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 13:13:18.714568511 +0000 2024-04-22T13:19:23.073 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 13:13:18.714568511 +0000 2024-04-22T13:19:23.073 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 13:13:18.714568511 +0000 2024-04-22T13:19:23.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T13:19:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T13:19:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T13:19:23.137 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000295948 s, 1.7 MB/s 2024-04-22T13:19:23.138 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T13:19:23.194 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 13:19:23.247296328 +0000 2024-04-22T13:19:23.249 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 13:13:19.055562618 +0000 2024-04-22T13:19:23.250 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 13:13:19.055562618 +0000 2024-04-22T13:19:23.250 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 13:13:19.055562618 +0000 2024-04-22T13:19:23.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T13:19:23.312 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T13:19:23.313 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T13:19:23.313 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000249476 s, 2.1 MB/s 2024-04-22T13:19:23.314 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T13:19:23.371 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-04-22T13:19:23.426 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T13:19:23.426 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:23.426 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-22T13:19:23.426 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:23.427 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:23.427 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 13:19:23.427293328 +0000 2024-04-22T13:19:23.427 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 13:13:19.406556553 +0000 2024-04-22T13:19:23.427 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 13:13:19.406556553 +0000 2024-04-22T13:19:23.427 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 13:13:19.406556553 +0000 2024-04-22T13:19:23.427 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T13:19:23.489 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T13:19:23.489 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T13:19:23.489 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000200536 s, 2.6 MB/s 2024-04-22T13:19:23.491 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T13:19:23.548 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 13:19:23.428293311 +0000 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 13:13:19.745550694 +0000 2024-04-22T13:19:23.605 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 13:13:19.745550694 +0000 2024-04-22T13:19:23.606 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 13:13:19.745550694 +0000 2024-04-22T13:19:23.606 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T13:19:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T13:19:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T13:19:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000278203 s, 1.8 MB/s 2024-04-22T13:19:23.670 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T13:19:23.726 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-04-22T13:19:23.727 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T13:19:23.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:23 smithi107 ceph-mon[31859]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:19:24.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:23 smithi049 ceph-mon[31379]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:24.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:23 smithi005 ceph-mon[28006]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:24.286 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:24.186+0000 7fd28701d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:19:24.569 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:24.377+0000 7fd28701d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:19:24.570 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:24.486+0000 7fd28701d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:19:24.894 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:24.810+0000 7fd28701d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:19:24.894 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:24.896+0000 7fd28701d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:19:25.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:25.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:25.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:19:25.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:25.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:25.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:25 smithi049 ceph-mon[31379]: Deploying daemon mgr.y on smithi049 2024-04-22T13:19:25.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:25.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:25.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:25 smithi107 ceph-mon[31859]: Deploying daemon mgr.y on smithi049 2024-04-22T13:19:25.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:25.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:25 smithi005 ceph-mon[28006]: Deploying daemon mgr.y on smithi049 2024-04-22T13:19:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T13:19:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T13:19:25.751 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-22T13:19:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-22T13:19:25.751 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834682 s, 1.3 GB/s 2024-04-22T13:19:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-22T13:19:26.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-04-22T13:19:26.415 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:26.085+0000 7fd28701d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:19:26.415 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:26.250+0000 7fd28701d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:19:26.416 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:26.417+0000 7fd28701d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:19:26.498 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 systemd[1]: Starting Ceph mgr.y for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:19:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:19:26.772 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 podman[32740]: 2024-04-22 13:19:26.498105013 +0000 UTC m=+0.106418252 container create 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:19:26.772 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 podman[32740]: 2024-04-22 13:19:26.409947017 +0000 UTC m=+0.018260269 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:19:26.772 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 podman[32740]: 2024-04-22 13:19:26.67531985 +0000 UTC m=+0.283633092 container init 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-22T13:19:26.772 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 podman[32740]: 2024-04-22 13:19:26.677844489 +0000 UTC m=+0.286157730 container start 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:19:26.773 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 ceph-mgr[32759]: -- 172.21.15.49:0/1504658463 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e892b8a5a0 con 0x55e892b5ec00 2024-04-22T13:19:26.773 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 bash[32740]: 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 2024-04-22T13:19:26.773 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 systemd[1]: Started Ceph mgr.y for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:19:26.985 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:26.697+0000 7fd28701d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:19:26.986 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:26.788+0000 7fd28701d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:19:26.986 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:26.988+0000 7fd28701d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:19:27.160 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:26.856+0000 7f6a6ffd4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:19:27.160 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:27.049+0000 7f6a6ffd4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:19:27.160 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:27.162+0000 7f6a6ffd4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:19:27.285 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:27.188+0000 7fd28701d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:19:27.500 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:27.502+0000 7f6a6ffd4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:19:27.772 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:27.599+0000 7f6a6ffd4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:19:27.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:27 smithi005 ceph-mon[28006]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:27.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:27.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:27.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:27.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:27.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:27.831+0000 7fd28701d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-mon[31859]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.081 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 -- 172.21.15.5:0/2973525872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94105e60 msgr2=0x7eff94108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2973525872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94105e60 0x7eff94108250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff88009990 tx=0x7eff88009ca0 comp rx=0 tx=0).stop 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 -- 172.21.15.5:0/2973525872 shutdown_connections 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2973525872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94103530 0x7eff94105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2973525872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff94108790 0x7eff9410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2973525872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94105e60 0x7eff94108250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 -- 172.21.15.5:0/2973525872 >> 172.21.15.5:0/2973525872 conn(0x7eff940fcf40 msgr2=0x7eff940ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 -- 172.21.15.5:0/2973525872 shutdown_connections 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.134+0000 7eff98dbb700 1 -- 172.21.15.5:0/2973525872 wait complete. 2024-04-22T13:19:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 Processor -- start 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 -- start start 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 0x7eff94198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94105e60 0x7eff941991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff94108790 0x7eff941a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410e580 con 0x7eff94103530 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410e400 con 0x7eff94105e60 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff98dbb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410e700 con 0x7eff94108790 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff9259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 0x7eff94198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff9259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 0x7eff94198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37714/0 (socket says 172.21.15.5:37714) 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff9259c700 1 -- 172.21.15.5:0/2874647159 learned_addr learned my addr 172.21.15.5:0/2874647159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:19:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff91d9b700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94105e60 0x7eff941991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.135+0000 7eff92d9d700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff94108790 0x7eff941a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff9259c700 1 -- 172.21.15.5:0/2874647159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff94108790 msgr2=0x7eff941a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff9259c700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff94108790 0x7eff941a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff9259c700 1 -- 172.21.15.5:0/2874647159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94105e60 msgr2=0x7eff941991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff9259c700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94105e60 0x7eff941991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff9259c700 1 -- 172.21.15.5:0/2874647159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff941a0920 con 0x7eff94103530 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff9259c700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 0x7eff94198ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff8400d950 tx=0x7eff8400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84016070 con 0x7eff94103530 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff8400fd40 con 0x7eff94103530 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff941a0c10 con 0x7eff94103530 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff941a1150 con 0x7eff94103530 2024-04-22T13:19:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.136+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8400e3b0 con 0x7eff94103530 2024-04-22T13:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.137+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7eff84019070 con 0x7eff94103530 2024-04-22T13:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.137+0000 7eff837fe700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7eff7c039890 0x7eff7c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.137+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff94069730 con 0x7eff94103530 2024-04-22T13:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.137+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7eff8404cad0 con 0x7eff94103530 2024-04-22T13:19:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.137+0000 7eff91d9b700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7eff7c039890 0x7eff7c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.138+0000 7eff91d9b700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7eff7c039890 0x7eff7c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff9419a1b0 tx=0x7eff88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.142+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff840106a0 con 0x7eff94103530 2024-04-22T13:19:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-mon[31379]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:28.360+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7eff94062e80 con 0x7eff7c039890 2024-04-22T13:19:28.360 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:28.262+0000 7fd28701d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:19:28.360 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:28.363+0000 7fd28701d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:19:28.785 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:28.570+0000 7fd28701d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:29.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.088 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:29.089 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:28.937+0000 7f6a6ffd4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:29.113+0000 7f6a6ffd4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:29.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:19:29.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:19:29.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:29.421 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:29.090+0000 7fd28701d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:19:29.421 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:29.183+0000 7fd28701d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:19:29.522 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:29.289+0000 7f6a6ffd4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:19:29.760 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:29.423+0000 7fd28701d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:19:29.761 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:29.554+0000 7fd28701d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:19:29.761 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:29.649+0000 7fd28701d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:19:29.761 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:29.762+0000 7fd28701d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:19:29.891 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-mon[31379]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:29.891 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-mon[31379]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:29.891 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-mon[31379]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T13:19:29.891 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-mon[31379]: Reconfiguring daemon mgr.x on smithi005 2024-04-22T13:19:29.892 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:29.892 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:29.588+0000 7f6a6ffd4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:19:29.892 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:29.683+0000 7f6a6ffd4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:19:29.892 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:29.893+0000 7f6a6ffd4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:19:29.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:29 smithi005 ceph-mon[28006]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:29.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:29 smithi005 ceph-mon[28006]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:29.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:29 smithi005 ceph-mon[28006]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T13:19:29.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:29 smithi005 ceph-mon[28006]: Reconfiguring daemon mgr.x on smithi005 2024-04-22T13:19:29.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:30.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-mon[31859]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:30.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-mon[31859]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:30.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-mon[31859]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T13:19:30.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-mon[31859]: Reconfiguring daemon mgr.x on smithi005 2024-04-22T13:19:30.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:30.272 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:30.091+0000 7f6a6ffd4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:19:30.535 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:30.064+0000 7fd28701d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:19:31.022 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:30.716+0000 7f6a6ffd4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:19:31.035 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:30.656+0000 7fd28701d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:19:31.035 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:19:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:19:30.867+0000 7fd28701d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:19:31.462 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:31.155+0000 7f6a6ffd4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:19:31.462 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:31.254+0000 7f6a6ffd4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:19:31.462 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:31.463+0000 7f6a6ffd4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:19:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:31.821+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) ==== 92601+0+0 (secure 0 0 0) 0x7eff84001ab0 con 0x7eff94103530 2024-04-22T13:19:32.111 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:32.017+0000 7f6a6ffd4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:19:32.111 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:32.112+0000 7f6a6ffd4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:19:32.111 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: Standby manager daemon z started 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:32.112 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: Standby manager daemon z started 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: Standby manager daemon z started 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/3770380847' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:32.723 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:32.367+0000 7f6a6ffd4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:19:32.723 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:32.507+0000 7f6a6ffd4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:19:32.723 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:32.606+0000 7f6a6ffd4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:19:32.723 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:32.725+0000 7f6a6ffd4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:19:33.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-22T13:19:33.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:19:33.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2361796830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]: dispatch 2024-04-22T13:19:33.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]: dispatch 2024-04-22T13:19:33.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]': finished 2024-04-22T13:19:33.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T13:19:33.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:33.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:32 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3031406874' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2361796830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]: dispatch 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]: dispatch 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]': finished 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:33.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:32 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3031406874' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2361796830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]: dispatch 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]: dispatch 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39983697-bdf7-4808-8aaf-f2ae5083be8d"}]': finished 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:32 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3031406874' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:33.522 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:33.047+0000 7f6a6ffd4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:19:34.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:33 smithi049 ceph-mon[31379]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:34.023 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:33.675+0000 7f6a6ffd4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:19:34.023 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:19:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:19:33.900+0000 7f6a6ffd4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:19:34.210 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:33 smithi005 ceph-mon[28006]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:33 smithi107 ceph-mon[31859]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:34.834+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7eff84023350 con 0x7eff94103530 2024-04-22T13:19:35.284 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:34 smithi049 ceph-mon[31379]: Standby manager daemon y started 2024-04-22T13:19:35.284 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:34 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:19:35.284 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:34 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:19:35.284 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:34 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:19:35.284 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:34 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:19:35.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:34 smithi107 ceph-mon[31859]: Standby manager daemon y started 2024-04-22T13:19:35.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:34 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:19:35.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:34 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:19:35.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:34 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:19:35.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:34 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:19:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:34 smithi005 ceph-mon[28006]: Standby manager daemon y started 2024-04-22T13:19:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:34 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:19:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:34 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:19:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:34 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:19:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:34 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/2410785157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:19:36.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:35 smithi107 ceph-mon[31859]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:36.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:35 smithi107 ceph-mon[31859]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-22T13:19:36.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:35 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:19:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:35 smithi005 ceph-mon[28006]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:36.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:35 smithi005 ceph-mon[28006]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-22T13:19:36.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:35 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:19:36.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:35 smithi049 ceph-mon[31379]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:36.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:35 smithi049 ceph-mon[31379]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-22T13:19:36.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:35 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:19:38.050 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:37 smithi005 ceph-mon[28006]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:38.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:37 smithi049 ceph-mon[31379]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:38.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:37 smithi107 ceph-mon[31859]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:40.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:39 smithi005 ceph-mon[28006]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:40.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T13:19:40.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:40.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:39 smithi049 ceph-mon[31379]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:40.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T13:19:40.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:40.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:39 smithi107 ceph-mon[31859]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:40.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T13:19:40.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:40.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:40 smithi005 ceph-mon[28006]: Deploying daemon osd.0 on smithi005 2024-04-22T13:19:41.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:40 smithi049 ceph-mon[31379]: Deploying daemon osd.0 on smithi005 2024-04-22T13:19:41.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:40 smithi107 ceph-mon[31859]: Deploying daemon osd.0 on smithi005 2024-04-22T13:19:42.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:41 smithi005 ceph-mon[28006]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:41 smithi049 ceph-mon[31379]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:41 smithi107 ceph-mon[31859]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:43 smithi005 ceph-mon[28006]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:44.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:43 smithi049 ceph-mon[31379]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:44.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:43 smithi107 ceph-mon[31859]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:45.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:45 smithi005 ceph-mon[28006]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:45.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:45.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:45.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:46.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:45 smithi049 ceph-mon[31379]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:46.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:46.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:46.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:46.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:45 smithi107 ceph-mon[31859]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:46.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:46.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:46.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:47 smithi005 ceph-mon[28006]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:47.920 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-04-22T13:19:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.922+0000 7eff837fe700 1 -- 172.21.15.5:0/2874647159 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff94062e80 con 0x7eff7c039890 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.924+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7eff7c039890 msgr2=0x7eff7c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.924+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7eff7c039890 0x7eff7c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff9419a1b0 tx=0x7eff88005fb0 comp rx=0 tx=0).stop 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.924+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 msgr2=0x7eff94198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.924+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 0x7eff94198ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff8400d950 tx=0x7eff8400dc60 comp rx=0 tx=0).stop 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 shutdown_connections 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff94108790 0x7eff941a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff94103530 0x7eff94198ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7eff7c039890 0x7eff7c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 --2- 172.21.15.5:0/2874647159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94105e60 0x7eff941991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 >> 172.21.15.5:0/2874647159 conn(0x7eff940fcf40 msgr2=0x7eff94103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 shutdown_connections 2024-04-22T13:19:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:19:47.925+0000 7eff98dbb700 1 -- 172.21.15.5:0/2874647159 wait complete. 2024-04-22T13:19:48.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:47 smithi049 ceph-mon[31379]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:48.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:48.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:48.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:48.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:48.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:48.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:47 smithi107 ceph-mon[31859]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:48.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:48.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:48.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:48.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:48.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:48.550 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.0.service 2024-04-22T13:19:48.553 INFO:tasks.cephadm:Deploying osd.1 on smithi049 with /dev/vg_nvme/lv_4... 2024-04-22T13:19:48.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T13:19:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:19:49.178 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:48 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:49.178 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:48 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:49.178 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:48 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:49.178 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:48 smithi049 ceph-mon[31379]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T13:19:49.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:48 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:49.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:48 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:49.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:48 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:49.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:48 smithi107 ceph-mon[31859]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T13:19:49.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:48 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:48 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:48 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:48 smithi005 ceph-mon[28006]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T13:19:50.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-mon[28006]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:50.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-mon[28006]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T13:19:50.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-mon[28006]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T13:19:50.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-mon[28006]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:19:50.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:50.000 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0[42998]: 2024-04-22T13:19:49.906+0000 7fdcf1418700 -1 osd.0 0 waiting for initial osdmap 2024-04-22T13:19:50.000 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:19:49 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0[42998]: 2024-04-22T13:19:49.909+0000 7fdced20b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:19:50.100 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:49 smithi049 ceph-mon[31379]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:50.100 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:49 smithi049 ceph-mon[31379]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T13:19:50.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:49 smithi049 ceph-mon[31379]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T13:19:50.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:49 smithi049 ceph-mon[31379]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:19:50.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:50.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:49 smithi107 ceph-mon[31859]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:50.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:49 smithi107 ceph-mon[31859]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T13:19:50.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:49 smithi107 ceph-mon[31859]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T13:19:50.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:49 smithi107 ceph-mon[31859]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:19:50.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:50.690 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T13:19:50.690 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T13:19:50.691 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-04-22T13:19:50.691 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-04-22T13:19:50.691 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00993242 s, 1.1 GB/s 2024-04-22T13:19:50.691 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-04-22T13:19:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:50 smithi005 ceph-mon[28006]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:19:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:50 smithi005 ceph-mon[28006]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T13:19:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:50 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:50 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:51.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:50 smithi049 ceph-mon[31379]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:19:51.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:50 smithi049 ceph-mon[31379]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T13:19:51.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:50 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:51.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:50 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:51.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:50 smithi107 ceph-mon[31859]: from='osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:19:51.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:50 smithi107 ceph-mon[31859]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T13:19:51.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:50 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:51.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:50 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:51.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-04-22T13:19:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] boot 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:52.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] boot 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:52.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:19:52.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: osd.0 [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] boot 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:19:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 -- 172.21.15.49:0/2803787725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 msgr2=0x7fb8441041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 --2- 172.21.15.49:0/2803787725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb8441041a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb838009990 tx=0x7fb838009ca0 comp rx=0 tx=0).stop 2024-04-22T13:19:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 -- 172.21.15.49:0/2803787725 shutdown_connections 2024-04-22T13:19:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 --2- 172.21.15.49:0/2803787725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8441046e0 0x7fb84410af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 --2- 172.21.15.49:0/2803787725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb844100d60 0x7fb844101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:53.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 --2- 172.21.15.49:0/2803787725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb8441041a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 -- 172.21.15.49:0/2803787725 >> 172.21.15.49:0/2803787725 conn(0x7fb8440fc6a0 msgr2=0x7fb8440feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 -- 172.21.15.49:0/2803787725 shutdown_connections 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 -- 172.21.15.49:0/2803787725 wait complete. 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.110+0000 7fb84943e700 1 Processor -- start 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 -- start start 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb844100d60 0x7fb844198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb844199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8441046e0 0x7fb8441a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410e610 con 0x7fb844100d60 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410e490 con 0x7fb844103d40 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb84943e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410e790 con 0x7fb8441046e0 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb844199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:53.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb844199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39608/0 (socket says 172.21.15.49:39608) 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 -- 172.21.15.49:0/4254217484 learned_addr learned my addr 172.21.15.49:0/4254217484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb842ffd700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb844100d60 0x7fb844198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8437fe700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8441046e0 0x7fb8441a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 -- 172.21.15.49:0/4254217484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8441046e0 msgr2=0x7fb8441a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8441046e0 0x7fb8441a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 -- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb844100d60 msgr2=0x7fb844198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb844100d60 0x7fb844198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 -- 172.21.15.49:0/4254217484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8441a0910 con 0x7fb844103d40 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.111+0000 7fb8427fc700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb844199220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb838003790 tx=0x7fb838009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.112+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838027070 con 0x7fb844103d40 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.112+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8380040c0 con 0x7fb844103d40 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.112+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8441a0ba0 con 0x7fb844103d40 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.112+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838022ca0 con 0x7fb844103d40 2024-04-22T13:19:53.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.112+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8441a0ff0 con 0x7fb844103d40 2024-04-22T13:19:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.113+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb844069730 con 0x7fb844103d40 2024-04-22T13:19:53.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.114+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7fb83802a070 con 0x7fb844103d40 2024-04-22T13:19:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.114+0000 7fb82bfff700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb82c0a3bb0 0x7fb82c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.115+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb8380cfc40 con 0x7fb844103d40 2024-04-22T13:19:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.115+0000 7fb842ffd700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb82c0a3bb0 0x7fb82c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:53.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.115+0000 7fb842ffd700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb82c0a3bb0 0x7fb82c0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8440feaa0 tx=0x7fb834009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:53.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.118+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb838031070 con 0x7fb844103d40 2024-04-22T13:19:53.116 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:52 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi005 2024-04-22T13:19:53.116 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:52 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi005 to 17093M 2024-04-22T13:19:53.116 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:52 smithi049 ceph-mon[31379]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T13:19:53.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:52 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi005 2024-04-22T13:19:53.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:52 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi005 to 17093M 2024-04-22T13:19:53.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:52 smithi107 ceph-mon[31859]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T13:19:53.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:52 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi005 2024-04-22T13:19:53.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:52 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi005 to 17093M 2024-04-22T13:19:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:52 smithi005 ceph-mon[28006]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T13:19:53.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:19:53.333+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb844062e80 con 0x7fb82c0a3bb0 2024-04-22T13:19:54.252 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:53 smithi049 ceph-mon[31379]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:54.253 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:54.253 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:54.253 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:53 smithi107 ceph-mon[31859]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:54.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:53 smithi005 ceph-mon[28006]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:54.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:54.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:54.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:55.143 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:54 smithi049 ceph-mon[31379]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:55.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:54 smithi107 ceph-mon[31859]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:55.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:54 smithi005 ceph-mon[28006]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:56.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:55 smithi049 ceph-mon[31379]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:56.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:55 smithi107 ceph-mon[31859]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:56.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:55 smithi005 ceph-mon[28006]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:56 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/2135287706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]: dispatch 2024-04-22T13:19:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:56 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]: dispatch 2024-04-22T13:19:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:56 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]': finished 2024-04-22T13:19:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:56 smithi049 ceph-mon[31379]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T13:19:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:56 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:19:57.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:56 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/3662854080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:56 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/2135287706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]: dispatch 2024-04-22T13:19:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:56 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]: dispatch 2024-04-22T13:19:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:56 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]': finished 2024-04-22T13:19:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:56 smithi107 ceph-mon[31859]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T13:19:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:56 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:19:57.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:56 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/3662854080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:57.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:56 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/2135287706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]: dispatch 2024-04-22T13:19:57.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:56 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]: dispatch 2024-04-22T13:19:57.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:56 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "555f251b-3c1d-48a2-a456-d45129ace7df"}]': finished 2024-04-22T13:19:57.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:56 smithi005 ceph-mon[28006]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T13:19:57.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:56 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:19:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:56 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/3662854080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:58.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:57 smithi049 ceph-mon[31379]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:58.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:57 smithi107 ceph-mon[31859]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:19:58.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:57 smithi005 ceph-mon[28006]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:00.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:19:59 smithi049 ceph-mon[31379]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:00.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:19:59 smithi107 ceph-mon[31859]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:00.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:19:59 smithi005 ceph-mon[28006]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:01.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:00 smithi049 ceph-mon[31379]: overall HEALTH_OK 2024-04-22T13:20:01.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:00 smithi107 ceph-mon[31859]: overall HEALTH_OK 2024-04-22T13:20:01.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:00 smithi005 ceph-mon[28006]: overall HEALTH_OK 2024-04-22T13:20:02.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:01 smithi049 ceph-mon[31379]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:02.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:01 smithi107 ceph-mon[31859]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:02.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:01 smithi005 ceph-mon[28006]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:03 smithi049 ceph-mon[31379]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T13:20:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:03 smithi107 ceph-mon[31859]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T13:20:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:03 smithi005 ceph-mon[28006]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T13:20:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:05.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:04 smithi049 ceph-mon[31379]: Deploying daemon osd.1 on smithi049 2024-04-22T13:20:05.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:04 smithi107 ceph-mon[31859]: Deploying daemon osd.1 on smithi049 2024-04-22T13:20:05.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:04 smithi005 ceph-mon[28006]: Deploying daemon osd.1 on smithi049 2024-04-22T13:20:06.200 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:05 smithi049 ceph-mon[31379]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:05 smithi107 ceph-mon[31859]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:05 smithi005 ceph-mon[28006]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:08.259 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:08 smithi049 ceph-mon[31379]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:08 smithi107 ceph-mon[31859]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:08.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:08 smithi005 ceph-mon[28006]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:09.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:09 smithi049 ceph-mon[31379]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:09.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:09.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:09.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:09.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:09 smithi107 ceph-mon[31859]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:09.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:09.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:09.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:09.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:09 smithi005 ceph-mon[28006]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:09.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:09.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:09.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:11 smithi049 ceph-mon[31379]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:11.788 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:11 smithi107 ceph-mon[31859]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:11.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:11 smithi005 ceph-mon[28006]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:11.853 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 1 on host 'smithi049' 2024-04-22T13:20:11.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.853+0000 7fb82bfff700 1 -- 172.21.15.49:0/4254217484 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb844062e80 con 0x7fb82c0a3bb0 2024-04-22T13:20:11.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb82c0a3bb0 msgr2=0x7fb82c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:11.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb82c0a3bb0 0x7fb82c0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8440feaa0 tx=0x7fb834009190 comp rx=0 tx=0).stop 2024-04-22T13:20:11.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 msgr2=0x7fb844199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb844199220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb838003790 tx=0x7fb838009580 comp rx=0 tx=0).stop 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 shutdown_connections 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8441046e0 0x7fb8441a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb844100d60 0x7fb844198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb82c0a3bb0 0x7fb82c0a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 --2- 172.21.15.49:0/4254217484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb844103d40 0x7fb844199220 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.855+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 >> 172.21.15.49:0/4254217484 conn(0x7fb8440fc6a0 msgr2=0x7fb844107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.856+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 shutdown_connections 2024-04-22T13:20:11.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:20:11.856+0000 7fb84943e700 1 -- 172.21.15.49:0/4254217484 wait complete. 2024-04-22T13:20:12.576 DEBUG:teuthology.orchestra.run.smithi049:osd.1> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.1.service 2024-04-22T13:20:12.578 INFO:tasks.cephadm:Deploying osd.2 on smithi107 with /dev/vg_nvme/lv_4... 2024-04-22T13:20:12.578 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T13:20:12.824 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:20:12.946 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:12.946 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:12.947 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:12.947 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:12.947 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:12 smithi107 ceph-mon[31859]: from='osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:20:12.947 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:12 smithi107 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:20:13.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:12 smithi005 ceph-mon[28006]: from='osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:20:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:12 smithi005 ceph-mon[28006]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:20:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:13.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:12 smithi049 ceph-mon[31379]: from='osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:20:13.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:12 smithi049 ceph-mon[31379]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:20:14.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-mon[31379]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:14.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-mon[31379]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T13:20:14.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-mon[31379]: from='osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:20:14.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-mon[31379]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T13:20:14.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:14.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-mon[31379]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:20:14.023 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1[35957]: 2024-04-22T13:20:13.873+0000 7f668ed62700 -1 osd.1 0 waiting for initial osdmap 2024-04-22T13:20:14.023 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1[35957]: 2024-04-22T13:20:13.878+0000 7f668ab55700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:20:14.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:13 smithi107 ceph-mon[31859]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:14.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:13 smithi107 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T13:20:14.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:13 smithi107 ceph-mon[31859]: from='osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:20:14.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:13 smithi107 ceph-mon[31859]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T13:20:14.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:13 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:14.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:13 smithi107 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:20:14.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:13 smithi005 ceph-mon[28006]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:14.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:13 smithi005 ceph-mon[28006]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T13:20:14.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:13 smithi005 ceph-mon[28006]: from='osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:20:14.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:13 smithi005 ceph-mon[28006]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T13:20:14.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:13 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:14.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:13 smithi005 ceph-mon[28006]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:20:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T13:20:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T13:20:14.632 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T13:20:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T13:20:14.632 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113752 s, 922 MB/s 2024-04-22T13:20:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T13:20:14.986 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:14 smithi049 ceph-mon[31379]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:20:14.986 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:14 smithi049 ceph-mon[31379]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T13:20:14.986 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:14.986 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:15.213 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:14 smithi107 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:20:15.214 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:14 smithi107 ceph-mon[31859]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T13:20:15.214 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:15.214 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:15.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-04-22T13:20:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:14 smithi005 ceph-mon[28006]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:20:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:14 smithi005 ceph-mon[28006]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T13:20:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:15.479 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:20:16.195 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:20:16.195 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] boot 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.196 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:16.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:16.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:16.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:20:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:20:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] boot 2024-04-22T13:20:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T13:20:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:16.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: osd.1 [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] boot 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:16.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:16.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:16.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.979+0000 7f0503768700 1 -- 172.21.15.107:0/2463665177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc103560 msgr2=0x7f04fc109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.979+0000 7f0503768700 1 --2- 172.21.15.107:0/2463665177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc103560 0x7f04fc109e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04f400b0a0 tx=0x7f04f400b3b0 comp rx=0 tx=0).stop 2024-04-22T13:20:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 -- 172.21.15.107:0/2463665177 shutdown_connections 2024-04-22T13:20:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 --2- 172.21.15.107:0/2463665177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc103560 0x7f04fc109e10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 --2- 172.21.15.107:0/2463665177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04fc1019c0 0x7f04fc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 --2- 172.21.15.107:0/2463665177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04fc102bc0 0x7f04fc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 -- 172.21.15.107:0/2463665177 >> 172.21.15.107:0/2463665177 conn(0x7f04fc0fcf60 msgr2=0x7f04fc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 -- 172.21.15.107:0/2463665177 shutdown_connections 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 -- 172.21.15.107:0/2463665177 wait complete. 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.980+0000 7f0503768700 1 Processor -- start 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 -- start start 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04fc1019c0 0x7f04fc197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 0x7f04fc198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04fc103560 0x7f04fc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d550 con 0x7f04fc1019c0 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d3d0 con 0x7f04fc103560 2024-04-22T13:20:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0503768700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d6d0 con 0x7f04fc102bc0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 0x7f04fc198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 0x7f04fc198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35460/0 (socket says 172.21.15.107:35460) 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 -- 172.21.15.107:0/674603280 learned_addr learned my addr 172.21.15.107:0/674603280 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0501504700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04fc1019c0 0x7f04fc197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0501d05700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04fc103560 0x7f04fc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 -- 172.21.15.107:0/674603280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04fc103560 msgr2=0x7f04fc19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04fc103560 0x7f04fc19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 -- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04fc1019c0 msgr2=0x7f04fc197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04fc1019c0 0x7f04fc197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 -- 172.21.15.107:0/674603280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc19f830 con 0x7f04fc102bc0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f0500d03700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 0x7f04fc198110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00b560 tx=0x7f04ec00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.981+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec015020 con 0x7f04fc102bc0 2024-04-22T13:20:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.982+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ec011ce0 con 0x7f04fc102bc0 2024-04-22T13:20:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.982+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc19fb20 con 0x7f04fc102bc0 2024-04-22T13:20:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.982+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec010d20 con 0x7f04fc102bc0 2024-04-22T13:20:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.982+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1a0060 con 0x7f04fc102bc0 2024-04-22T13:20:16.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.982+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc069730 con 0x7f04fc102bc0 2024-04-22T13:20:16.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.984+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f04ec018020 con 0x7f04fc102bc0 2024-04-22T13:20:16.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.985+0000 7f04f27fc700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f04e80a3bb0 0x7f04e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:16.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.985+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f04ec0ce840 con 0x7f04fc102bc0 2024-04-22T13:20:16.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.985+0000 7f0501504700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f04e80a3bb0 0x7f04e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:16.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.986+0000 7f0501504700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f04e80a3bb0 0x7f04e80a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04fc102a20 tx=0x7f04f8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:16.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:16.989+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04ec01f020 con 0x7f04fc102bc0 2024-04-22T13:20:17.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:16 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi049 2024-04-22T13:20:17.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:16 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi049 to 17088M 2024-04-22T13:20:17.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:16 smithi107 ceph-mon[31859]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T13:20:17.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:17.195+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f04fc062e80 con 0x7f04e80a3bb0 2024-04-22T13:20:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:16 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi049 2024-04-22T13:20:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:16 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi049 to 17088M 2024-04-22T13:20:17.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:16 smithi049 ceph-mon[31379]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T13:20:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:16 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi049 2024-04-22T13:20:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:16 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi049 to 17088M 2024-04-22T13:20:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:16 smithi005 ceph-mon[28006]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T13:20:18.193 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:17 smithi107 ceph-mon[31859]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:18.194 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:18.194 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:18.194 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:18.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:17 smithi049 ceph-mon[31379]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:18.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:18.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:18.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:17 smithi005 ceph-mon[28006]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:18.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:19.146 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:18 smithi107 ceph-mon[31859]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:18 smithi049 ceph-mon[31379]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:19.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:18 smithi005 ceph-mon[28006]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:19 smithi049 ceph-mon[31379]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:20.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:19 smithi107 ceph-mon[31859]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:19 smithi005 ceph-mon[28006]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:21.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:20 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/664131407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]: dispatch 2024-04-22T13:20:21.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:20 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]: dispatch 2024-04-22T13:20:21.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:20 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]': finished 2024-04-22T13:20:21.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:20 smithi049 ceph-mon[31379]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T13:20:21.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:21.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:20 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/2529151797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:20 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/664131407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]: dispatch 2024-04-22T13:20:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:20 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]: dispatch 2024-04-22T13:20:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:20 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]': finished 2024-04-22T13:20:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:20 smithi107 ceph-mon[31859]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T13:20:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:20 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/2529151797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:21.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:20 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/664131407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]: dispatch 2024-04-22T13:20:21.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:20 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]: dispatch 2024-04-22T13:20:21.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:20 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d526a13-cdf8-49ac-9871-b8a577898db3"}]': finished 2024-04-22T13:20:21.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:20 smithi005 ceph-mon[28006]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T13:20:21.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:21.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:20 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/2529151797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:22.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:22 smithi049 ceph-mon[31379]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:22.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:22 smithi107 ceph-mon[31859]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:22.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:22 smithi005 ceph-mon[28006]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:24.088 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:24 smithi107 ceph-mon[31859]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:24 smithi049 ceph-mon[31379]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:24.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:24 smithi005 ceph-mon[28006]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:26 smithi107 ceph-mon[31859]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:26 smithi049 ceph-mon[31379]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:26 smithi005 ceph-mon[28006]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:28.130 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:28 smithi107 ceph-mon[31859]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:28.130 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T13:20:28.130 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:28 smithi049 ceph-mon[31379]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T13:20:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:28.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:28 smithi005 ceph-mon[28006]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:28.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T13:20:28.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:29.269 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:29 smithi107 ceph-mon[31859]: Deploying daemon osd.2 on smithi107 2024-04-22T13:20:29.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:29 smithi049 ceph-mon[31379]: Deploying daemon osd.2 on smithi107 2024-04-22T13:20:29.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:29 smithi005 ceph-mon[28006]: Deploying daemon osd.2 on smithi107 2024-04-22T13:20:30.146 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:30 smithi107 ceph-mon[31859]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:30 smithi049 ceph-mon[31379]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:30.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:30 smithi005 ceph-mon[28006]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:32.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:32 smithi049 ceph-mon[31379]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:32.280 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:32 smithi107 ceph-mon[31859]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:32.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:32 smithi005 ceph-mon[28006]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:33.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:33 smithi107 ceph-mon[31859]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:33.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:33.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:33.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:33 smithi005 ceph-mon[28006]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:34.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:33 smithi049 ceph-mon[31379]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:34.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:34.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:34.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:35.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.718+0000 7f04f27fc700 1 -- 172.21.15.107:0/674603280 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f04fc062e80 con 0x7f04e80a3bb0 2024-04-22T13:20:35.716 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 2 on host 'smithi107' 2024-04-22T13:20:35.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f04e80a3bb0 msgr2=0x7f04e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:35.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f04e80a3bb0 0x7f04e80a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04fc102a20 tx=0x7f04f8009190 comp rx=0 tx=0).stop 2024-04-22T13:20:35.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 msgr2=0x7f04fc198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 0x7f04fc198110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00b560 tx=0x7f04ec00b870 comp rx=0 tx=0).stop 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 shutdown_connections 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f04e80a3bb0 0x7f04e80a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04fc103560 0x7f04fc19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04fc1019c0 0x7f04fc197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 --2- 172.21.15.107:0/674603280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04fc102bc0 0x7f04fc198110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 >> 172.21.15.107:0/674603280 conn(0x7f04fc0fcf60 msgr2=0x7f04fc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 shutdown_connections 2024-04-22T13:20:35.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:20:35.721+0000 7f0503768700 1 -- 172.21.15.107:0/674603280 wait complete. 2024-04-22T13:20:35.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:35 smithi005 ceph-mon[28006]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:36.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:35 smithi049 ceph-mon[31379]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:36.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:35 smithi107 ceph-mon[31859]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:36.676 DEBUG:teuthology.orchestra.run.smithi107:osd.2> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.2.service 2024-04-22T13:20:36.679 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-04-22T13:20:36.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T13:20:36.914 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.914 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.915 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.915 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.915 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:36 smithi107 ceph-mon[31859]: from='osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:20:36.915 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:36 smithi107 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:20:36.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:36.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:36 smithi005 ceph-mon[28006]: from='osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:20:36.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:36 smithi005 ceph-mon[28006]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:20:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:20:37.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:37.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:37.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:37.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:37.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:36 smithi049 ceph-mon[31379]: from='osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:20:37.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:36 smithi049 ceph-mon[31379]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:20:37.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:37 smithi005 ceph-mon[28006]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:37.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:37 smithi005 ceph-mon[28006]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T13:20:37.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:37 smithi005 ceph-mon[28006]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T13:20:37.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:37 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:37.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:37 smithi005 ceph-mon[28006]: from='osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:20:37.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:37 smithi005 ceph-mon[28006]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:20:37.845 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:37 smithi107 ceph-mon[31859]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:37.846 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:37 smithi107 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T13:20:37.846 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:37 smithi107 ceph-mon[31859]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T13:20:37.846 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:37 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:37.846 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:37 smithi107 ceph-mon[31859]: from='osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:20:37.846 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:37 smithi107 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:20:38.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:37 smithi049 ceph-mon[31379]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:38.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:37 smithi049 ceph-mon[31379]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T13:20:38.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:37 smithi049 ceph-mon[31379]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T13:20:38.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:37 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:38.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:37 smithi049 ceph-mon[31379]: from='osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:20:38.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:37 smithi049 ceph-mon[31379]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:20:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T13:20:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T13:20:39.213 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-22T13:20:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-22T13:20:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00847272 s, 1.2 GB/s 2024-04-22T13:20:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-22T13:20:39.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-mon[31859]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:39.495 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:20:39.495 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-mon[31859]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T13:20:39.495 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:39.495 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:39.495 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-mon[31859]: from='osd.2 ' entity='osd.2' 2024-04-22T13:20:39.495 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2[37029]: 2024-04-22T13:20:39.259+0000 7f18ad01e700 -1 osd.2 0 waiting for initial osdmap 2024-04-22T13:20:39.495 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2[37029]: 2024-04-22T13:20:39.385+0000 7f18a4e09700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:20:39.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:39 smithi005 ceph-mon[28006]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:39.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:39 smithi005 ceph-mon[28006]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:20:39.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:39 smithi005 ceph-mon[28006]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T13:20:39.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:39.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:39.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:39 smithi005 ceph-mon[28006]: from='osd.2 ' entity='osd.2' 2024-04-22T13:20:39.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:39 smithi049 ceph-mon[31379]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:39.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:39 smithi049 ceph-mon[31379]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:20:39.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:39 smithi049 ceph-mon[31379]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T13:20:39.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:39.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:39.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:39 smithi049 ceph-mon[31379]: from='osd.2 ' entity='osd.2' 2024-04-22T13:20:39.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-04-22T13:20:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:20:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi107 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi107 to 17088M 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:40 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:20:40.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:20:40.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:40.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi107 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi107 to 17088M 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:40.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:40 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:20:40.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi107 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi107 to 17088M 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:40.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:40 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3722905544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 msgr2=0x7f9b8410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3722905544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b8410abf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009990 tx=0x7f9b74009ca0 comp rx=0 tx=0).stop 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3722905544 shutdown_connections 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3722905544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b8410abf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3722905544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b84103520 0x7f9b84105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3722905544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84105e50 0x7f9b84108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3722905544 >> 172.21.15.5:0/3722905544 conn(0x7f9b840fcf30 msgr2=0x7f9b840ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3722905544 shutdown_connections 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.566+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3722905544 wait complete. 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 Processor -- start 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 -- start start 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84103520 0x7f9b84198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b84105e50 0x7f9b841991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b841a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8410e5f0 con 0x7f9b84108780 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8410e470 con 0x7f9b84103520 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8bde7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8410e770 con 0x7f9b84105e50 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b841a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b841a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37694/0 (socket says 172.21.15.5:37694) 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 -- 172.21.15.5:0/3634100801 learned_addr learned my addr 172.21.15.5:0/3634100801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:20:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b89382700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b84105e50 0x7f9b841991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b89b83700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84103520 0x7f9b84198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 -- 172.21.15.5:0/3634100801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b84105e50 msgr2=0x7f9b841991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b84105e50 0x7f9b841991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 -- 172.21.15.5:0/3634100801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84103520 msgr2=0x7f9b84198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84103520 0x7f9b84198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b8a384700 1 -- 172.21.15.5:0/3634100801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b841a0910 con 0x7f9b84108780 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.567+0000 7f9b89b83700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84103520 0x7f9b84198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.568+0000 7f9b8a384700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b841a0240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b740047b0 tx=0x7f9b74005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.568+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74027070 con 0x7f9b84108780 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.568+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b841a0ba0 con 0x7f9b84108780 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.568+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b74004100 con 0x7f9b84108780 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.568+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74022e10 con 0x7f9b84108780 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.568+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a1050 con 0x7f9b84108780 2024-04-22T13:20:41.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.569+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b84069730 con 0x7f9b84108780 2024-04-22T13:20:41.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.570+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f9b7402a070 con 0x7f9b84108780 2024-04-22T13:20:41.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:41 smithi005 ceph-mon[28006]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:41.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:41 smithi005 ceph-mon[28006]: osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] boot 2024-04-22T13:20:41.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:41 smithi005 ceph-mon[28006]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T13:20:41.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:41.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.570+0000 7f9b82ffd700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b6c0a3bb0 0x7f9b6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:41.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.571+0000 7f9b89b83700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b6c0a3bb0 0x7f9b6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:41.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.571+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f9b740cfce0 con 0x7f9b84108780 2024-04-22T13:20:41.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.571+0000 7f9b89b83700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b6c0a3bb0 0x7f9b6c0a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b84199b70 tx=0x7f9b7c009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.575+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b74013070 con 0x7f9b84108780 2024-04-22T13:20:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:41 smithi049 ceph-mon[31379]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:41 smithi049 ceph-mon[31379]: osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] boot 2024-04-22T13:20:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:41 smithi049 ceph-mon[31379]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T13:20:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:41 smithi107 ceph-mon[31859]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:20:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:41 smithi107 ceph-mon[31859]: osd.2 [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] boot 2024-04-22T13:20:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:41 smithi107 ceph-mon[31859]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T13:20:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:20:41.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:41.793+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9b84062e80 con 0x7f9b6c0a3bb0 2024-04-22T13:20:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:42 smithi005 ceph-mon[28006]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T13:20:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:42 smithi005 ceph-mon[28006]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:20:42.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:42 smithi049 ceph-mon[31379]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T13:20:42.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:42 smithi049 ceph-mon[31379]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:42.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:42.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:42.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:42.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:20:42.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:42 smithi107 ceph-mon[31859]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T13:20:42.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:42 smithi107 ceph-mon[31859]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:42.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:42.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:42.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:42.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:20:43.612 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 ceph-mon[31859]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:43.612 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T13:20:43.612 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 ceph-mon[31859]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T13:20:43.612 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:20:43.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 ceph-mon[28006]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:43.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T13:20:43.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 ceph-mon[28006]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T13:20:43.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47418]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47418]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47421]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47421]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47421]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:43.677 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47421]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:43.751 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 ceph-mon[31379]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:43.751 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T13:20:43.751 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 ceph-mon[31379]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T13:20:43.751 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39803]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39803]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39803]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39806]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:43.752 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39806]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47424]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47424]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47424]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47424]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47427]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:43.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47427]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:43.939 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:43 smithi005 sudo[47427]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39809]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39809]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39812]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:44.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:43 smithi049 sudo[39812]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41099]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:20:44.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:44.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41099]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:44.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41099]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41102]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:20:44.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41102]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:44.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41102]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:44.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41102]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41093]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41093]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41093]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41096]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:20:44.036 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:20:43 smithi107 sudo[41096]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:20:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:20:44.677+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9b74099c50 con 0x7f9b84108780 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.676 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:44 smithi005 ceph-mon[28006]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:20:44.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T13:20:44.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.774 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.774 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.774 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.774 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:44 smithi049 ceph-mon[31379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:20:44.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T13:20:44.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:44.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:20:44.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:20:44.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:20:44.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:20:44.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:44 smithi107 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3993742561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8732d14a-c8a2-4cbe-ac81-8d7eccbecd08"}]: dispatch 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3993742561' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8732d14a-c8a2-4cbe-ac81-8d7eccbecd08"}]': finished 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:20:45.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T13:20:45.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:45 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3720253685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:45.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:45.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T13:20:45.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3993742561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8732d14a-c8a2-4cbe-ac81-8d7eccbecd08"}]: dispatch 2024-04-22T13:20:45.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3993742561' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8732d14a-c8a2-4cbe-ac81-8d7eccbecd08"}]': finished 2024-04-22T13:20:45.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T13:20:45.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:20:45.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T13:20:45.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:45 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3720253685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T13:20:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3993742561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8732d14a-c8a2-4cbe-ac81-8d7eccbecd08"}]: dispatch 2024-04-22T13:20:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3993742561' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8732d14a-c8a2-4cbe-ac81-8d7eccbecd08"}]': finished 2024-04-22T13:20:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T13:20:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:20:45.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T13:20:45.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:45 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3720253685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:47 smithi049 ceph-mon[31379]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:47 smithi107 ceph-mon[31859]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:47 smithi005 ceph-mon[28006]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:49.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:49 smithi005 ceph-mon[28006]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:49.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:49 smithi049 ceph-mon[31379]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:49 smithi107 ceph-mon[31859]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:51.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:51 smithi005 ceph-mon[28006]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:51.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:51 smithi049 ceph-mon[31379]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:51.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:51 smithi107 ceph-mon[31859]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:52.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T13:20:52.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:52.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:52 smithi005 ceph-mon[28006]: Deploying daemon osd.3 on smithi005 2024-04-22T13:20:52.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T13:20:52.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:52.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:52 smithi049 ceph-mon[31379]: Deploying daemon osd.3 on smithi005 2024-04-22T13:20:52.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T13:20:52.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:52.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:52 smithi107 ceph-mon[31859]: Deploying daemon osd.3 on smithi005 2024-04-22T13:20:53.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:53 smithi005 ceph-mon[28006]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:53.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:53 smithi049 ceph-mon[31379]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:53.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:53 smithi107 ceph-mon[31859]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:55 smithi005 ceph-mon[28006]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:55.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:55 smithi049 ceph-mon[31379]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:55.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:55 smithi107 ceph-mon[31859]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:57.767 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:57 smithi049 ceph-mon[31379]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:57.767 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:57.767 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:57.768 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:57.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:57 smithi107 ceph-mon[31859]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:57.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:57.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:57.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:57 smithi005 ceph-mon[28006]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:20:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:59.688 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:20:59 smithi107 ceph-mon[31859]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:59.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:20:59 smithi049 ceph-mon[31379]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:20:59.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:20:59 smithi005 ceph-mon[28006]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-04-22T13:21:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.980+0000 7f9b82ffd700 1 -- 172.21.15.5:0/3634100801 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b84062e80 con 0x7f9b6c0a3bb0 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b6c0a3bb0 msgr2=0x7f9b6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b6c0a3bb0 0x7f9b6c0a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b84199b70 tx=0x7f9b7c009570 comp rx=0 tx=0).stop 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 msgr2=0x7f9b841a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b841a0240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b740047b0 tx=0x7f9b74005e80 comp rx=0 tx=0).stop 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 shutdown_connections 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b6c0a3bb0 0x7f9b6c0a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b84103520 0x7f9b84198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b84108780 0x7f9b841a0240 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 --2- 172.21.15.5:0/3634100801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b84105e50 0x7f9b841991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 >> 172.21.15.5:0/3634100801 conn(0x7f9b840fcf30 msgr2=0x7f9b84103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:00.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 shutdown_connections 2024-04-22T13:21:00.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:00.983+0000 7f9b8bde7700 1 -- 172.21.15.5:0/3634100801 wait complete. 2024-04-22T13:21:01.285 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:01.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:01.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:01.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:01.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:01.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:01.571 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.3.service 2024-04-22T13:21:01.574 INFO:tasks.cephadm:Deploying osd.4 on smithi049 with /dev/vg_nvme/lv_3... 2024-04-22T13:21:01.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T13:21:01.830 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:21:02.268 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:02 smithi049 ceph-mon[31379]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T13:21:02.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:02 smithi005 ceph-mon[28006]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T13:21:02.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:02 smithi107 ceph-mon[31859]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T13:21:03.125 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:03 smithi049 ceph-mon[31379]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T13:21:03.125 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:03 smithi049 ceph-mon[31379]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T13:21:03.125 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:03 smithi049 ceph-mon[31379]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:21:03.126 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:03.126 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:03 smithi049 ceph-mon[31379]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:03.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-mon[28006]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T13:21:03.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-mon[28006]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T13:21:03.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-mon[28006]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:21:03.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:03.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-mon[28006]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:03.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:03 smithi107 ceph-mon[31859]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T13:21:03.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:03 smithi107 ceph-mon[31859]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T13:21:03.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:03 smithi107 ceph-mon[31859]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:21:03.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:03.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:03 smithi107 ceph-mon[31859]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:21:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T13:21:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T13:21:03.670 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-04-22T13:21:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-04-22T13:21:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00952617 s, 1.1 GB/s 2024-04-22T13:21:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] boot 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T13:21:04.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.216 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:21:03.870+0000 7faf93dca700 -1 osd.3 0 waiting for initial osdmap 2024-04-22T13:21:04.216 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:21:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:21:03.876+0000 7faf8bbb5700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:21:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:21:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:21:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:21:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T13:21:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' 2024-04-22T13:21:04.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] boot 2024-04-22T13:21:04.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T13:21:04.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:04 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-04-22T13:21:04.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:21:04.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:21:04.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: from='osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912]' entity='osd.3' 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: osd.3 [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] boot 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T13:21:04.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:21:04.619 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:21:05.826 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:05.826 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi005 2024-04-22T13:21:05.826 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.826 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi005 to 8546M 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.827 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:05 smithi049 ceph-mon[31379]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T13:21:05.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:05.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi005 2024-04-22T13:21:05.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi005 to 8546M 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:05.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:05 smithi005 ceph-mon[28006]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T13:21:06.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:06.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi005 2024-04-22T13:21:06.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:06.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:06.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi005 to 8546M 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:06.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:05 smithi107 ceph-mon[31859]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T13:21:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 -- 172.21.15.49:0/1659847024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c102ba0 msgr2=0x7f4f4c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:06.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1659847024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c103000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f340099c0 tx=0x7f4f34009cd0 comp rx=0 tx=0).stop 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 -- 172.21.15.49:0/1659847024 shutdown_connections 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1659847024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c103540 0x7f4f4c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1659847024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c1019a0 0x7f4f4c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1659847024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c103000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 -- 172.21.15.49:0/1659847024 >> 172.21.15.49:0/1659847024 conn(0x7f4f4c0fcf40 msgr2=0x7f4f4c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 -- 172.21.15.49:0/1659847024 shutdown_connections 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.313+0000 7f4f514c9700 1 -- 172.21.15.49:0/1659847024 wait complete. 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 Processor -- start 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 -- start start 2024-04-22T13:21:06.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c1019a0 0x7f4f4c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 0x7f4f4c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10d440 con 0x7f4f4c102ba0 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10d2c0 con 0x7f4f4c103540 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f514c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10d5c0 con 0x7f4f4c1019a0 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f4b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 0x7f4f4c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f4b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 0x7f4f4c1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34812/0 (socket says 172.21.15.49:34812) 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f4b7fe700 1 -- 172.21.15.49:0/1735024697 learned_addr learned my addr 172.21.15.49:0/1735024697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.314+0000 7f4f4a7fc700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:06.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4affd700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c1019a0 0x7f4f4c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4a7fc700 1 -- 172.21.15.49:0/1735024697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c1019a0 msgr2=0x7f4f4c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4a7fc700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c1019a0 0x7f4f4c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4a7fc700 1 -- 172.21.15.49:0/1735024697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 msgr2=0x7f4f4c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4a7fc700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 0x7f4f4c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4a7fc700 1 -- 172.21.15.49:0/1735024697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c1a0830 con 0x7f4f4c102ba0 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4b7fe700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 0x7f4f4c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f4a7fc700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c199110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4f34003790 tx=0x7f4f340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34027070 con 0x7f4f4c102ba0 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c1a0ac0 con 0x7f4f4c102ba0 2024-04-22T13:21:06.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34004020 con 0x7f4f4c102ba0 2024-04-22T13:21:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34022c70 con 0x7f4f4c102ba0 2024-04-22T13:21:06.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.315+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c1a0fc0 con 0x7f4f4c102ba0 2024-04-22T13:21:06.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.316+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c069730 con 0x7f4f4c102ba0 2024-04-22T13:21:06.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.318+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f4f3402a070 con 0x7f4f4c102ba0 2024-04-22T13:21:06.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.318+0000 7f4f43fff700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f380a3c40 0x7f4f380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:06.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.318+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f4f340d0440 con 0x7f4f4c102ba0 2024-04-22T13:21:06.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.318+0000 7f4f4affd700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f380a3c40 0x7f4f380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:06.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.319+0000 7f4f4affd700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f380a3c40 0x7f4f380a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c102a00 tx=0x7f4f3c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.326+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f34031070 con 0x7f4f4c102ba0 2024-04-22T13:21:06.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:06.544+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4f4c0ff150 con 0x7f4f380a3c40 2024-04-22T13:21:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:07 smithi005 ceph-mon[28006]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T13:21:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:07 smithi005 ceph-mon[28006]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:07 smithi005 ceph-mon[28006]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:07.417 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:07 smithi049 ceph-mon[31379]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T13:21:07.418 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:07 smithi049 ceph-mon[31379]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:07.418 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:07 smithi049 ceph-mon[31379]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:07.418 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:07.418 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:07.418 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:07.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:07 smithi107 ceph-mon[31859]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T13:21:07.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:07 smithi107 ceph-mon[31859]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:07.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:07 smithi107 ceph-mon[31859]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:07.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:07.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:07.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:08 smithi107 ceph-mon[31859]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T13:21:08.389 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:08 smithi049 ceph-mon[31379]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T13:21:08.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:08 smithi005 ceph-mon[28006]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T13:21:09.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:09 smithi049 ceph-mon[31379]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T13:21:09.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:09 smithi049 ceph-mon[31379]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:09.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:09 smithi107 ceph-mon[31859]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T13:21:09.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:09 smithi107 ceph-mon[31859]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:09.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:09 smithi005 ceph-mon[28006]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T13:21:09.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:09 smithi005 ceph-mon[28006]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:10.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:10 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/2999060955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]: dispatch 2024-04-22T13:21:10.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:10 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]: dispatch 2024-04-22T13:21:10.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:10 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]': finished 2024-04-22T13:21:10.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:10 smithi049 ceph-mon[31379]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T13:21:10.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:10 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:10.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:10 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/1430103788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:10.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:10 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/2999060955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]: dispatch 2024-04-22T13:21:10.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:10 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]: dispatch 2024-04-22T13:21:10.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:10 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]': finished 2024-04-22T13:21:10.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:10 smithi107 ceph-mon[31859]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T13:21:10.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:10 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:10.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:10 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/1430103788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:10.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:10 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/2999060955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]: dispatch 2024-04-22T13:21:10.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:10 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]: dispatch 2024-04-22T13:21:10.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:10 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc694d4-956f-424a-a8a7-e56d4e3b1213"}]': finished 2024-04-22T13:21:10.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:10 smithi005 ceph-mon[28006]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T13:21:10.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:10 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:10.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:10 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/1430103788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:11.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:11 smithi049 ceph-mon[31379]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:11.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:11 smithi107 ceph-mon[31859]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:11.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:11 smithi005 ceph-mon[28006]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:13.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:13 smithi049 ceph-mon[31379]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:21:13.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:13 smithi107 ceph-mon[31859]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:21:13.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:13 smithi005 ceph-mon[28006]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:21:15.341 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:15 smithi049 ceph-mon[31379]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-22T13:21:15.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:15 smithi107 ceph-mon[31859]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-22T13:21:15.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:15 smithi005 ceph-mon[28006]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-22T13:21:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:17 smithi005 ceph-mon[28006]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:21:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T13:21:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:17.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:17 smithi005 ceph-mon[28006]: Deploying daemon osd.4 on smithi049 2024-04-22T13:21:17.441 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:17 smithi049 ceph-mon[31379]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:21:17.441 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T13:21:17.441 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:17.441 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:17 smithi049 ceph-mon[31379]: Deploying daemon osd.4 on smithi049 2024-04-22T13:21:17.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:17 smithi107 ceph-mon[31859]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:21:17.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T13:21:17.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:17.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:17 smithi107 ceph-mon[31859]: Deploying daemon osd.4 on smithi049 2024-04-22T13:21:19.375 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:19 smithi049 ceph-mon[31379]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:21:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:19 smithi107 ceph-mon[31859]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:21:19.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:19 smithi005 ceph-mon[28006]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:21:21.416 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:21 smithi049 ceph-mon[31379]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-22T13:21:21.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:21 smithi107 ceph-mon[31859]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-22T13:21:21.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:21 smithi005 ceph-mon[28006]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-22T13:21:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:24.234 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:24 smithi049 ceph-mon[31379]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:21:24.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:24 smithi107 ceph-mon[31859]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:21:24.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:24 smithi005 ceph-mon[28006]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:21:25.307 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 4 on host 'smithi049' 2024-04-22T13:21:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.308+0000 7f4f43fff700 1 -- 172.21.15.49:0/1735024697 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f4c0ff150 con 0x7f4f380a3c40 2024-04-22T13:21:25.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f380a3c40 msgr2=0x7f4f380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f380a3c40 0x7f4f380a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c102a00 tx=0x7f4f3c007400 comp rx=0 tx=0).stop 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c102ba0 msgr2=0x7f4f4c199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c199110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4f34003790 tx=0x7f4f340047b0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 shutdown_connections 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f380a3c40 0x7f4f380a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c103540 0x7f4f4c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c1019a0 0x7f4f4c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 --2- 172.21.15.49:0/1735024697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4c102ba0 0x7f4f4c199110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.310+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 >> 172.21.15.49:0/1735024697 conn(0x7f4f4c0fcf40 msgr2=0x7f4f4c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.311+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 shutdown_connections 2024-04-22T13:21:25.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:21:25.311+0000 7f4f514c9700 1 -- 172.21.15.49:0/1735024697 wait complete. 2024-04-22T13:21:25.910 DEBUG:teuthology.orchestra.run.smithi049:osd.4> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.4.service 2024-04-22T13:21:25.913 INFO:tasks.cephadm:Deploying osd.5 on smithi107 with /dev/vg_nvme/lv_3... 2024-04-22T13:21:25.913 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T13:21:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:21:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:26 smithi049 ceph-mon[31379]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:26.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:26 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:26.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:26 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:26.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:26 smithi049 ceph-mon[31379]: from='osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:21:26.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:26 smithi049 ceph-mon[31379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:21:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:26 smithi107 ceph-mon[31859]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:26 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:26 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:26 smithi107 ceph-mon[31859]: from='osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:21:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:26 smithi107 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:21:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:26 smithi005 ceph-mon[28006]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:26 smithi005 ceph-mon[28006]: from='osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:21:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:26 smithi005 ceph-mon[28006]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: from='osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:27.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:27 smithi049 ceph-mon[31379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: from='osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:27.328 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:27 smithi107 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:21:27.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:27.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:27.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:27.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T13:21:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: from='osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:21:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T13:21:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:27 smithi005 ceph-mon[28006]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0159659 s, 657 MB/s 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2024-04-22T13:21:27.941 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T13:21:28.272 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4[42642]: 2024-04-22T13:21:28.038+0000 7f0b2bdf1700 -1 osd.4 0 waiting for initial osdmap 2024-04-22T13:21:28.272 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4[42642]: 2024-04-22T13:21:28.045+0000 7f0b23bdc700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:21:28.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:28 smithi005 ceph-mon[28006]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:21:28.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:28 smithi005 ceph-mon[28006]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T13:21:28.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:28.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:28 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:28.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:28 smithi005 ceph-mon[28006]: from='osd.4 ' entity='osd.4' 2024-04-22T13:21:28.569 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:28 smithi107 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:21:28.569 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:28 smithi107 ceph-mon[31859]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T13:21:28.569 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:28.569 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:28 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:28.569 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:28 smithi107 ceph-mon[31859]: from='osd.4 ' entity='osd.4' 2024-04-22T13:21:28.571 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-04-22T13:21:28.722 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-mon[31379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:21:28.722 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-mon[31379]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T13:21:28.722 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:28.722 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:28.722 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:28 smithi049 ceph-mon[31379]: from='osd.4 ' entity='osd.4' 2024-04-22T13:21:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:21:29.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:21:29.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:21:29.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:29.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:29.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] boot 2024-04-22T13:21:29.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T13:21:29.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] boot 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T13:21:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: osd.4 [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] boot 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T13:21:29.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 -- 172.21.15.107:0/2876794387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170104640 msgr2=0x7f917010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2876794387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170104640 0x7f917010aef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f916800b0a0 tx=0x7f916800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 -- 172.21.15.107:0/2876794387 shutdown_connections 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2876794387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9170102aa0 0x7f9170102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2876794387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170104640 0x7f917010aef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2876794387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9170103ca0 0x7f9170104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 -- 172.21.15.107:0/2876794387 >> 172.21.15.107:0/2876794387 conn(0x7f91700fe040 msgr2=0x7f9170100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 -- 172.21.15.107:0/2876794387 shutdown_connections 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.359+0000 7f9176dd1700 1 -- 172.21.15.107:0/2876794387 wait complete. 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 Processor -- start 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 -- start start 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 0x7f9170198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9170103ca0 0x7f91701990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9170104640 0x7f91701a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f917010e520 con 0x7f9170103ca0 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f917010e3a0 con 0x7f9170104640 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9176dd1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f917010e6a0 con 0x7f9170102aa0 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9174b6d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 0x7f9170198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9174b6d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 0x7f9170198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40954/0 (socket says 172.21.15.107:40954) 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f9174b6d700 1 -- 172.21.15.107:0/2219973835 learned_addr learned my addr 172.21.15.107:0/2219973835 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f917536e700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9170104640 0x7f91701a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.360+0000 7f916ffff700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9170103ca0 0x7f91701990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9174b6d700 1 -- 172.21.15.107:0/2219973835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9170104640 msgr2=0x7f91701a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9174b6d700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9170104640 0x7f91701a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9174b6d700 1 -- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9170103ca0 msgr2=0x7f91701990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9174b6d700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9170103ca0 0x7f91701990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9174b6d700 1 -- 172.21.15.107:0/2219973835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91701a07c0 con 0x7f9170102aa0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9174b6d700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 0x7f9170198b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f916400d9b0 tx=0x7f916400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164017030 con 0x7f9170102aa0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91701a0ab0 con 0x7f9170102aa0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91640044d0 con 0x7f9170102aa0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91640038e0 con 0x7f9170102aa0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.361+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701a1040 con 0x7f9170102aa0 2024-04-22T13:21:30.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.362+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9170069730 con 0x7f9170102aa0 2024-04-22T13:21:30.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.363+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f916401a070 con 0x7f9170102aa0 2024-04-22T13:21:30.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.364+0000 7f916dffb700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91600a3c40 0x7f91600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.364+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f91640cec40 con 0x7f9170102aa0 2024-04-22T13:21:30.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.364+0000 7f916ffff700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91600a3c40 0x7f91600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.364+0000 7f916ffff700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91600a3c40 0x7f91600a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f915c005950 tx=0x7f915c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:30.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.368+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f916409c690 con 0x7f9170102aa0 2024-04-22T13:21:30.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:30.576+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9170062e80 con 0x7f91600a3c40 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi049 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi049 to 8544M 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:30.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:30.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:30.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi049 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi049 to 8544M 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:30.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:30.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:30.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi049 2024-04-22T13:21:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi049 to 8544M 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:31.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:31 smithi049 ceph-mon[31379]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:31.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:31 smithi049 ceph-mon[31379]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:31.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:31.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:31.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:31.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:31 smithi049 ceph-mon[31379]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T13:21:31.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:31 smithi107 ceph-mon[31859]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:31.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:31 smithi107 ceph-mon[31859]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:31.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:31.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:31.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:31.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:31 smithi107 ceph-mon[31859]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T13:21:31.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:31 smithi005 ceph-mon[28006]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:31.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:31 smithi005 ceph-mon[28006]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:31.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:31.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:31.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:31.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:31 smithi005 ceph-mon[28006]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T13:21:33.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:33 smithi049 ceph-mon[31379]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:33.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:33 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/1995040560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]: dispatch 2024-04-22T13:21:33.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:33 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]: dispatch 2024-04-22T13:21:33.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:33 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]': finished 2024-04-22T13:21:33.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:33 smithi049 ceph-mon[31379]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T13:21:33.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:33.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:33 smithi107 ceph-mon[31859]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:33.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:33 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/1995040560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]: dispatch 2024-04-22T13:21:33.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:33 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]: dispatch 2024-04-22T13:21:33.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:33 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]': finished 2024-04-22T13:21:33.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:33 smithi107 ceph-mon[31859]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T13:21:33.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:33 smithi005 ceph-mon[28006]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:33 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/1995040560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]: dispatch 2024-04-22T13:21:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:33 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]: dispatch 2024-04-22T13:21:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:33 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2da21bf6-32ce-49f7-93ba-362c202d18cb"}]': finished 2024-04-22T13:21:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:33 smithi005 ceph-mon[28006]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T13:21:33.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:34.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/2303781216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:34.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/2303781216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:34.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/2303781216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:35.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:35 smithi049 ceph-mon[31379]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:35.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:35 smithi107 ceph-mon[31859]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:35.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:35 smithi005 ceph-mon[28006]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:37.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:37 smithi049 ceph-mon[31379]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:37.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:37 smithi107 ceph-mon[31859]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:37.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:37 smithi005 ceph-mon[28006]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:39.669 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:39 smithi107 ceph-mon[31859]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:39.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:39 smithi049 ceph-mon[31379]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:39 smithi005 ceph-mon[28006]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:41.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:41 smithi107 ceph-mon[31859]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:41.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T13:21:41.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:41.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:41 smithi107 ceph-mon[31859]: Deploying daemon osd.5 on smithi107 2024-04-22T13:21:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:41 smithi049 ceph-mon[31379]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T13:21:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:41.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:41 smithi049 ceph-mon[31379]: Deploying daemon osd.5 on smithi107 2024-04-22T13:21:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:41 smithi005 ceph-mon[28006]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T13:21:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:41 smithi005 ceph-mon[28006]: Deploying daemon osd.5 on smithi107 2024-04-22T13:21:43.554 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:43 smithi107 ceph-mon[31859]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:43 smithi049 ceph-mon[31379]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:43 smithi005 ceph-mon[28006]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:45.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:45 smithi049 ceph-mon[31379]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:45.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:45 smithi107 ceph-mon[31859]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:45.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:45 smithi005 ceph-mon[28006]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:47.470 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:47 smithi107 ceph-mon[31859]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:47.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:47.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:47.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:47.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:47 smithi005 ceph-mon[28006]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:47.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:47.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:47.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:47 smithi049 ceph-mon[31379]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:49.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.795+0000 7f916dffb700 1 -- 172.21.15.107:0/2219973835 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9170062e80 con 0x7f91600a3c40 2024-04-22T13:21:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 5 on host 'smithi107' 2024-04-22T13:21:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91600a3c40 msgr2=0x7f91600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91600a3c40 0x7f91600a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f915c005950 tx=0x7f915c00b410 comp rx=0 tx=0).stop 2024-04-22T13:21:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 msgr2=0x7f9170198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 0x7f9170198b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f916400d9b0 tx=0x7f916400dcc0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 shutdown_connections 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91600a3c40 0x7f91600a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9170104640 0x7f91701a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170102aa0 0x7f9170198b60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 --2- 172.21.15.107:0/2219973835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9170103ca0 0x7f91701990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 >> 172.21.15.107:0/2219973835 conn(0x7f91700fe040 msgr2=0x7f9170106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 shutdown_connections 2024-04-22T13:21:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:21:49.797+0000 7f9176dd1700 1 -- 172.21.15.107:0/2219973835 wait complete. 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:50.468 DEBUG:teuthology.orchestra.run.smithi107:osd.5> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.5.service 2024-04-22T13:21:50.470 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-04-22T13:21:50.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T13:21:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:21:51.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:50 smithi107 ceph-mon[31859]: from='osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:21:51.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:50 smithi107 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:21:51.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:50 smithi005 ceph-mon[28006]: from='osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:21:51.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:50 smithi005 ceph-mon[28006]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:21:51.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:50 smithi049 ceph-mon[31379]: from='osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:21:51.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:50 smithi049 ceph-mon[31379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:21:52.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:51 smithi107 ceph-mon[31859]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:52.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:51 smithi107 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T13:21:52.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:51 smithi107 ceph-mon[31859]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T13:21:52.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:51 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:52.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:51 smithi107 ceph-mon[31859]: from='osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:21:52.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:51 smithi107 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:21:52.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:51 smithi005 ceph-mon[28006]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:52.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:51 smithi005 ceph-mon[28006]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T13:21:52.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:51 smithi005 ceph-mon[28006]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T13:21:52.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:51 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:52.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:51 smithi005 ceph-mon[28006]: from='osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:21:52.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:51 smithi005 ceph-mon[28006]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:21:52.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:51 smithi049 ceph-mon[31379]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:52.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:51 smithi049 ceph-mon[31379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T13:21:52.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:51 smithi049 ceph-mon[31379]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T13:21:52.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:51 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:52.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:51 smithi049 ceph-mon[31379]: from='osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:21:52.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:51 smithi049 ceph-mon[31379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:21:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T13:21:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T13:21:52.644 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-22T13:21:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-22T13:21:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00837744 s, 1.3 GB/s 2024-04-22T13:21:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-22T13:21:53.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:21:53.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T13:21:53.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: osdmap e39: 6 total, 5 up, 6 in 2024-04-22T13:21:53.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-mon[31859]: from='osd.5 ' entity='osd.5' 2024-04-22T13:21:53.008 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:21:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:21:52.851+0000 7fb37f672700 -1 osd.5 0 waiting for initial osdmap 2024-04-22T13:21:53.008 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:21:53.010+0000 7fb37745d700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: osdmap e39: 6 total, 5 up, 6 in 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:52 smithi005 ceph-mon[28006]: from='osd.5 ' entity='osd.5' 2024-04-22T13:21:53.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: osdmap e39: 6 total, 5 up, 6 in 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:52 smithi049 ceph-mon[31379]: from='osd.5 ' entity='osd.5' 2024-04-22T13:21:53.526 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:21:54.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:21:54.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:21:54.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:54.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] boot 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:53 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] boot 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:54.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:54.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:53 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: osd.5 [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] boot 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:21:54.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:54.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:53 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:21:55.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 -- 172.21.15.5:0/3346340867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c100d50 msgr2=0x7f3d9c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:55.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 --2- 172.21.15.5:0/3346340867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c100d50 0x7f3d9c101170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400b0a0 tx=0x7f3d9400b3b0 comp rx=0 tx=0).stop 2024-04-22T13:21:55.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 -- 172.21.15.5:0/3346340867 shutdown_connections 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 --2- 172.21.15.5:0/3346340867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9c1046d0 0x7f3d9c10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 --2- 172.21.15.5:0/3346340867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c100d50 0x7f3d9c101170 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 --2- 172.21.15.5:0/3346340867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d9c103d30 0x7f3d9c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 -- 172.21.15.5:0/3346340867 >> 172.21.15.5:0/3346340867 conn(0x7f3d9c0fc6d0 msgr2=0x7f3d9c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 -- 172.21.15.5:0/3346340867 shutdown_connections 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.019+0000 7f3da3023700 1 -- 172.21.15.5:0/3346340867 wait complete. 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 Processor -- start 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 -- start start 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d9c100d50 0x7f3d9c103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 0x7f3d9c101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9c1046d0 0x7f3d9c102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10e570 con 0x7f3d9c103d30 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10e3f0 con 0x7f3d9c100d50 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da3023700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10e6f0 con 0x7f3d9c1046d0 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 0x7f3d9c101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 0x7f3d9c101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38360/0 (socket says 172.21.15.5:38360) 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 -- 172.21.15.5:0/3543051679 learned_addr learned my addr 172.21.15.5:0/3543051679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da0dbf700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d9c100d50 0x7f3d9c103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3da15c0700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9c1046d0 0x7f3d9c102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 -- 172.21.15.5:0/3543051679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9c1046d0 msgr2=0x7f3d9c102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9c1046d0 0x7f3d9c102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 -- 172.21.15.5:0/3543051679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d9c100d50 msgr2=0x7f3d9c103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d9c100d50 0x7f3d9c103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.020+0000 7f3d9bfff700 1 -- 172.21.15.5:0/3543051679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c102ad0 con 0x7f3d9c103d30 2024-04-22T13:21:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.021+0000 7f3d9bfff700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 0x7f3d9c101ce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b530 tx=0x7f3d9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.021+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90015020 con 0x7f3d9c103d30 2024-04-22T13:21:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.021+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c1a2e00 con 0x7f3d9c103d30 2024-04-22T13:21:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.021+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d90011ce0 con 0x7f3d9c103d30 2024-04-22T13:21:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.021+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90010dd0 con 0x7f3d9c103d30 2024-04-22T13:21:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.021+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c1a32e0 con 0x7f3d9c103d30 2024-04-22T13:21:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.022+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c069730 con 0x7f3d9c103d30 2024-04-22T13:21:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.023+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f3d90018020 con 0x7f3d9c103d30 2024-04-22T13:21:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.024+0000 7f3d99ffb700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d840a3c40 0x7f3d840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.024+0000 7f3da0dbf700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d840a3c40 0x7f3d840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.024+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f3d900cecf0 con 0x7f3d9c103d30 2024-04-22T13:21:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.024+0000 7f3da0dbf700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d840a3c40 0x7f3d840a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009fd0 tx=0x7f3d9400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:55.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.029+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d9001f020 con 0x7f3d9c103d30 2024-04-22T13:21:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:21:55.249+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3d9c062e80 con 0x7f3d840a3c40 2024-04-22T13:21:55.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:54 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi107 2024-04-22T13:21:55.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:54 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi107 to 8544M 2024-04-22T13:21:55.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:54 smithi005 ceph-mon[28006]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T13:21:55.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:54 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi107 2024-04-22T13:21:55.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:54 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi107 to 8544M 2024-04-22T13:21:55.291 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:54 smithi107 ceph-mon[31859]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T13:21:55.291 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:54 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi107 2024-04-22T13:21:55.291 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:54 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi107 to 8544M 2024-04-22T13:21:55.292 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:54 smithi049 ceph-mon[31379]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T13:21:56.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:55 smithi005 ceph-mon[28006]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:21:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:55 smithi005 ceph-mon[28006]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T13:21:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:56.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:56.186 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:55 smithi107 ceph-mon[31859]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:21:56.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:55 smithi107 ceph-mon[31859]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T13:21:56.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:56.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:56.187 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:56.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:55 smithi049 ceph-mon[31379]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:21:56.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:55 smithi049 ceph-mon[31379]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T13:21:56.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:21:56.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:21:56.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:57.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:56 smithi005 ceph-mon[28006]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:57.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:56 smithi005 ceph-mon[28006]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T13:21:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:56 smithi049 ceph-mon[31379]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:57.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:56 smithi049 ceph-mon[31379]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T13:21:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:56 smithi107 ceph-mon[31859]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:57.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:56 smithi107 ceph-mon[31859]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T13:21:58.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:57 smithi049 ceph-mon[31379]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:21:58.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:57 smithi049 ceph-mon[31379]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T13:21:58.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:57 smithi107 ceph-mon[31859]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:21:58.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:57 smithi107 ceph-mon[31859]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T13:21:58.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:57 smithi005 ceph-mon[28006]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:21:58.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:57 smithi005 ceph-mon[28006]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T13:21:59.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2102861732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b51d6e8a-1cb9-4856-9945-94e6ecb354d9"}]: dispatch 2024-04-22T13:21:59.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2102861732' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b51d6e8a-1cb9-4856-9945-94e6ecb354d9"}]': finished 2024-04-22T13:21:59.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:58 smithi049 ceph-mon[31379]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T13:21:59.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:58 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:21:59.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/856264461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:59.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2102861732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b51d6e8a-1cb9-4856-9945-94e6ecb354d9"}]: dispatch 2024-04-22T13:21:59.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2102861732' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b51d6e8a-1cb9-4856-9945-94e6ecb354d9"}]': finished 2024-04-22T13:21:59.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:58 smithi107 ceph-mon[31859]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T13:21:59.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:58 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:21:59.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/856264461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:21:59.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2102861732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b51d6e8a-1cb9-4856-9945-94e6ecb354d9"}]: dispatch 2024-04-22T13:21:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2102861732' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b51d6e8a-1cb9-4856-9945-94e6ecb354d9"}]': finished 2024-04-22T13:21:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:58 smithi005 ceph-mon[28006]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T13:21:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:58 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:21:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/856264461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:00.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:21:59 smithi049 ceph-mon[31379]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-22T13:22:00.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:21:59 smithi107 ceph-mon[31859]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-22T13:22:00.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:21:59 smithi005 ceph-mon[28006]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-22T13:22:02.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:01 smithi049 ceph-mon[31379]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-22T13:22:02.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:01 smithi005 ceph-mon[28006]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-22T13:22:02.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:01 smithi107 ceph-mon[31859]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-22T13:22:04.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:03 smithi005 ceph-mon[28006]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:03 smithi049 ceph-mon[31379]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:03 smithi107 ceph-mon[31859]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:06.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:05 smithi005 ceph-mon[28006]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:06.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T13:22:06.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:05 smithi049 ceph-mon[31379]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T13:22:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:05 smithi107 ceph-mon[31859]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T13:22:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:06 smithi049 ceph-mon[31379]: Deploying daemon osd.6 on smithi005 2024-04-22T13:22:07.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:06 smithi107 ceph-mon[31859]: Deploying daemon osd.6 on smithi005 2024-04-22T13:22:07.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:06 smithi005 ceph-mon[28006]: Deploying daemon osd.6 on smithi005 2024-04-22T13:22:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:07 smithi049 ceph-mon[31379]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:07 smithi107 ceph-mon[31859]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:08.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:07 smithi005 ceph-mon[28006]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:10.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:09 smithi049 ceph-mon[31379]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:09 smithi107 ceph-mon[31859]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:10.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:09 smithi005 ceph-mon[28006]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:11 smithi049 ceph-mon[31379]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:11 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:11 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:11 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:11 smithi107 ceph-mon[31859]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:11 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:11 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:11 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:12.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:11 smithi005 ceph-mon[28006]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:12.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:11 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:12.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:11 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:12.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:11 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:13 smithi049 ceph-mon[31379]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:13 smithi107 ceph-mon[31859]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:14.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:13 smithi005 ceph-mon[28006]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:14.724 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-04-22T13:22:14.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.726+0000 7f3d99ffb700 1 -- 172.21.15.5:0/3543051679 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3d9c062e80 con 0x7f3d840a3c40 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d840a3c40 msgr2=0x7f3d840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d840a3c40 0x7f3d840a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009fd0 tx=0x7f3d9400b7b0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 msgr2=0x7f3d9c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 0x7f3d9c101ce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b530 tx=0x7f3d9000b840 comp rx=0 tx=0).stop 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 shutdown_connections 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d840a3c40 0x7f3d840a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d9c100d50 0x7f3d9c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9c1046d0 0x7f3d9c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 --2- 172.21.15.5:0/3543051679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c103d30 0x7f3d9c101ce0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 >> 172.21.15.5:0/3543051679 conn(0x7f3d9c0fc6d0 msgr2=0x7f3d9c107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 shutdown_connections 2024-04-22T13:22:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:22:14.729+0000 7f3da3023700 1 -- 172.21.15.5:0/3543051679 wait complete. 2024-04-22T13:22:15.429 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.6.service 2024-04-22T13:22:15.432 INFO:tasks.cephadm:Deploying osd.7 on smithi049 with /dev/vg_nvme/lv_2... 2024-04-22T13:22:15.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T13:22:15.692 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:22:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:15 smithi049 ceph-mon[31379]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:15 smithi049 ceph-mon[31379]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T13:22:15.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:15.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:15 smithi107 ceph-mon[31859]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:16.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:15 smithi107 ceph-mon[31859]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T13:22:16.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:15 smithi005 ceph-mon[28006]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:16.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:15 smithi005 ceph-mon[28006]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T13:22:16.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:16.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:16 smithi049 ceph-mon[31379]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T13:22:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:16 smithi049 ceph-mon[31379]: osdmap e46: 7 total, 6 up, 7 in 2024-04-22T13:22:17.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:16 smithi049 ceph-mon[31379]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:22:17.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:16 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:17.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:16 smithi005 ceph-mon[28006]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T13:22:17.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:16 smithi005 ceph-mon[28006]: osdmap e46: 7 total, 6 up, 7 in 2024-04-22T13:22:17.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:16 smithi005 ceph-mon[28006]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:22:17.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:16 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:17.059 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:22:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:22:16.987+0000 7f38d433d700 -1 osd.6 0 waiting for initial osdmap 2024-04-22T13:22:17.059 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:22:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:22:16.997+0000 7f38d0130700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:22:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:16 smithi107 ceph-mon[31859]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T13:22:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:16 smithi107 ceph-mon[31859]: osdmap e46: 7 total, 6 up, 7 in 2024-04-22T13:22:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:16 smithi107 ceph-mon[31859]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:22:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:16 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:17.672 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T13:22:17.672 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T13:22:17.672 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-04-22T13:22:17.672 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-04-22T13:22:17.672 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00931354 s, 1.1 GB/s 2024-04-22T13:22:17.672 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-04-22T13:22:18.230 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:17 smithi049 ceph-mon[31379]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:18.230 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:17 smithi049 ceph-mon[31379]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:22:18.231 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:17 smithi049 ceph-mon[31379]: osdmap e47: 7 total, 6 up, 7 in 2024-04-22T13:22:18.231 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:18.231 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:18.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-04-22T13:22:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:17 smithi107 ceph-mon[31859]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:17 smithi107 ceph-mon[31859]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:22:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:17 smithi107 ceph-mon[31859]: osdmap e47: 7 total, 6 up, 7 in 2024-04-22T13:22:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:18.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:17 smithi005 ceph-mon[28006]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:22:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:17 smithi005 ceph-mon[28006]: from='osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:22:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:17 smithi005 ceph-mon[28006]: osdmap e47: 7 total, 6 up, 7 in 2024-04-22T13:22:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:22:19.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:18 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:22:19.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:18 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:22:19.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:18 smithi005 ceph-mon[28006]: osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] boot 2024-04-22T13:22:19.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:18 smithi005 ceph-mon[28006]: osdmap e48: 7 total, 7 up, 7 in 2024-04-22T13:22:19.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:18 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:18 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:22:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:18 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:22:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:18 smithi049 ceph-mon[31379]: osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] boot 2024-04-22T13:22:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:18 smithi049 ceph-mon[31379]: osdmap e48: 7 total, 7 up, 7 in 2024-04-22T13:22:19.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:18 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:19.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:18 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:22:19.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:18 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:22:19.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:18 smithi107 ceph-mon[31859]: osd.6 [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] boot 2024-04-22T13:22:19.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:18 smithi107 ceph-mon[31859]: osdmap e48: 7 total, 7 up, 7 in 2024-04-22T13:22:19.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:18 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:22:20.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.022+0000 7f0b267fc700 1 -- 172.21.15.49:0/264123938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b20108850 msgr2=0x7f0b2010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:20.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.022+0000 7f0b267fc700 1 --2- 172.21.15.49:0/264123938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b20108850 0x7f0b2010ac40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800b0a0 tx=0x7f0b1800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:22:20.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 -- 172.21.15.49:0/264123938 shutdown_connections 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 --2- 172.21.15.49:0/264123938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b20108850 0x7f0b2010ac40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 --2- 172.21.15.49:0/264123938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b201035f0 0x7f0b201059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 --2- 172.21.15.49:0/264123938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b20105f20 0x7f0b20108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 -- 172.21.15.49:0/264123938 >> 172.21.15.49:0/264123938 conn(0x7f0b200fcf40 msgr2=0x7f0b200ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 -- 172.21.15.49:0/264123938 shutdown_connections 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 -- 172.21.15.49:0/264123938 wait complete. 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 Processor -- start 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.023+0000 7f0b267fc700 1 -- start start 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 0x7f0b20102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105f20 0x7f0b201015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b20108850 0x7f0b20101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010e510 con 0x7f0b20108850 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010e390 con 0x7f0b201035f0 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010e690 con 0x7f0b20105f20 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 0x7f0b20102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 0x7f0b20102f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60896/0 (socket says 172.21.15.49:60896) 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 -- 172.21.15.49:0/3987252169 learned_addr learned my addr 172.21.15.49:0/3987252169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1f7fe700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105f20 0x7f0b201015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b24d99700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b20108850 0x7f0b20101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 -- 172.21.15.49:0/3987252169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105f20 msgr2=0x7f0b201015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105f20 0x7f0b201015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 -- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b20108850 msgr2=0x7f0b20101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b20108850 0x7f0b20101ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 -- 172.21.15.49:0/3987252169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b20102390 con 0x7f0b201035f0 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1ffff700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 0x7f0b20102f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c970 tx=0x7f0b1000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100078d0 con 0x7f0b201035f0 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b10018480 con 0x7f0b201035f0 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000f6b0 con 0x7f0b201035f0 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.024+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201a2d10 con 0x7f0b201035f0 2024-04-22T13:22:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.025+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a3170 con 0x7f0b201035f0 2024-04-22T13:22:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.025+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2002df80 con 0x7f0b201035f0 2024-04-22T13:22:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.027+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0b10018930 con 0x7f0b201035f0 2024-04-22T13:22:20.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.027+0000 7f0b1d7fa700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0b080a3cc0 0x7f0b080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:20.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.027+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f0b100cf9e0 con 0x7f0b201035f0 2024-04-22T13:22:20.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.027+0000 7f0b1f7fe700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0b080a3cc0 0x7f0b080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:20.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.028+0000 7f0b1f7fe700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0b080a3cc0 0x7f0b080a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005950 tx=0x7f0b1400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:20.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.031+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b1009d1f0 con 0x7f0b201035f0 2024-04-22T13:22:20.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:20.249+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0b20062e80 con 0x7f0b080a3cc0 2024-04-22T13:22:20.246 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:20.246 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: osdmap e49: 7 total, 7 up, 7 in 2024-04-22T13:22:20.246 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:20.247 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: osdmap e49: 7 total, 7 up, 7 in 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:20.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:20.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:20.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:20.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:20.288 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: osdmap e49: 7 total, 7 up, 7 in 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:20.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:21.215 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:20 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi005 2024-04-22T13:22:21.215 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:20 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi005 to 5697M 2024-04-22T13:22:21.215 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:22:21.215 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:22:21.215 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:21.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:20 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi005 2024-04-22T13:22:21.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:20 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi005 to 5697M 2024-04-22T13:22:21.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:22:21.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:22:21.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:21.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:20 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi005 2024-04-22T13:22:21.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:20 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi005 to 5697M 2024-04-22T13:22:21.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:22:21.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:22:21.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:22.058 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:22 smithi049 ceph-mon[31379]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:22.058 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:22 smithi049 ceph-mon[31379]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:22.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:22 smithi107 ceph-mon[31859]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:22.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:22 smithi107 ceph-mon[31859]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:22.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:22 smithi005 ceph-mon[28006]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:22.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:22 smithi005 ceph-mon[28006]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:23 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/1983591379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]: dispatch 2024-04-22T13:22:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:23 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]: dispatch 2024-04-22T13:22:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:23 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]': finished 2024-04-22T13:22:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:23 smithi049 ceph-mon[31379]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T13:22:23.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:23 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/1983591379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]: dispatch 2024-04-22T13:22:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:23 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]: dispatch 2024-04-22T13:22:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:23 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]': finished 2024-04-22T13:22:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:23 smithi107 ceph-mon[31859]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T13:22:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/1983591379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]: dispatch 2024-04-22T13:22:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:23 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]: dispatch 2024-04-22T13:22:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:23 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49b9738e-63b0-4716-8f64-d9508227837a"}]': finished 2024-04-22T13:22:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:23 smithi005 ceph-mon[28006]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T13:22:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:24 smithi049 ceph-mon[31379]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:24 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/1856018617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:24 smithi107 ceph-mon[31859]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:24 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/1856018617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:24.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:24 smithi005 ceph-mon[28006]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:24.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:24 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/1856018617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:26 smithi049 ceph-mon[31379]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:26.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:26 smithi107 ceph-mon[31859]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:26.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:26 smithi005 ceph-mon[28006]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:28.082 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:28 smithi049 ceph-mon[31379]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:28.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:28 smithi107 ceph-mon[31859]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:28.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:28 smithi005 ceph-mon[28006]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:30.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:30 smithi049 ceph-mon[31379]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:30 smithi107 ceph-mon[31859]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:30.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:30 smithi005 ceph-mon[28006]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:31.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T13:22:31.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:31.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T13:22:31.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:31.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T13:22:31.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:32.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:32 smithi049 ceph-mon[31379]: Deploying daemon osd.7 on smithi049 2024-04-22T13:22:32.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:32 smithi049 ceph-mon[31379]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:32 smithi107 ceph-mon[31859]: Deploying daemon osd.7 on smithi049 2024-04-22T13:22:32.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:32 smithi107 ceph-mon[31859]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:32.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:32 smithi005 ceph-mon[28006]: Deploying daemon osd.7 on smithi049 2024-04-22T13:22:32.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:32 smithi005 ceph-mon[28006]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:34.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:34 smithi049 ceph-mon[31379]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:34 smithi107 ceph-mon[31859]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:34 smithi005 ceph-mon[28006]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:36.199 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:36 smithi049 ceph-mon[31379]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:36.199 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:36.199 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:36.199 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:36 smithi107 ceph-mon[31859]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:36.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:36 smithi005 ceph-mon[28006]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:36.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:36.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:36.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:38.226 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:38 smithi049 ceph-mon[31379]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:38.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:38 smithi107 ceph-mon[31859]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:38.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:38 smithi005 ceph-mon[28006]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:39.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.217+0000 7f0b1d7fa700 1 -- 172.21.15.49:0/3987252169 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b20062e80 con 0x7f0b080a3cc0 2024-04-22T13:22:39.215 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 7 on host 'smithi049' 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0b080a3cc0 msgr2=0x7f0b080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0b080a3cc0 0x7f0b080a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005950 tx=0x7f0b1400a300 comp rx=0 tx=0).stop 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 msgr2=0x7f0b20102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 0x7f0b20102f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c970 tx=0x7f0b1000cc80 comp rx=0 tx=0).stop 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 shutdown_connections 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0b080a3cc0 0x7f0b080a6180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b201035f0 0x7f0b20102f50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b20108850 0x7f0b20101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 --2- 172.21.15.49:0/3987252169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105f20 0x7f0b201015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 >> 172.21.15.49:0/3987252169 conn(0x7f0b200fcf40 msgr2=0x7f0b20103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:39.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 shutdown_connections 2024-04-22T13:22:39.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:22:39.219+0000 7f0b267fc700 1 -- 172.21.15.49:0/3987252169 wait complete. 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.434 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:39.781 DEBUG:teuthology.orchestra.run.smithi049:osd.7> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.7.service 2024-04-22T13:22:39.783 INFO:tasks.cephadm:Deploying osd.8 on smithi107 with /dev/vg_nvme/lv_2... 2024-04-22T13:22:39.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:39.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:40.038 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:22:40.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:40 smithi049 ceph-mon[31379]: from='osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:22:40.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:40 smithi049 ceph-mon[31379]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:22:40.776 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:40 smithi107 ceph-mon[31859]: from='osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:22:40.776 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:40 smithi107 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:22:40.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:40 smithi005 ceph-mon[28006]: from='osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:22:40.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:40 smithi005 ceph-mon[28006]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:22:41.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:41 smithi049 ceph-mon[31379]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:41.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:41 smithi049 ceph-mon[31379]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T13:22:41.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:41 smithi049 ceph-mon[31379]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T13:22:41.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:41.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:41 smithi049 ceph-mon[31379]: from='osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:22:41.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:41 smithi049 ceph-mon[31379]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:22:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:41 smithi107 ceph-mon[31859]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:41 smithi107 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T13:22:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:41 smithi107 ceph-mon[31859]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T13:22:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:41.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:41 smithi107 ceph-mon[31859]: from='osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:22:41.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:41 smithi107 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:22:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:41 smithi005 ceph-mon[28006]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:41 smithi005 ceph-mon[28006]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T13:22:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:41 smithi005 ceph-mon[28006]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T13:22:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:41 smithi005 ceph-mon[28006]: from='osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:22:41.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:41 smithi005 ceph-mon[28006]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:22:41.908 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T13:22:41.908 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T13:22:41.908 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T13:22:41.908 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T13:22:41.908 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125826 s, 833 MB/s 2024-04-22T13:22:41.908 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: osdmap e52: 8 total, 7 up, 8 in 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: osdmap e53: 8 total, 7 up, 8 in 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-mon[31379]: from='osd.7 ' entity='osd.7' 2024-04-22T13:22:42.522 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:22:42.268+0000 7f761bbb3700 -1 osd.7 0 waiting for initial osdmap 2024-04-22T13:22:42.523 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:22:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:22:42.279+0000 7f761399e700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: osdmap e52: 8 total, 7 up, 8 in 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: osdmap e53: 8 total, 7 up, 8 in 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.650 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:42 smithi107 ceph-mon[31859]: from='osd.7 ' entity='osd.7' 2024-04-22T13:22:42.676 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: osdmap e52: 8 total, 7 up, 8 in 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: osdmap e53: 8 total, 7 up, 8 in 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:42.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:42 smithi005 ceph-mon[28006]: from='osd.7 ' entity='osd.7' 2024-04-22T13:22:42.920 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:22:43.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:22:43.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:22:43.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi049 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi049 to 5696M 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] boot 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T13:22:43.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi049 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi049 to 5696M 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] boot 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T13:22:43.773 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi049 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi049 to 5696M 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: osd.7 [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] boot 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T13:22:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:22:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 -- 172.21.15.107:0/2628538656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 msgr2=0x7f249c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 --2- 172.21.15.107:0/2628538656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c104100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f248c009990 tx=0x7f248c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 -- 172.21.15.107:0/2628538656 shutdown_connections 2024-04-22T13:22:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 --2- 172.21.15.107:0/2628538656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f249c104640 0x7f249c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 --2- 172.21.15.107:0/2628538656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c102aa0 0x7f249c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 --2- 172.21.15.107:0/2628538656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c104100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 -- 172.21.15.107:0/2628538656 >> 172.21.15.107:0/2628538656 conn(0x7f249c0fe040 msgr2=0x7f249c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 -- 172.21.15.107:0/2628538656 shutdown_connections 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 -- 172.21.15.107:0/2628538656 wait complete. 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 Processor -- start 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.432+0000 7f24a351a700 1 -- start start 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f249c102aa0 0x7f249c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c104640 0x7f249c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f249c10e630 con 0x7f249c104640 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f249c10e4b0 con 0x7f249c102aa0 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f249c10e7b0 con 0x7f249c103ca0 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33618/0 (socket says 172.21.15.107:33618) 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 -- 172.21.15.107:0/1674483967 learned_addr learned my addr 172.21.15.107:0/1674483967 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a12b6700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f249c102aa0 0x7f249c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a1ab7700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c104640 0x7f249c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 -- 172.21.15.107:0/1674483967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f249c102aa0 msgr2=0x7f249c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f249c102aa0 0x7f249c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 -- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c104640 msgr2=0x7f249c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c104640 0x7f249c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 -- 172.21.15.107:0/1674483967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c1a0940 con 0x7f249c103ca0 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a1ab7700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c104640 0x7f249c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a0ab5700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c199230 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f248c0036f0 tx=0x7f248c003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c027070 con 0x7f249c103ca0 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248c022470 con 0x7f249c103ca0 2024-04-22T13:22:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249c1a0bd0 con 0x7f249c103ca0 2024-04-22T13:22:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c00f6c0 con 0x7f249c103ca0 2024-04-22T13:22:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.433+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249c1a1020 con 0x7f249c103ca0 2024-04-22T13:22:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.434+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249c02df80 con 0x7f249c103ca0 2024-04-22T13:22:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.436+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f248c02a070 con 0x7f249c103ca0 2024-04-22T13:22:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.436+0000 7f24927fc700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f24880a3c40 0x7f24880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.436+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7f248c0d0f90 con 0x7f249c103ca0 2024-04-22T13:22:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.436+0000 7f24a12b6700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f24880a3c40 0x7f24880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.437+0000 7f24a12b6700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f24880a3c40 0x7f24880a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f249c19a1d0 tx=0x7f2498009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.440+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f248c031070 con 0x7f249c103ca0 2024-04-22T13:22:44.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:22:44.646+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f249c062e80 con 0x7f24880a3c40 2024-04-22T13:22:45.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:45 smithi107 ceph-mon[31859]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:45.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:45 smithi107 ceph-mon[31859]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T13:22:45.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:45 smithi107 ceph-mon[31859]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:45.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:22:45.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:22:45.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:45.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:45 smithi049 ceph-mon[31379]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:45.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:45 smithi049 ceph-mon[31379]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T13:22:45.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:45 smithi049 ceph-mon[31379]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:45.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:22:45.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:22:45.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:45.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:45 smithi005 ceph-mon[28006]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:45.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:45 smithi005 ceph-mon[28006]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T13:22:45.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:45 smithi005 ceph-mon[28006]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:45.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:22:45.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:22:45.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:46.434 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:46 smithi107 ceph-mon[31859]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T13:22:46.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:46 smithi049 ceph-mon[31379]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T13:22:46.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:46 smithi005 ceph-mon[28006]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T13:22:47.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:47 smithi049 ceph-mon[31379]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:47.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:47 smithi049 ceph-mon[31379]: osdmap e57: 8 total, 8 up, 8 in 2024-04-22T13:22:47.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:47 smithi107 ceph-mon[31859]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:47.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:47 smithi107 ceph-mon[31859]: osdmap e57: 8 total, 8 up, 8 in 2024-04-22T13:22:47.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:47 smithi005 ceph-mon[28006]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:47.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:47 smithi005 ceph-mon[28006]: osdmap e57: 8 total, 8 up, 8 in 2024-04-22T13:22:48.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: osdmap e58: 8 total, 8 up, 8 in 2024-04-22T13:22:48.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/415272820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]: dispatch 2024-04-22T13:22:48.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]: dispatch 2024-04-22T13:22:48.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]': finished 2024-04-22T13:22:48.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T13:22:48.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:22:48.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:48 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/2511671578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:48.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: osdmap e58: 8 total, 8 up, 8 in 2024-04-22T13:22:48.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/415272820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]: dispatch 2024-04-22T13:22:48.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]: dispatch 2024-04-22T13:22:48.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]': finished 2024-04-22T13:22:48.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T13:22:48.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:22:48.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:48 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/2511671578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: osdmap e58: 8 total, 8 up, 8 in 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/415272820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]: dispatch 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]: dispatch 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759"}]': finished 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:22:48.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:48 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/2511671578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:22:49.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:49 smithi005 ceph-mon[28006]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:49.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:49 smithi049 ceph-mon[31379]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:49 smithi107 ceph-mon[31859]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:51.514 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:51 smithi107 ceph-mon[31859]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:51.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:51 smithi005 ceph-mon[28006]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:51.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:51 smithi049 ceph-mon[31379]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:53.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:53 smithi005 ceph-mon[28006]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:53.607 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:53 smithi107 ceph-mon[31859]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:53.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:53 smithi049 ceph-mon[31379]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:55.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:55 smithi107 ceph-mon[31859]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T13:22:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:55 smithi107 ceph-mon[31859]: Deploying daemon osd.8 on smithi107 2024-04-22T13:22:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:55 smithi005 ceph-mon[28006]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T13:22:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:55 smithi005 ceph-mon[28006]: Deploying daemon osd.8 on smithi107 2024-04-22T13:22:55.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:55 smithi049 ceph-mon[31379]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:55.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T13:22:55.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:55.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:55 smithi049 ceph-mon[31379]: Deploying daemon osd.8 on smithi107 2024-04-22T13:22:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:57 smithi107 ceph-mon[31859]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:57 smithi005 ceph-mon[28006]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:57.573 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:57 smithi049 ceph-mon[31379]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:22:59 smithi005 ceph-mon[28006]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:59.588 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:22:59 smithi107 ceph-mon[31859]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:22:59.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:22:59 smithi049 ceph-mon[31379]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:01.496 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:01 smithi107 ceph-mon[31859]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:01.496 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:01.496 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:01.496 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:01.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:01 smithi049 ceph-mon[31379]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:01.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:01.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:01.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:01.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:01 smithi005 ceph-mon[28006]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:03.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.669+0000 7f24927fc700 1 -- 172.21.15.107:0/1674483967 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f249c062e80 con 0x7f24880a3c40 2024-04-22T13:23:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 8 on host 'smithi107' 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f24880a3c40 msgr2=0x7f24880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f24880a3c40 0x7f24880a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f249c19a1d0 tx=0x7f2498009190 comp rx=0 tx=0).stop 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 msgr2=0x7f249c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c199230 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f248c0036f0 tx=0x7f248c003940 comp rx=0 tx=0).stop 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 shutdown_connections 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f24880a3c40 0x7f24880a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f249c102aa0 0x7f249c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f249c104640 0x7f249c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 --2- 172.21.15.107:0/1674483967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f249c103ca0 0x7f249c199230 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 >> 172.21.15.107:0/1674483967 conn(0x7f249c0fe040 msgr2=0x7f249c106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:03.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 shutdown_connections 2024-04-22T13:23:03.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:23:03.671+0000 7f24a351a700 1 -- 172.21.15.107:0/1674483967 wait complete. 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.225 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.254 DEBUG:teuthology.orchestra.run.smithi107:osd.8> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.8.service 2024-04-22T13:23:04.256 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-04-22T13:23:04.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:04.514 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:04 smithi005 ceph-mon[28006]: from='osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:23:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:04 smithi005 ceph-mon[28006]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:23:05.206 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:04 smithi107 ceph-mon[31859]: from='osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:23:05.206 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:04 smithi107 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:23:05.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:04 smithi049 ceph-mon[31379]: from='osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:23:05.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:04 smithi049 ceph-mon[31379]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:23:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:05 smithi049 ceph-mon[31379]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:05 smithi049 ceph-mon[31379]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T13:23:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:05 smithi049 ceph-mon[31379]: osdmap e60: 9 total, 8 up, 9 in 2024-04-22T13:23:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:05 smithi049 ceph-mon[31379]: from='osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:23:06.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:05 smithi049 ceph-mon[31379]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:23:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:05 smithi107 ceph-mon[31859]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:05 smithi107 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T13:23:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:05 smithi107 ceph-mon[31859]: osdmap e60: 9 total, 8 up, 9 in 2024-04-22T13:23:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:05 smithi107 ceph-mon[31859]: from='osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:23:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:05 smithi107 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:23:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:05 smithi005 ceph-mon[28006]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:05 smithi005 ceph-mon[28006]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T13:23:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:05 smithi005 ceph-mon[28006]: osdmap e60: 9 total, 8 up, 9 in 2024-04-22T13:23:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:05 smithi005 ceph-mon[28006]: from='osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:23:06.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:05 smithi005 ceph-mon[28006]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T13:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T13:23:06.510 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-22T13:23:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-22T13:23:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00846053 s, 1.2 GB/s 2024-04-22T13:23:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-22T13:23:06.979 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:23:06.979 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-mon[31859]: osdmap e61: 9 total, 8 up, 9 in 2024-04-22T13:23:06.979 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:06.979 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:06.979 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-mon[31859]: from='osd.8 ' entity='osd.8' 2024-04-22T13:23:06.980 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:23:06.789+0000 7f99871d1700 -1 osd.8 0 waiting for initial osdmap 2024-04-22T13:23:06.980 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:23:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:23:06.949+0000 7f997efbc700 -1 osd.8 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:23:07.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:06 smithi005 ceph-mon[28006]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:23:07.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:06 smithi005 ceph-mon[28006]: osdmap e61: 9 total, 8 up, 9 in 2024-04-22T13:23:07.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:06 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:07.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:06 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:07.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:06 smithi005 ceph-mon[28006]: from='osd.8 ' entity='osd.8' 2024-04-22T13:23:07.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:06 smithi049 ceph-mon[31379]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:23:07.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:06 smithi049 ceph-mon[31379]: osdmap e61: 9 total, 8 up, 9 in 2024-04-22T13:23:07.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:06 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:07.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:06 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:07.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:06 smithi049 ceph-mon[31379]: from='osd.8 ' entity='osd.8' 2024-04-22T13:23:07.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-04-22T13:23:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: osdmap e62: 9 total, 8 up, 9 in 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi107 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:08.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:08.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: osdmap e62: 9 total, 8 up, 9 in 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi107 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:08.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:08.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: osdmap e62: 9 total, 8 up, 9 in 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi107 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:08.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:09.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:08 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi107 to 5696M 2024-04-22T13:23:09.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:08 smithi005 ceph-mon[28006]: osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] boot 2024-04-22T13:23:09.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:08 smithi005 ceph-mon[28006]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T13:23:09.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:08 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:09.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:08 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi107 to 5696M 2024-04-22T13:23:09.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:08 smithi049 ceph-mon[31379]: osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] boot 2024-04-22T13:23:09.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:08 smithi049 ceph-mon[31379]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T13:23:09.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:08 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:09.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:08 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi107 to 5696M 2024-04-22T13:23:09.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:08 smithi107 ceph-mon[31859]: osd.8 [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] boot 2024-04-22T13:23:09.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:08 smithi107 ceph-mon[31859]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T13:23:09.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:08 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:23:09.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 -- 172.21.15.5:0/3177628860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 msgr2=0x7fa34410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 --2- 172.21.15.5:0/3177628860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa34410af10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00b0d0 tx=0x7fa33c00b3e0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 -- 172.21.15.5:0/3177628860 shutdown_connections 2024-04-22T13:23:09.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 --2- 172.21.15.5:0/3177628860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344102ac0 0x7fa344102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 --2- 172.21.15.5:0/3177628860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa34410af10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 --2- 172.21.15.5:0/3177628860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344103cc0 0x7fa344104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 -- 172.21.15.5:0/3177628860 >> 172.21.15.5:0/3177628860 conn(0x7fa3440fe060 msgr2=0x7fa3441004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 -- 172.21.15.5:0/3177628860 shutdown_connections 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.288+0000 7fa349f43700 1 -- 172.21.15.5:0/3177628860 wait complete. 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 Processor -- start 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 -- start start 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344102ac0 0x7fa344198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103cc0 0x7fa344199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa3441a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa34410e560 con 0x7fa344104660 2024-04-22T13:23:09.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa34410e3e0 con 0x7fa344103cc0 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa349f43700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa34410e6e0 con 0x7fa344102ac0 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa3441a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa3441a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35312/0 (socket says 172.21.15.5:35312) 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 -- 172.21.15.5:0/3290268012 learned_addr learned my addr 172.21.15.5:0/3290268012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa342ffd700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103cc0 0x7fa344199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa3437fe700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344102ac0 0x7fa344198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 -- 172.21.15.5:0/3290268012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344102ac0 msgr2=0x7fa344198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344102ac0 0x7fa344198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 -- 172.21.15.5:0/3290268012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103cc0 msgr2=0x7fa344199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.289+0000 7fa343fff700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103cc0 0x7fa344199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa343fff700 1 -- 172.21.15.5:0/3290268012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3441a07b0 con 0x7fa344104660 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa342ffd700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103cc0 0x7fa344199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa343fff700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa3441a0110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009850 tx=0x7fa33c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c01f030 con 0x7fa344104660 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3441a0a40 con 0x7fa344104660 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c007a80 con 0x7fa344104660 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c01b690 con 0x7fa344104660 2024-04-22T13:23:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.290+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a0ef0 con 0x7fa344104660 2024-04-22T13:23:09.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.291+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa344069730 con 0x7fa344104660 2024-04-22T13:23:09.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.292+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fa33c022030 con 0x7fa344104660 2024-04-22T13:23:09.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.293+0000 7fa340ff9700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3c40 0x7fa32c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.293+0000 7fa3437fe700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3c40 0x7fa32c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.293+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5858+0+0 (secure 0 0 0) 0x7fa33c00f020 con 0x7fa344104660 2024-04-22T13:23:09.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.293+0000 7fa3437fe700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3c40 0x7fa32c0a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa344103b20 tx=0x7fa334009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.299+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa33c029020 con 0x7fa344104660 2024-04-22T13:23:09.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:09.516+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa344062e80 con 0x7fa32c0a3c40 2024-04-22T13:23:10.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:09 smithi005 ceph-mon[28006]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:10.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:09 smithi005 ceph-mon[28006]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T13:23:10.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:23:10.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:23:10.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:10.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:09 smithi049 ceph-mon[31379]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:10.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:09 smithi049 ceph-mon[31379]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T13:23:10.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:23:10.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:23:10.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:09 smithi107 ceph-mon[31859]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:23:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:09 smithi107 ceph-mon[31859]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T13:23:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:23:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:23:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:11.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:10 smithi005 ceph-mon[28006]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:11.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:10 smithi005 ceph-mon[28006]: osdmap e65: 9 total, 9 up, 9 in 2024-04-22T13:23:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:10 smithi049 ceph-mon[31379]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:11.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:10 smithi049 ceph-mon[31379]: osdmap e65: 9 total, 9 up, 9 in 2024-04-22T13:23:11.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:10 smithi107 ceph-mon[31859]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:11.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:10 smithi107 ceph-mon[31859]: osdmap e65: 9 total, 9 up, 9 in 2024-04-22T13:23:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:11 smithi049 ceph-mon[31379]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:11 smithi049 ceph-mon[31379]: osdmap e66: 9 total, 9 up, 9 in 2024-04-22T13:23:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:11 smithi107 ceph-mon[31859]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:11 smithi107 ceph-mon[31859]: osdmap e66: 9 total, 9 up, 9 in 2024-04-22T13:23:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:11 smithi005 ceph-mon[28006]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:11 smithi005 ceph-mon[28006]: osdmap e66: 9 total, 9 up, 9 in 2024-04-22T13:23:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:12 smithi049 ceph-mon[31379]: osdmap e67: 9 total, 9 up, 9 in 2024-04-22T13:23:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:12 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/160963246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2392b48b-bdb9-4875-8725-20edf192d686"}]: dispatch 2024-04-22T13:23:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:12 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/160963246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2392b48b-bdb9-4875-8725-20edf192d686"}]': finished 2024-04-22T13:23:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:12 smithi049 ceph-mon[31379]: osdmap e68: 10 total, 9 up, 10 in 2024-04-22T13:23:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:12 smithi107 ceph-mon[31859]: osdmap e67: 9 total, 9 up, 9 in 2024-04-22T13:23:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:12 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/160963246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2392b48b-bdb9-4875-8725-20edf192d686"}]: dispatch 2024-04-22T13:23:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:12 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/160963246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2392b48b-bdb9-4875-8725-20edf192d686"}]': finished 2024-04-22T13:23:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:12 smithi107 ceph-mon[31859]: osdmap e68: 10 total, 9 up, 10 in 2024-04-22T13:23:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:13.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:12 smithi005 ceph-mon[28006]: osdmap e67: 9 total, 9 up, 9 in 2024-04-22T13:23:13.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:12 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/160963246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2392b48b-bdb9-4875-8725-20edf192d686"}]: dispatch 2024-04-22T13:23:13.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:12 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/160963246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2392b48b-bdb9-4875-8725-20edf192d686"}]': finished 2024-04-22T13:23:13.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:12 smithi005 ceph-mon[28006]: osdmap e68: 10 total, 9 up, 10 in 2024-04-22T13:23:13.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:13 smithi049 ceph-mon[31379]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/796810632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:23:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:13 smithi107 ceph-mon[31859]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/796810632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:23:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:13 smithi005 ceph-mon[28006]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:14.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/796810632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:23:16.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:15 smithi005 ceph-mon[28006]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-22T13:23:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:15 smithi049 ceph-mon[31379]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-22T13:23:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:15 smithi107 ceph-mon[31859]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-22T13:23:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:17 smithi049 ceph-mon[31379]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:23:18.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:17 smithi107 ceph-mon[31859]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:23:18.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:17 smithi005 ceph-mon[28006]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:23:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:19 smithi005 ceph-mon[28006]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-22T13:23:20.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T13:23:20.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:19 smithi049 ceph-mon[31379]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-22T13:23:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T13:23:20.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:20.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:19 smithi107 ceph-mon[31859]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-22T13:23:20.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T13:23:20.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:21.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:20 smithi005 ceph-mon[28006]: Deploying daemon osd.9 on smithi005 2024-04-22T13:23:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:20 smithi049 ceph-mon[31379]: Deploying daemon osd.9 on smithi005 2024-04-22T13:23:21.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:20 smithi107 ceph-mon[31859]: Deploying daemon osd.9 on smithi005 2024-04-22T13:23:22.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:21 smithi005 ceph-mon[28006]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-22T13:23:22.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:21 smithi049 ceph-mon[31379]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-22T13:23:22.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:21 smithi107 ceph-mon[31859]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-22T13:23:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:23 smithi049 ceph-mon[31379]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:23:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:23 smithi107 ceph-mon[31859]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:23:24.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:23 smithi005 ceph-mon[28006]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:23:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:25 smithi049 ceph-mon[31379]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-22T13:23:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:26.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:26.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:25 smithi107 ceph-mon[31859]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-22T13:23:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:26.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:25 smithi005 ceph-mon[28006]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-22T13:23:26.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:26.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:26.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:28.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:27 smithi005 ceph-mon[28006]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:28.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:27 smithi049 ceph-mon[31379]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:28.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:27 smithi107 ceph-mon[31859]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.777+0000 7fa340ff9700 1 -- 172.21.15.5:0/3290268012 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa344062e80 con 0x7fa32c0a3c40 2024-04-22T13:23:29.775 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-04-22T13:23:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3c40 msgr2=0x7fa32c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3c40 0x7fa32c0a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa344103b20 tx=0x7fa334009260 comp rx=0 tx=0).stop 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 msgr2=0x7fa3441a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa3441a0110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009850 tx=0x7fa33c004e20 comp rx=0 tx=0).stop 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 shutdown_connections 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344104660 0x7fa3441a0110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344102ac0 0x7fa344198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3c40 0x7fa32c0a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 --2- 172.21.15.5:0/3290268012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103cc0 0x7fa344199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 >> 172.21.15.5:0/3290268012 conn(0x7fa3440fe060 msgr2=0x7fa344106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 shutdown_connections 2024-04-22T13:23:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:23:29.780+0000 7fa349f43700 1 -- 172.21.15.5:0/3290268012 wait complete. 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:29 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:30.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:29 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:30.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:29 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:30.623 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.9.service 2024-04-22T13:23:30.626 INFO:tasks.cephadm:Deploying osd.10 on smithi049 with /dev/vg_nvme/lv_1... 2024-04-22T13:23:30.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T13:23:30.883 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:23:31.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:30 smithi005 ceph-mon[28006]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T13:23:31.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:30 smithi005 ceph-mon[28006]: osdmap e69: 10 total, 9 up, 10 in 2024-04-22T13:23:31.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:30 smithi005 ceph-mon[28006]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:23:31.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:30 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:31.237 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:30 smithi049 ceph-mon[31379]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T13:23:31.237 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:30 smithi049 ceph-mon[31379]: osdmap e69: 10 total, 9 up, 10 in 2024-04-22T13:23:31.237 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:30 smithi049 ceph-mon[31379]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:23:31.237 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:30 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:31.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:30 smithi107 ceph-mon[31859]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T13:23:31.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:30 smithi107 ceph-mon[31859]: osdmap e69: 10 total, 9 up, 10 in 2024-04-22T13:23:31.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:30 smithi107 ceph-mon[31859]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-22T13:23:31.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:30 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:31.728 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:23:31.690+0000 7f8e6fe8d700 -1 osd.9 0 waiting for initial osdmap 2024-04-22T13:23:31.728 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:23:31.704+0000 7f8e67c78700 -1 osd.9 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:23:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-mon[28006]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-mon[28006]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:23:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-mon[28006]: osdmap e70: 10 total, 9 up, 10 in 2024-04-22T13:23:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:31 smithi005 ceph-mon[28006]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' 2024-04-22T13:23:32.202 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:31 smithi049 ceph-mon[31379]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:32.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:31 smithi049 ceph-mon[31379]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:23:32.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:31 smithi049 ceph-mon[31379]: osdmap e70: 10 total, 9 up, 10 in 2024-04-22T13:23:32.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:32.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:32.203 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:31 smithi049 ceph-mon[31379]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' 2024-04-22T13:23:32.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:31 smithi107 ceph-mon[31859]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:32.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:31 smithi107 ceph-mon[31859]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-22T13:23:32.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:31 smithi107 ceph-mon[31859]: osdmap e70: 10 total, 9 up, 10 in 2024-04-22T13:23:32.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:32.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:32.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:31 smithi107 ceph-mon[31859]: from='osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882]' entity='osd.9' 2024-04-22T13:23:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T13:23:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T13:23:32.781 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-04-22T13:23:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-04-22T13:23:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00956018 s, 1.1 GB/s 2024-04-22T13:23:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-04-22T13:23:33.130 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:32 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:23:33.131 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:32 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:23:33.131 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:32 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:33.131 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:32 smithi049 ceph-mon[31379]: osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] boot 2024-04-22T13:23:33.131 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:32 smithi049 ceph-mon[31379]: osdmap e71: 10 total, 10 up, 10 in 2024-04-22T13:23:33.131 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:32 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:33.210 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:32 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:23:33.210 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:32 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:23:33.210 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:32 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:33.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:32 smithi005 ceph-mon[28006]: osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] boot 2024-04-22T13:23:33.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:32 smithi005 ceph-mon[28006]: osdmap e71: 10 total, 10 up, 10 in 2024-04-22T13:23:33.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:32 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:32 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:23:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:32 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:23:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:32 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:32 smithi107 ceph-mon[31859]: osd.9 [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] boot 2024-04-22T13:23:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:32 smithi107 ceph-mon[31859]: osdmap e71: 10 total, 10 up, 10 in 2024-04-22T13:23:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:32 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:23:33.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-04-22T13:23:33.681 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:23:34.100 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:34.100 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:34.101 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:34.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:34.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 -- 172.21.15.49:0/3703022212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c104640 msgr2=0x7f9c8c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/3703022212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c104640 0x7f9c8c10aef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0099c0 tx=0x7f9c7c009cd0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 -- 172.21.15.49:0/3703022212 shutdown_connections 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/3703022212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c104640 0x7f9c8c10aef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/3703022212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c102aa0 0x7f9c8c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/3703022212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 -- 172.21.15.49:0/3703022212 >> 172.21.15.49:0/3703022212 conn(0x7f9c8c0fe040 msgr2=0x7f9c8c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 -- 172.21.15.49:0/3703022212 shutdown_connections 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.212+0000 7f9c93d1a700 1 -- 172.21.15.49:0/3703022212 wait complete. 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 Processor -- start 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 -- start start 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c102aa0 0x7f9c8c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c104640 0x7f9c8c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10e540 con 0x7f9c8c104640 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10e3c0 con 0x7f9c8c103ca0 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c93d1a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10e6c0 con 0x7f9c8c102aa0 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44156/0 (socket says 172.21.15.49:44156) 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 -- 172.21.15.49:0/1082201809 learned_addr learned my addr 172.21.15.49:0/1082201809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c91ab6700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c102aa0 0x7f9c8c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c922b7700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c104640 0x7f9c8c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 -- 172.21.15.49:0/1082201809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c102aa0 msgr2=0x7f9c8c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c102aa0 0x7f9c8c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 -- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c104640 msgr2=0x7f9c8c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c104640 0x7f9c8c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.213+0000 7f9c912b5700 1 -- 172.21.15.49:0/1082201809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c075c20 con 0x7f9c8c103ca0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.214+0000 7f9c912b5700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c074e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c9b0 tx=0x7f9c8000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.214+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80007b00 con 0x7f9c8c103ca0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.214+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c1a2c50 con 0x7f9c8c103ca0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.214+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c1a3190 con 0x7f9c8c103ca0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.214+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8000ce50 con 0x7f9c8c103ca0 2024-04-22T13:23:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.214+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80005490 con 0x7f9c8c103ca0 2024-04-22T13:23:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.215+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c069730 con 0x7f9c8c103ca0 2024-04-22T13:23:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.216+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9c80004750 con 0x7f9c8c103ca0 2024-04-22T13:23:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.217+0000 7f9c8affd700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3cc0 0x7f9c740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.217+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6261+0+0 (secure 0 0 0) 0x7f9c800cfc10 con 0x7f9c8c103ca0 2024-04-22T13:23:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.217+0000 7f9c91ab6700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3cc0 0x7f9c740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.218+0000 7f9c91ab6700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3cc0 0x7f9c740a6180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c840060b0 tx=0x7f9c8400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.221+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c80014030 con 0x7f9c8c103ca0 2024-04-22T13:23:35.217 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:34 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi005 2024-04-22T13:23:35.217 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:34 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi005 to 4273M 2024-04-22T13:23:35.218 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:34 smithi049 ceph-mon[31379]: osdmap e72: 10 total, 10 up, 10 in 2024-04-22T13:23:35.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:34 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi005 2024-04-22T13:23:35.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:34 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi005 to 4273M 2024-04-22T13:23:35.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:34 smithi107 ceph-mon[31859]: osdmap e72: 10 total, 10 up, 10 in 2024-04-22T13:23:35.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:34 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi005 2024-04-22T13:23:35.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:34 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi005 to 4273M 2024-04-22T13:23:35.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:34 smithi005 ceph-mon[28006]: osdmap e72: 10 total, 10 up, 10 in 2024-04-22T13:23:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:35.437+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9c8c062e80 con 0x7f9c740a3cc0 2024-04-22T13:23:36.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:35 smithi005 ceph-mon[28006]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:36.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:35 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:23:36.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:35 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:23:36.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:35 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:36.251 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:35 smithi049 ceph-mon[31379]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:36.251 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:35 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:23:36.251 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:35 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:23:36.252 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:35 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:35 smithi107 ceph-mon[31859]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:35 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:23:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:35 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:23:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:35 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:37.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:36 smithi107 ceph-mon[31859]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:37.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:36 smithi049 ceph-mon[31379]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:36 smithi005 ceph-mon[28006]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:38.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:37 smithi049 ceph-mon[31379]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:38.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:37 smithi107 ceph-mon[31859]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:38.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:37 smithi005 ceph-mon[28006]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:38 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/1497432911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]: dispatch 2024-04-22T13:23:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:38 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]: dispatch 2024-04-22T13:23:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:38 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]': finished 2024-04-22T13:23:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:38 smithi049 ceph-mon[31379]: osdmap e73: 11 total, 10 up, 11 in 2024-04-22T13:23:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:38 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:39.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:38 smithi049 ceph-mon[31379]: from='client.? 172.21.15.49:0/3135423445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:23:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:38 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/1497432911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]: dispatch 2024-04-22T13:23:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:38 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]: dispatch 2024-04-22T13:23:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:38 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]': finished 2024-04-22T13:23:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:38 smithi107 ceph-mon[31859]: osdmap e73: 11 total, 10 up, 11 in 2024-04-22T13:23:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:38 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:38 smithi107 ceph-mon[31859]: from='client.? 172.21.15.49:0/3135423445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:23:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:38 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/1497432911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]: dispatch 2024-04-22T13:23:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:38 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]: dispatch 2024-04-22T13:23:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:38 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b"}]': finished 2024-04-22T13:23:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:38 smithi005 ceph-mon[28006]: osdmap e73: 11 total, 10 up, 11 in 2024-04-22T13:23:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:38 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:38 smithi005 ceph-mon[28006]: from='client.? 172.21.15.49:0/3135423445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:23:40.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:39 smithi049 ceph-mon[31379]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:40.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:39 smithi107 ceph-mon[31859]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:40.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:39 smithi005 ceph-mon[28006]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:42.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:41 smithi049 ceph-mon[31379]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:42.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:41 smithi107 ceph-mon[31859]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:42.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:41 smithi005 ceph-mon[28006]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:44.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:43 smithi049 ceph-mon[31379]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:44.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:43 smithi107 ceph-mon[31859]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:44.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:43 smithi005 ceph-mon[28006]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:45 smithi049 ceph-mon[31379]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T13:23:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:45 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:45 smithi107 ceph-mon[31859]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:46.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T13:23:46.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:45 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:45 smithi005 ceph-mon[28006]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:46.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T13:23:46.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:45 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:47.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:46 smithi049 ceph-mon[31379]: Deploying daemon osd.10 on smithi049 2024-04-22T13:23:47.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:46 smithi107 ceph-mon[31859]: Deploying daemon osd.10 on smithi049 2024-04-22T13:23:47.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:46 smithi005 ceph-mon[28006]: Deploying daemon osd.10 on smithi049 2024-04-22T13:23:48.207 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:47 smithi049 ceph-mon[31379]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:48.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:47 smithi107 ceph-mon[31859]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:48.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:47 smithi005 ceph-mon[28006]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:50.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:50 smithi049 ceph-mon[31379]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:50.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:50 smithi107 ceph-mon[31859]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:50.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:50 smithi005 ceph-mon[28006]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:52.168 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:52 smithi049 ceph-mon[31379]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:52.168 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:52.168 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:52.168 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:52.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:52 smithi107 ceph-mon[31859]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:52.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:52.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:52.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:52.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:52 smithi005 ceph-mon[28006]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:52.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:52.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:52.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:54.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:54 smithi049 ceph-mon[31379]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:54.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:54 smithi107 ceph-mon[31859]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:54.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:54 smithi005 ceph-mon[28006]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:55.102 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 10 on host 'smithi049' 2024-04-22T13:23:55.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.104+0000 7f9c8affd700 1 -- 172.21.15.49:0/1082201809 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9c8c062e80 con 0x7f9c740a3cc0 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3cc0 msgr2=0x7f9c740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3cc0 0x7f9c740a6180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c840060b0 tx=0x7f9c8400c040 comp rx=0 tx=0).stop 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 msgr2=0x7f9c8c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c074e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c9b0 tx=0x7f9c8000ccc0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 shutdown_connections 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c104640 0x7f9c8c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c102aa0 0x7f9c8c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3cc0 0x7f9c740a6180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 --2- 172.21.15.49:0/1082201809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c103ca0 0x7f9c8c074e30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.106+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 >> 172.21.15.49:0/1082201809 conn(0x7f9c8c0fe040 msgr2=0x7f9c8c107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.107+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 shutdown_connections 2024-04-22T13:23:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-22T13:23:55.107+0000 7f9c93d1a700 1 -- 172.21.15.49:0/1082201809 wait complete. 2024-04-22T13:23:55.415 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:55.415 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.415 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.415 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:55.415 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:55.416 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.416 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.416 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.416 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:55.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:55.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:55.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:55.646 DEBUG:teuthology.orchestra.run.smithi049:osd.10> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.10.service 2024-04-22T13:23:55.649 INFO:tasks.cephadm:Deploying osd.11 on smithi107 with /dev/vg_nvme/lv_1... 2024-04-22T13:23:55.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T13:23:55.896 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:23:56.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:56 smithi107 ceph-mon[31859]: from='osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:23:56.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:56 smithi107 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:23:56.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:56 smithi005 ceph-mon[28006]: from='osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:23:56.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:56 smithi005 ceph-mon[28006]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:23:56.642 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:56 smithi049 ceph-mon[31379]: from='osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:23:56.642 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:56 smithi049 ceph-mon[31379]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:23:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:57 smithi107 ceph-mon[31859]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:57 smithi107 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T13:23:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:57 smithi107 ceph-mon[31859]: from='osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:23:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:57 smithi107 ceph-mon[31859]: osdmap e74: 11 total, 10 up, 11 in 2024-04-22T13:23:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:57.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:57 smithi107 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:23:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:57 smithi005 ceph-mon[28006]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:57 smithi005 ceph-mon[28006]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T13:23:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:57 smithi005 ceph-mon[28006]: from='osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:23:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:57 smithi005 ceph-mon[28006]: osdmap e74: 11 total, 10 up, 11 in 2024-04-22T13:23:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:57 smithi005 ceph-mon[28006]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:23:57.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:57 smithi049 ceph-mon[31379]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:57.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:57 smithi049 ceph-mon[31379]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T13:23:57.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:57 smithi049 ceph-mon[31379]: from='osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:23:57.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:57 smithi049 ceph-mon[31379]: osdmap e74: 11 total, 10 up, 11 in 2024-04-22T13:23:57.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:57.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:57 smithi049 ceph-mon[31379]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115279 s, 910 MB/s 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2024-04-22T13:23:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T13:23:58.199 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:23:58.106+0000 7f561d972700 -1 osd.10 0 waiting for initial osdmap 2024-04-22T13:23:58.199 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:23:58.122+0000 7f5614f5c700 -1 osd.10 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:23:58.483 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-mon[31379]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:23:58.483 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-mon[31379]: osdmap e75: 11 total, 10 up, 11 in 2024-04-22T13:23:58.483 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:58.483 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:58.483 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:58 smithi049 ceph-mon[31379]: from='osd.10 ' entity='osd.10' 2024-04-22T13:23:58.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:58 smithi107 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:23:58.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:58 smithi107 ceph-mon[31859]: osdmap e75: 11 total, 10 up, 11 in 2024-04-22T13:23:58.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:58 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:58.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:58 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:58.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:58 smithi107 ceph-mon[31859]: from='osd.10 ' entity='osd.10' 2024-04-22T13:23:58.525 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-04-22T13:23:58.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:58 smithi005 ceph-mon[28006]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-04-22T13:23:58.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:58 smithi005 ceph-mon[28006]: osdmap e75: 11 total, 10 up, 11 in 2024-04-22T13:23:58.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:58 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:58.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:58 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:58.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:58 smithi005 ceph-mon[28006]: from='osd.10 ' entity='osd.10' 2024-04-22T13:23:58.764 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:23:59.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:23:59.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] boot 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi049 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi049 to 4272M 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:59.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:23:59 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] boot 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi049 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi049 to 4272M 2024-04-22T13:23:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:23:59 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: osd.10 [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] boot 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi049 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.658 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:23:59.659 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi049 to 4272M 2024-04-22T13:23:59.659 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:23:59.659 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:59.659 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:59.659 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:23:59 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 -- 172.21.15.107:0/4170369761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 msgr2=0x7f8374101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 --2- 172.21.15.107:0/4170369761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374101600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f836400aaa0 tx=0x7f836400adb0 comp rx=0 tx=0).stop 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 -- 172.21.15.107:0/4170369761 shutdown_connections 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 --2- 172.21.15.107:0/4170369761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8374102d80 0x7f8374109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 --2- 172.21.15.107:0/4170369761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374101600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 --2- 172.21.15.107:0/4170369761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83741023e0 0x7f8374102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 -- 172.21.15.107:0/4170369761 >> 172.21.15.107:0/4170369761 conn(0x7f83740fc780 msgr2=0x7f83740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 -- 172.21.15.107:0/4170369761 shutdown_connections 2024-04-22T13:24:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 -- 172.21.15.107:0/4170369761 wait complete. 2024-04-22T13:24:00.343 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:00 smithi107 ceph-mon[31859]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 Processor -- start 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.345+0000 7f837a622700 1 -- start start 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83741023e0 0x7f83741992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8374102d80 0x7f83741a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f837410c460 con 0x7f83741023e0 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f837410c2e0 con 0x7f8374102d80 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f837410c5e0 con 0x7f83741011e0 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34900/0 (socket says 172.21.15.107:34900) 2024-04-22T13:24:00.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 -- 172.21.15.107:0/1858786364 learned_addr learned my addr 172.21.15.107:0/1858786364 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8378bbf700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8374102d80 0x7f83741a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f83737fe700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83741023e0 0x7f83741992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 -- 172.21.15.107:0/1858786364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8374102d80 msgr2=0x7f83741a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8374102d80 0x7f83741a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 -- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83741023e0 msgr2=0x7f83741992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83741023e0 0x7f83741992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 -- 172.21.15.107:0/1858786364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83741a0970 con 0x7f83741011e0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f8373fff700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374198d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f836400cda0 tx=0x7f836400cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364023020 con 0x7f83741011e0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83640040c0 con 0x7f83741011e0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83741a0c00 con 0x7f83741011e0 2024-04-22T13:24:00.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364016680 con 0x7f83741011e0 2024-04-22T13:24:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.346+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83741a1050 con 0x7f83741011e0 2024-04-22T13:24:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.347+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8374069730 con 0x7f83741011e0 2024-04-22T13:24:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.348+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8364026020 con 0x7f83741011e0 2024-04-22T13:24:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.349+0000 7f83717fa700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f835c0a3c40 0x7f835c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.349+0000 7f83737fe700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f835c0a3c40 0x7f835c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.349+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6723+0+0 (secure 0 0 0) 0x7f83640d1610 con 0x7f83741011e0 2024-04-22T13:24:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.350+0000 7f83737fe700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f835c0a3c40 0x7f835c0a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f836800bbd0 tx=0x7f836800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.354+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f836402d020 con 0x7f83741011e0 2024-04-22T13:24:00.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:00.559+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8374109010 con 0x7f835c0a3c40 2024-04-22T13:24:00.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:00 smithi005 ceph-mon[28006]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T13:24:00.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:00 smithi049 ceph-mon[31379]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T13:24:01.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:01 smithi107 ceph-mon[31859]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 644 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:01.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:01 smithi107 ceph-mon[31859]: osdmap e78: 11 total, 11 up, 11 in 2024-04-22T13:24:01.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:01 smithi107 ceph-mon[31859]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:01.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:24:01.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:24:01.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:01.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:01 smithi005 ceph-mon[28006]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 644 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:01 smithi005 ceph-mon[28006]: osdmap e78: 11 total, 11 up, 11 in 2024-04-22T13:24:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:01 smithi005 ceph-mon[28006]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:24:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:24:01.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:01.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:01 smithi049 ceph-mon[31379]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 644 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:01.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:01 smithi049 ceph-mon[31379]: osdmap e78: 11 total, 11 up, 11 in 2024-04-22T13:24:01.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:01 smithi049 ceph-mon[31379]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:01.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:24:01.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:24:01.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:02.415 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:02 smithi107 ceph-mon[31859]: osdmap e79: 11 total, 11 up, 11 in 2024-04-22T13:24:02.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:02 smithi049 ceph-mon[31379]: osdmap e79: 11 total, 11 up, 11 in 2024-04-22T13:24:02.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:02 smithi005 ceph-mon[28006]: osdmap e79: 11 total, 11 up, 11 in 2024-04-22T13:24:03.687 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:03 smithi107 ceph-mon[31859]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 644 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:03.688 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:03 smithi107 ceph-mon[31859]: osdmap e80: 11 total, 11 up, 11 in 2024-04-22T13:24:03.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:03 smithi049 ceph-mon[31379]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 644 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:03.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:03 smithi049 ceph-mon[31379]: osdmap e80: 11 total, 11 up, 11 in 2024-04-22T13:24:03.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:03 smithi005 ceph-mon[28006]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 644 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:03.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:03 smithi005 ceph-mon[28006]: osdmap e80: 11 total, 11 up, 11 in 2024-04-22T13:24:04.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:04 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/262121923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]: dispatch 2024-04-22T13:24:04.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:04 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]: dispatch 2024-04-22T13:24:04.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:04 smithi049 ceph-mon[31379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]': finished 2024-04-22T13:24:04.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:04 smithi049 ceph-mon[31379]: osdmap e81: 12 total, 11 up, 12 in 2024-04-22T13:24:04.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:04 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:04.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:04 smithi049 ceph-mon[31379]: from='client.? 172.21.15.107:0/3661711140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:24:04.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:04 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/262121923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]: dispatch 2024-04-22T13:24:04.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:04 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]: dispatch 2024-04-22T13:24:04.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:04 smithi107 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]': finished 2024-04-22T13:24:04.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:04 smithi107 ceph-mon[31859]: osdmap e81: 12 total, 11 up, 12 in 2024-04-22T13:24:04.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:04 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:04.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:04 smithi107 ceph-mon[31859]: from='client.? 172.21.15.107:0/3661711140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:24:04.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:04 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/262121923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]: dispatch 2024-04-22T13:24:04.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:04 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]: dispatch 2024-04-22T13:24:04.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:04 smithi005 ceph-mon[28006]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c592b75-f30f-4e7f-b38c-5b09e1252184"}]': finished 2024-04-22T13:24:04.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:04 smithi005 ceph-mon[28006]: osdmap e81: 12 total, 11 up, 12 in 2024-04-22T13:24:04.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:04 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:04.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:04 smithi005 ceph-mon[28006]: from='client.? 172.21.15.107:0/3661711140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:24:05.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:05 smithi049 ceph-mon[31379]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:24:05.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:05 smithi107 ceph-mon[31859]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:24:05.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:05 smithi005 ceph-mon[28006]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:24:07.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:07 smithi107 ceph-mon[31859]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-22T13:24:07.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:07 smithi049 ceph-mon[31379]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-22T13:24:07.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:07 smithi005 ceph-mon[28006]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-22T13:24:09.714 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:09 smithi107 ceph-mon[31859]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-22T13:24:09.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:09 smithi049 ceph-mon[31379]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-22T13:24:09.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:09 smithi005 ceph-mon[28006]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-22T13:24:11.724 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:11 smithi107 ceph-mon[31859]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:24:11.724 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:11 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T13:24:11.724 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:11 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:11.724 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:11 smithi107 ceph-mon[31859]: Deploying daemon osd.11 on smithi107 2024-04-22T13:24:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:11 smithi049 ceph-mon[31379]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:24:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:11 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T13:24:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:11 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:11 smithi049 ceph-mon[31379]: Deploying daemon osd.11 on smithi107 2024-04-22T13:24:11.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:11 smithi005 ceph-mon[28006]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:24:11.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:11 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T13:24:11.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:11 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:11.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:11 smithi005 ceph-mon[28006]: Deploying daemon osd.11 on smithi107 2024-04-22T13:24:13.692 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:13 smithi107 ceph-mon[31859]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-22T13:24:13.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:13 smithi049 ceph-mon[31379]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-22T13:24:13.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:13 smithi005 ceph-mon[28006]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-22T13:24:15.708 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:15 smithi107 ceph-mon[31859]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-22T13:24:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:15 smithi049 ceph-mon[31379]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-22T13:24:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:15 smithi005 ceph-mon[28006]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-22T13:24:17.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:17 smithi107 ceph-mon[31859]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:17.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:17.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:17.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:17 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:17.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:17 smithi005 ceph-mon[28006]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:17.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:17.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:17.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:17 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:17 smithi049 ceph-mon[31379]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:17 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:19 smithi107 ceph-mon[31859]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:19.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:19 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:19 smithi005 ceph-mon[28006]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:19.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:19.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:19.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:19 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:19 smithi049 ceph-mon[31379]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:19.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:19.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:19.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:19 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:19.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.978+0000 7f83717fa700 1 -- 172.21.15.107:0/1858786364 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8374109010 con 0x7f835c0a3c40 2024-04-22T13:24:19.976 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 11 on host 'smithi107' 2024-04-22T13:24:19.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f835c0a3c40 msgr2=0x7f835c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f835c0a3c40 0x7f835c0a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f836800bbd0 tx=0x7f836800b5b0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 msgr2=0x7f8374198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374198d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f836400cda0 tx=0x7f836400cdd0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 shutdown_connections 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f835c0a3c40 0x7f835c0a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8374102d80 0x7f83741a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83741011e0 0x7f8374198d70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 --2- 172.21.15.107:0/1858786364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83741023e0 0x7f83741992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 >> 172.21.15.107:0/1858786364 conn(0x7f83740fc780 msgr2=0x7f83741078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 shutdown_connections 2024-04-22T13:24:19.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:19.981+0000 7f837a622700 1 -- 172.21.15.107:0/1858786364 wait complete. 2024-04-22T13:24:20.698 DEBUG:teuthology.orchestra.run.smithi107:osd.11> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.11.service 2024-04-22T13:24:20.700 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-22T13:24:20.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T13:24:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:24:21.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:21.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:21.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:20 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:21.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:20 smithi005 ceph-mon[28006]: from='osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:24:21.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:20 smithi005 ceph-mon[28006]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:24:21.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:21.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:21.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:20 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:21.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:20 smithi107 ceph-mon[31859]: from='osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:24:21.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:20 smithi107 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:24:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:20 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:20 smithi049 ceph-mon[31379]: from='osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:24:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:20 smithi049 ceph-mon[31379]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:24:21.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-mon[31859]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:21.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T13:24:21.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-mon[31859]: from='osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:24:21.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-mon[31859]: osdmap e82: 12 total, 11 up, 12 in 2024-04-22T13:24:21.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:21.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:24:21.986 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:24:21.970+0000 7f0ffa59a700 -1 osd.11 0 waiting for initial osdmap 2024-04-22T13:24:21.986 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:24:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:24:21.988+0000 7f0ff638d700 -1 osd.11 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:24:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:21 smithi049 ceph-mon[31379]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:21 smithi049 ceph-mon[31379]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T13:24:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:21 smithi049 ceph-mon[31379]: from='osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:24:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:21 smithi049 ceph-mon[31379]: osdmap e82: 12 total, 11 up, 12 in 2024-04-22T13:24:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:21 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:21 smithi049 ceph-mon[31379]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:24:22.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:21 smithi005 ceph-mon[28006]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:22.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:21 smithi005 ceph-mon[28006]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T13:24:22.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:21 smithi005 ceph-mon[28006]: from='osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:24:22.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:21 smithi005 ceph-mon[28006]: osdmap e82: 12 total, 11 up, 12 in 2024-04-22T13:24:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:21 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:21 smithi005 ceph-mon[28006]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T13:24:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 -- 172.21.15.5:0/3214231247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 msgr2=0x7f1b1c105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 --2- 172.21.15.5:0/3214231247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c105940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009990 tx=0x7f1b18009ca0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 -- 172.21.15.5:0/3214231247 shutdown_connections 2024-04-22T13:24:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 --2- 172.21.15.5:0/3214231247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b1c1087b0 0x7f1b1c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 --2- 172.21.15.5:0/3214231247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c105940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 --2- 172.21.15.5:0/3214231247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b1c105e80 0x7f1b1c108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 -- 172.21.15.5:0/3214231247 >> 172.21.15.5:0/3214231247 conn(0x7f1b1c0fcf60 msgr2=0x7f1b1c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 -- 172.21.15.5:0/3214231247 shutdown_connections 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.821+0000 7f1b2333e700 1 -- 172.21.15.5:0/3214231247 wait complete. 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 Processor -- start 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 -- start start 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b1c105e80 0x7f1b1c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b1c1087b0 0x7f1b1c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c10e590 con 0x7f1b1c103550 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c10e410 con 0x7f1b1c1087b0 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b2333e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c10e710 con 0x7f1b1c105e80 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b210da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b210da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41970/0 (socket says 172.21.15.5:41970) 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b210da700 1 -- 172.21.15.5:0/2048561648 learned_addr learned my addr 172.21.15.5:0/2048561648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b208d9700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b1c105e80 0x7f1b1c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.822+0000 7f1b218db700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b1c1087b0 0x7f1b1c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b210da700 1 -- 172.21.15.5:0/2048561648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b1c105e80 msgr2=0x7f1b1c199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b210da700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b1c105e80 0x7f1b1c199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b210da700 1 -- 172.21.15.5:0/2048561648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b1c1087b0 msgr2=0x7f1b1c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b210da700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b1c1087b0 0x7f1b1c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b210da700 1 -- 172.21.15.5:0/2048561648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c1a08a0 con 0x7f1b1c103550 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b210da700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c198c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b180036f0 tx=0x7f1b180057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18026070 con 0x7f1b1c103550 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c1a0b30 con 0x7f1b1c103550 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b18004000 con 0x7f1b1c103550 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800fcb0 con 0x7f1b1c103550 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.823+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1a0f80 con 0x7f1b1c103550 2024-04-22T13:24:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.824+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c02df80 con 0x7f1b1c103550 2024-04-22T13:24:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.825+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f1b18029070 con 0x7f1b1c103550 2024-04-22T13:24:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.826+0000 7f1b127fc700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1b080a3c40 0x7f1b080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.826+0000 7f1b208d9700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1b080a3c40 0x7f1b080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.826+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6847+0+0 (secure 0 0 0) 0x7f1b180d0ef0 con 0x7f1b1c103550 2024-04-22T13:24:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.826+0000 7f1b208d9700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1b080a3c40 0x7f1b080a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c005950 tx=0x7f1b0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:22.830+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b18030080 con 0x7f1b1c103550 2024-04-22T13:24:23.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.045+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1b1c02e3a0 con 0x7f1b1c103550 2024-04-22T13:24:23.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.045+0000 7f1b127fc700 1 -- 172.21.15.5:0/2048561648 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7f1b18013070 con 0x7f1b1c103550 2024-04-22T13:24:23.042 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1b080a3c40 msgr2=0x7f1b080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1b080a3c40 0x7f1b080a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c005950 tx=0x7f1b0c00b410 comp rx=0 tx=0).stop 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 msgr2=0x7f1b1c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c198c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b180036f0 tx=0x7f1b180057c0 comp rx=0 tx=0).stop 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 shutdown_connections 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1b080a3c40 0x7f1b080a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b1c1087b0 0x7f1b1c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c103550 0x7f1b1c198c50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 --2- 172.21.15.5:0/2048561648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b1c105e80 0x7f1b1c199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:23.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.047+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 >> 172.21.15.5:0/2048561648 conn(0x7f1b1c0fcf60 msgr2=0x7f1b1c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:23.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.048+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 shutdown_connections 2024-04-22T13:24:23.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:23.048+0000 7f1b2333e700 1 -- 172.21.15.5:0/2048561648 wait complete. 2024-04-22T13:24:23.106 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:22 smithi107 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:24:23.107 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:22 smithi107 ceph-mon[31859]: osdmap e83: 12 total, 11 up, 12 in 2024-04-22T13:24:23.107 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:22 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:23.107 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:22 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:22 smithi049 ceph-mon[31379]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:24:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:22 smithi049 ceph-mon[31379]: osdmap e83: 12 total, 11 up, 12 in 2024-04-22T13:24:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:22 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:22 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:22 smithi005 ceph-mon[28006]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T13:24:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:22 smithi005 ceph-mon[28006]: osdmap e83: 12 total, 11 up, 12 in 2024-04-22T13:24:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:22 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:23.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:22 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:23.909 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1713792262,"num_in_osds":12,"osd_in_since":1713792243,"num_remapped_pgs":0} 2024-04-22T13:24:23.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T13:24:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:24:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: purged_snaps scrub starts 2024-04-22T13:24:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: purged_snaps scrub ok 2024-04-22T13:24:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] boot 2024-04-22T13:24:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T13:24:24.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2048561648' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:24.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:23 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: purged_snaps scrub starts 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: purged_snaps scrub ok 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] boot 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2048561648' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:23 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: purged_snaps scrub starts 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: purged_snaps scrub ok 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: osd.11 [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] boot 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:24.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2048561648' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:24.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:23 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:25.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:24 smithi049 ceph-mon[31379]: Detected new or changed devices on smithi107 2024-04-22T13:24:25.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:24 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi107 to 4272M 2024-04-22T13:24:25.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:24 smithi049 ceph-mon[31379]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T13:24:25.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:24 smithi107 ceph-mon[31859]: Detected new or changed devices on smithi107 2024-04-22T13:24:25.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:24 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi107 to 4272M 2024-04-22T13:24:25.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:24 smithi107 ceph-mon[31859]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T13:24:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:24 smithi005 ceph-mon[28006]: Detected new or changed devices on smithi107 2024-04-22T13:24:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:24 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi107 to 4272M 2024-04-22T13:24:25.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:24 smithi005 ceph-mon[28006]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 -- 172.21.15.5:0/1350543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406b1f0 msgr2=0x7f849406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 --2- 172.21.15.5:0/1350543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406b1f0 0x7f849406b670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8484009990 tx=0x7f8484009ca0 comp rx=0 tx=0).stop 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 -- 172.21.15.5:0/1350543646 shutdown_connections 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 --2- 172.21.15.5:0/1350543646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849406bbb0 0x7f849410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 --2- 172.21.15.5:0/1350543646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8494108a60 0x7f8494108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 --2- 172.21.15.5:0/1350543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406b1f0 0x7f849406b670 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.934+0000 7f849bf58700 1 -- 172.21.15.5:0/1350543646 >> 172.21.15.5:0/1350543646 conn(0x7f84940789e0 msgr2=0x7f8494078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 -- 172.21.15.5:0/1350543646 shutdown_connections 2024-04-22T13:24:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 -- 172.21.15.5:0/1350543646 wait complete. 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 Processor -- start 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 -- start start 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849406b1f0 0x7f8494199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 0x7f84941996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8494108a60 0x7f849419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8494112c60 con 0x7f849406bbb0 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8494112ae0 con 0x7f849406b1f0 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.935+0000 7f849bf58700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8494112de0 con 0x7f8494108a60 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 0x7f84941996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 0x7f84941996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41988/0 (socket says 172.21.15.5:41988) 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 -- 172.21.15.5:0/3092816832 learned_addr learned my addr 172.21.15.5:0/3092816832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f8499cf4700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849406b1f0 0x7f8494199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 -- 172.21.15.5:0/3092816832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8494108a60 msgr2=0x7f849419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8494108a60 0x7f849419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 -- 172.21.15.5:0/3092816832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849406b1f0 msgr2=0x7f8494199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849406b1f0 0x7f8494199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 -- 172.21.15.5:0/3092816832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849419e1a0 con 0x7f849406bbb0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f84994f3700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 0x7f84941996d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f848400b2e0 tx=0x7f8484004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484026070 con 0x7f849406bbb0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849419e430 con 0x7f849406bbb0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8484021410 con 0x7f849406bbb0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484017650 con 0x7f849406bbb0 2024-04-22T13:24:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.936+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941a5d10 con 0x7f849406bbb0 2024-04-22T13:24:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.937+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8494069730 con 0x7f849406bbb0 2024-04-22T13:24:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.938+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8484029070 con 0x7f849406bbb0 2024-04-22T13:24:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.939+0000 7f848affd700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f84800a3cc0 0x7f84800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.939+0000 7f8499cf4700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f84800a3cc0 0x7f84800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.939+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f84840d0db0 con 0x7f849406bbb0 2024-04-22T13:24:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.939+0000 7f8499cf4700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f84800a3cc0 0x7f84800a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84900045b0 tx=0x7f849000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:25.944+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f848405dff0 con 0x7f849406bbb0 2024-04-22T13:24:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.155+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f849402df80 con 0x7f849406bbb0 2024-04-22T13:24:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.156+0000 7f848affd700 1 -- 172.21.15.5:0/3092816832 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18352 (secure 0 0 0) 0x7f8484013070 con 0x7f849406bbb0 2024-04-22T13:24:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:24:23.973672+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f84800a3cc0 msgr2=0x7f84800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f84800a3cc0 0x7f84800a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84900045b0 tx=0x7f849000b3c0 comp rx=0 tx=0).stop 2024-04-22T13:24:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 msgr2=0x7f84941996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 0x7f84941996d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f848400b2e0 tx=0x7f8484004790 comp rx=0 tx=0).stop 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 shutdown_connections 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f84800a3cc0 0x7f84800a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849406b1f0 0x7f8494199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8494108a60 0x7f849419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 --2- 172.21.15.5:0/3092816832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f849406bbb0 0x7f84941996d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 >> 172.21.15.5:0/3092816832 conn(0x7f84940789e0 msgr2=0x7f84940ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 shutdown_connections 2024-04-22T13:24:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:26.159+0000 7f849bf58700 1 -- 172.21.15.5:0/3092816832 wait complete. 2024-04-22T13:24:26.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:25 smithi049 ceph-mon[31379]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:26.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:25 smithi107 ceph-mon[31859]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:26.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:25 smithi005 ceph-mon[28006]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:26.910 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:26.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-22T13:24:27.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:24:27.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:26 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3092816832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:27.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:26 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3092816832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:27.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:26 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3092816832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:28.270 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:27 smithi005 ceph-mon[28006]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:28.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:27 smithi049 ceph-mon[31379]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:28.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:27 smithi107 ceph-mon[31859]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 -- 172.21.15.5:0/2398769968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 msgr2=0x7f18c0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2398769968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0107d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009990 tx=0x7f18bc009ca0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 -- 172.21.15.5:0/2398769968 shutdown_connections 2024-04-22T13:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2398769968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c0102300 0x7f18c010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2398769968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0107d70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2398769968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c0101940 0x7f18c0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 -- 172.21.15.5:0/2398769968 >> 172.21.15.5:0/2398769968 conn(0x7f18c00788f0 msgr2=0x7f18c0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.904+0000 7f18c84ec700 1 -- 172.21.15.5:0/2398769968 shutdown_connections 2024-04-22T13:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 -- 172.21.15.5:0/2398769968 wait complete. 2024-04-22T13:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 Processor -- start 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 -- start start 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c0101940 0x7f18c0197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c0102300 0x7f18c0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111b00 con 0x7f18c0107990 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111980 con 0x7f18c0102300 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.905+0000 7f18c84ec700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111c80 con 0x7f18c0101940 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46034/0 (socket says 172.21.15.5:46034) 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 -- 172.21.15.5:0/2214482780 learned_addr learned my addr 172.21.15.5:0/2214482780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c5a87700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c0102300 0x7f18c0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6288700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c0101940 0x7f18c0197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 -- 172.21.15.5:0/2214482780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c0101940 msgr2=0x7f18c0197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c0101940 0x7f18c0197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 -- 172.21.15.5:0/2214482780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c0102300 msgr2=0x7f18c0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c0102300 0x7f18c0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 -- 172.21.15.5:0/2214482780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c01a7580 con 0x7f18c0107990 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c6a89700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0198790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18b800ef30 tx=0x7f18b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80092e0 con 0x7f18c0107990 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b8004bb0 con 0x7f18c0107990 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c01a7810 con 0x7f18c0107990 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8004280 con 0x7f18c0107990 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.906+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01a7d50 con 0x7f18c0107990 2024-04-22T13:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.907+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c0109eb0 con 0x7f18c0107990 2024-04-22T13:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.908+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f18b801b030 con 0x7f18c0107990 2024-04-22T13:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.909+0000 7f18b77fe700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f18ac0a3cc0 0x7f18ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.909+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f18b80d0350 con 0x7f18c0107990 2024-04-22T13:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.909+0000 7f18c6288700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f18ac0a3cc0 0x7f18ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.910+0000 7f18c6288700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f18ac0a3cc0 0x7f18ac0a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f18bc006010 tx=0x7f18bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:28.913+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18b8022020 con 0x7f18c0107990 2024-04-22T13:24:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.118+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f18c0109890 con 0x7f18c0107990 2024-04-22T13:24:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.119+0000 7f18b77fe700 1 -- 172.21.15.5:0/2214482780 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f18b8010040 con 0x7f18c0107990 2024-04-22T13:24:29.115 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f18ac0a3cc0 msgr2=0x7f18ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f18ac0a3cc0 0x7f18ac0a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f18bc006010 tx=0x7f18bc01a040 comp rx=0 tx=0).stop 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 msgr2=0x7f18c0198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0198790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18b800ef30 tx=0x7f18b800c3a0 comp rx=0 tx=0).stop 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 shutdown_connections 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18c0107990 0x7f18c0198790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c0101940 0x7f18c0197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f18ac0a3cc0 0x7f18ac0a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 --2- 172.21.15.5:0/2214482780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c0102300 0x7f18c0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 >> 172.21.15.5:0/2214482780 conn(0x7f18c00788f0 msgr2=0x7f18c01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 shutdown_connections 2024-04-22T13:24:29.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:29.121+0000 7f18c84ec700 1 -- 172.21.15.5:0/2214482780 wait complete. 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.b on smithi049 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.e on smithi049 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.h on smithi049 2024-04-22T13:24:29.825 INFO:tasks.cephadm:Adding mds.k on smithi049 2024-04-22T13:24:29.826 INFO:tasks.cephadm:Adding mds.c on smithi107 2024-04-22T13:24:29.826 INFO:tasks.cephadm:Adding mds.f on smithi107 2024-04-22T13:24:29.826 INFO:tasks.cephadm:Adding mds.i on smithi107 2024-04-22T13:24:29.826 INFO:tasks.cephadm:Adding mds.l on smithi107 2024-04-22T13:24:29.826 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l' 2024-04-22T13:24:30.075 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:24:30.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:29 smithi107 ceph-mon[31859]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:30.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:29 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2214482780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:30.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:29 smithi049 ceph-mon[31379]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:30.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:29 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2214482780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:30.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:29 smithi005 ceph-mon[28006]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:30.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:29 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2214482780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:31.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 -- 172.21.15.107:0/2285509766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000103ca0 msgr2=0x7fc000104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:31.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 --2- 172.21.15.107:0/2285509766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000103ca0 0x7fc000104100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbff40098f0 tx=0x7fbff4009c00 comp rx=0 tx=0).stop 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 -- 172.21.15.107:0/2285509766 shutdown_connections 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 --2- 172.21.15.107:0/2285509766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc000102aa0 0x7fc000102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 --2- 172.21.15.107:0/2285509766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000104640 0x7fc00010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 --2- 172.21.15.107:0/2285509766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000103ca0 0x7fc000104100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 -- 172.21.15.107:0/2285509766 >> 172.21.15.107:0/2285509766 conn(0x7fc0000fe040 msgr2=0x7fc000100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 -- 172.21.15.107:0/2285509766 shutdown_connections 2024-04-22T13:24:31.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.598+0000 7fc00516e700 1 -- 172.21.15.107:0/2285509766 wait complete. 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.599+0000 7fc00516e700 1 Processor -- start 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.599+0000 7fc00516e700 1 -- start start 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.599+0000 7fc00516e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 0x7fc000198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000103ca0 0x7fc0001991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc000104640 0x7fc0001a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010e530 con 0x7fc000103ca0 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010e3b0 con 0x7fc000104640 2024-04-22T13:24:31.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010e6b0 con 0x7fc000102aa0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 0x7fc000198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 0x7fc000198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42502/0 (socket says 172.21.15.107:42502) 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 -- 172.21.15.107:0/1207469452 learned_addr learned my addr 172.21.15.107:0/1207469452 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffe59c700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000103ca0 0x7fc0001991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 -- 172.21.15.107:0/1207469452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc000104640 msgr2=0x7fc0001a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbfff59e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc000104640 0x7fc0001a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc000104640 0x7fc0001a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 -- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000103ca0 msgr2=0x7fc0001991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000103ca0 0x7fc0001991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 -- 172.21.15.107:0/1207469452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0001a0900 con 0x7fc000102aa0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffe59c700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000103ca0 0x7fc0001991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbffed9d700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 0x7fc000198c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e9f0 tx=0x7fbff000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004230 con 0x7fc000102aa0 2024-04-22T13:24:31.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0001a0bf0 con 0x7fc000102aa0 2024-04-22T13:24:31.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff00043d0 con 0x7fc000102aa0 2024-04-22T13:24:31.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003cf0 con 0x7fc000102aa0 2024-04-22T13:24:31.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.600+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001a1100 con 0x7fc000102aa0 2024-04-22T13:24:31.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.602+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc000069730 con 0x7fc000102aa0 2024-04-22T13:24:31.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.602+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fbff000c7c0 con 0x7fc000102aa0 2024-04-22T13:24:31.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.603+0000 7fbfe7fff700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fbfe80a3c40 0x7fbfe80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:31.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.603+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fbff00d0380 con 0x7fc000102aa0 2024-04-22T13:24:31.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.603+0000 7fbffe59c700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fbfe80a3c40 0x7fbfe80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:31.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.604+0000 7fbffe59c700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fbfe80a3c40 0x7fbfe80a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbff400c010 tx=0x7fbff4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:31.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.607+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbff009d200 con 0x7fc000102aa0 2024-04-22T13:24:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.820+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l", "target": ["mon-mgr", ""]}) -- 0x7fc000063080 con 0x7fbfe80a3c40 2024-04-22T13:24:31.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.829+0000 7fbfe7fff700 1 -- 172.21.15.107:0/1207469452 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc000063080 con 0x7fbfe80a3c40 2024-04-22T13:24:31.826 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled mds.all update... 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fbfe80a3c40 msgr2=0x7fbfe80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fbfe80a3c40 0x7fbfe80a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbff400c010 tx=0x7fbff4019040 comp rx=0 tx=0).stop 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 msgr2=0x7fc000198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 0x7fc000198c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e9f0 tx=0x7fbff000ed00 comp rx=0 tx=0).stop 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 shutdown_connections 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fbfe80a3c40 0x7fbfe80a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc000104640 0x7fc0001a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc000102aa0 0x7fc000198c80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 --2- 172.21.15.107:0/1207469452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000103ca0 0x7fc0001991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 >> 172.21.15.107:0/1207469452 conn(0x7fc0000fe040 msgr2=0x7fc0001070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 shutdown_connections 2024-04-22T13:24:31.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T13:24:31.832+0000 7fc00516e700 1 -- 172.21.15.107:0/1207469452 wait complete. 2024-04-22T13:24:32.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:31 smithi107 ceph-mon[31859]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:32.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:32.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:31 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:32.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:31 smithi049 ceph-mon[31379]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:32.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:32.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:31 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:31 smithi005 ceph-mon[28006]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:32.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:31 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:32.348 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.a.service 2024-04-22T13:24:32.352 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.d.service 2024-04-22T13:24:32.395 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.g.service 2024-04-22T13:24:32.438 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.j.service 2024-04-22T13:24:32.482 DEBUG:teuthology.orchestra.run.smithi049:mds.b> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.b.service 2024-04-22T13:24:32.484 DEBUG:teuthology.orchestra.run.smithi049:mds.e> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.e.service 2024-04-22T13:24:32.527 DEBUG:teuthology.orchestra.run.smithi049:mds.h> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.h.service 2024-04-22T13:24:32.570 DEBUG:teuthology.orchestra.run.smithi049:mds.k> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.k.service 2024-04-22T13:24:32.613 DEBUG:teuthology.orchestra.run.smithi107:mds.c> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.c.service 2024-04-22T13:24:32.615 DEBUG:teuthology.orchestra.run.smithi107:mds.f> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service 2024-04-22T13:24:32.658 DEBUG:teuthology.orchestra.run.smithi107:mds.i> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.i.service 2024-04-22T13:24:32.702 DEBUG:teuthology.orchestra.run.smithi107:mds.l> sudo journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.l.service 2024-04-22T13:24:32.745 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-22T13:24:32.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 -- 172.21.15.5:0/2034943273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 msgr2=0x7f0704102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 --2- 172.21.15.5:0/2034943273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704102450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0700009900 tx=0x7f070002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 -- 172.21.15.5:0/2034943273 shutdown_connections 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 --2- 172.21.15.5:0/2034943273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704102450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 --2- 172.21.15.5:0/2034943273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0704102990 0x7f07041092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 --2- 172.21.15.5:0/2034943273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0704100e80 0x7f07041012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 -- 172.21.15.5:0/2034943273 >> 172.21.15.5:0/2034943273 conn(0x7f07040fc3f0 msgr2=0x7f07040fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 -- 172.21.15.5:0/2034943273 shutdown_connections 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 -- 172.21.15.5:0/2034943273 wait complete. 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.864+0000 7f070bfd5640 1 Processor -- start 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 -- start start 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0704100e80 0x7f07041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0704102990 0x7f070419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f070410c9d0 con 0x7f0704101fd0 2024-04-22T13:24:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f070410c850 con 0x7f0704100e80 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070bfd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f070410cb50 con 0x7f0704102990 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46044/0 (socket says 172.21.15.5:46044) 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 -- 172.21.15.5:0/283729663 learned_addr learned my addr 172.21.15.5:0/283729663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709d71640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0704100e80 0x7f07041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f070a572640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0704102990 0x7f070419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 -- 172.21.15.5:0/283729663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0704102990 msgr2=0x7f070419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0704102990 0x7f070419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 -- 172.21.15.5:0/283729663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0704100e80 msgr2=0x7f07041981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0704100e80 0x7f07041981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 -- 172.21.15.5:0/283729663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070419fdc0 con 0x7f0704101fd0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.865+0000 7f0709570640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704198730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0700000c00 tx=0x7f0700004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.866+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700043020 con 0x7f0704101fd0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.866+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07041a0050 con 0x7f0704101fd0 2024-04-22T13:24:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.866+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0700004440 con 0x7f0704101fd0 2024-04-22T13:24:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.866+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700003c90 con 0x7f0704101fd0 2024-04-22T13:24:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.866+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041a05a0 con 0x7f0704101fd0 2024-04-22T13:24:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.867+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07041a07a0 con 0x7f0704101fd0 2024-04-22T13:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.868+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0700046020 con 0x7f0704101fd0 2024-04-22T13:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.868+0000 7f06faffd640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06f00a3bb0 0x7f06f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.869+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f07000f2550 con 0x7f0704101fd0 2024-04-22T13:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.869+0000 7f0709d71640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06f00a3bb0 0x7f06f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.869+0000 7f0709d71640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06f00a3bb0 0x7f06f00a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06f400bbe0 tx=0x7f06f400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:32.872+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f070004d020 con 0x7f0704101fd0 2024-04-22T13:24:33.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l;count:12 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:33.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:33 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.063+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07041a0e90 con 0x7f0704101fd0 2024-04-22T13:24:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.064+0000 7f06faffd640 1 -- 172.21.15.5:0/283729663 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18352 (secure 0 0 0) 0x7f07000bf3d0 con 0x7f0704101fd0 2024-04-22T13:24:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:33.061 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:24:23.973672+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:33.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06f00a3bb0 msgr2=0x7f06f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06f00a3bb0 0x7f06f00a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06f400bbe0 tx=0x7f06f400b390 comp rx=0 tx=0).stop 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 msgr2=0x7f0704198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704198730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0700000c00 tx=0x7f0700004040 comp rx=0 tx=0).stop 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 shutdown_connections 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06f00a3bb0 0x7f06f00a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0704101fd0 0x7f0704198730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0704102990 0x7f070419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 --2- 172.21.15.5:0/283729663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0704100e80 0x7f07041981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 >> 172.21.15.5:0/283729663 conn(0x7f07040fc3f0 msgr2=0x7f0704069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 shutdown_connections 2024-04-22T13:24:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.066+0000 7f070bfd5640 1 -- 172.21.15.5:0/283729663 wait complete. 2024-04-22T13:24:33.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:33.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:33.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 -- 172.21.15.5:0/3137345326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334105340 msgr2=0x7fa334107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 --2- 172.21.15.5:0/3137345326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334105340 0x7fa334107730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009900 tx=0x7fa31c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:33.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 -- 172.21.15.5:0/3137345326 shutdown_connections 2024-04-22T13:24:33.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 --2- 172.21.15.5:0/3137345326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334105340 0x7fa334107730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 --2- 172.21.15.5:0/3137345326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334107c70 0x7fa33410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 --2- 172.21.15.5:0/3137345326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa334102a10 0x7fa334104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 -- 172.21.15.5:0/3137345326 >> 172.21.15.5:0/3137345326 conn(0x7fa3340fc420 msgr2=0x7fa3340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 -- 172.21.15.5:0/3137345326 shutdown_connections 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.182+0000 7fa339775640 1 -- 172.21.15.5:0/3137345326 wait complete. 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 Processor -- start 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 -- start start 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334102a10 0x7fa334198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa334105340 0x7fa334198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 0x7fa33419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410daa0 con 0x7fa334107c70 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410d920 con 0x7fa334102a10 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa339775640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410dc20 con 0x7fa334105340 2024-04-22T13:24:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa3337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 0x7fa33419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa3337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 0x7fa33419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46060/0 (socket says 172.21.15.5:46060) 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.183+0000 7fa3337fe640 1 -- 172.21.15.5:0/492732548 learned_addr learned my addr 172.21.15.5:0/492732548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa332ffd640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334102a10 0x7fa334198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3327fc640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa334105340 0x7fa334198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3337fe640 1 -- 172.21.15.5:0/492732548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa334105340 msgr2=0x7fa334198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3337fe640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa334105340 0x7fa334198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3337fe640 1 -- 172.21.15.5:0/492732548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334102a10 msgr2=0x7fa334198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3337fe640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334102a10 0x7fa334198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3337fe640 1 -- 172.21.15.5:0/492732548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33419fe40 con 0x7fa334107c70 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa332ffd640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334102a10 0x7fa334198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa3337fe640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 0x7fa33419f7d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00d750 tx=0x7fa32c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c017070 con 0x7fa334107c70 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32c0054d0 con 0x7fa334107c70 2024-04-22T13:24:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3341a0130 con 0x7fa334107c70 2024-04-22T13:24:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c004c60 con 0x7fa334107c70 2024-04-22T13:24:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.184+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a0600 con 0x7fa334107c70 2024-04-22T13:24:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.185+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3340fdf10 con 0x7fa334107c70 2024-04-22T13:24:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.186+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fa32c009bd0 con 0x7fa334107c70 2024-04-22T13:24:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.187+0000 7fa32bfff640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa3200a3bb0 0x7fa3200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.187+0000 7fa332ffd640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa3200a3bb0 0x7fa3200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.187+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fa32c0cfa40 con 0x7fa334107c70 2024-04-22T13:24:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.188+0000 7fa332ffd640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa3200a3bb0 0x7fa3200a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa324005c40 tx=0x7fa32400a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.192+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa32c09c820 con 0x7fa334107c70 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l;count:12 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:33.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:33 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l;count:12 2024-04-22T13:24:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:33.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:33 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:33.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.381+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa334062b50 con 0x7fa334107c70 2024-04-22T13:24:33.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.381+0000 7fa32bfff640 1 -- 172.21.15.5:0/492732548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fa32c09c120 con 0x7fa334107c70 2024-04-22T13:24:33.378 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:24:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.383+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa3200a3bb0 msgr2=0x7fa3200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.383+0000 7fa339775640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa3200a3bb0 0x7fa3200a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa324005c40 tx=0x7fa32400a430 comp rx=0 tx=0).stop 2024-04-22T13:24:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.383+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 msgr2=0x7fa33419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.383+0000 7fa339775640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 0x7fa33419f7d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00d750 tx=0x7fa32c00dc10 comp rx=0 tx=0).stop 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 shutdown_connections 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa3200a3bb0 0x7fa3200a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa334107c70 0x7fa33419f7d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa334105340 0x7fa334198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 --2- 172.21.15.5:0/492732548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa334102a10 0x7fa334198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 >> 172.21.15.5:0/492732548 conn(0x7fa3340fc420 msgr2=0x7fa3341048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 shutdown_connections 2024-04-22T13:24:33.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.384+0000 7fa339775640 1 -- 172.21.15.5:0/492732548 wait complete. 2024-04-22T13:24:33.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 -- 172.21.15.5:0/91167316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c101ff0 msgr2=0x7f6e4c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 --2- 172.21.15.5:0/91167316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c101ff0 0x7f6e4c102470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009900 tx=0x7f6e4002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 -- 172.21.15.5:0/91167316 shutdown_connections 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 --2- 172.21.15.5:0/91167316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c101ff0 0x7f6e4c102470 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 --2- 172.21.15.5:0/91167316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e4c1029b0 0x7f6e4c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 --2- 172.21.15.5:0/91167316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 -- 172.21.15.5:0/91167316 >> 172.21.15.5:0/91167316 conn(0x7f6e4c0fc410 msgr2=0x7f6e4c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 -- 172.21.15.5:0/91167316 shutdown_connections 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 -- 172.21.15.5:0/91167316 wait complete. 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.497+0000 7f6e51db2640 1 Processor -- start 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 -- start start 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e4c101ff0 0x7f6e4c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e4c1029b0 0x7f6e4c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10ca40 con 0x7f6e4c100ea0 2024-04-22T13:24:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10c8c0 con 0x7f6e4c101ff0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e51db2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10cbc0 con 0x7f6e4c1029b0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46082/0 (socket says 172.21.15.5:46082) 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 -- 172.21.15.5:0/3487047853 learned_addr learned my addr 172.21.15.5:0/3487047853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4bfff640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e4c1029b0 0x7f6e4c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 -- 172.21.15.5:0/3487047853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e4c1029b0 msgr2=0x7f6e4c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e4c1029b0 0x7f6e4c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 -- 172.21.15.5:0/3487047853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e4c101ff0 msgr2=0x7f6e4c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4affd640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e4c101ff0 0x7f6e4c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e4c101ff0 0x7f6e4c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 -- 172.21.15.5:0/3487047853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c19fdb0 con 0x7f6e4c100ea0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e4b7fe640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c198190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00d950 tx=0x7f6e3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.498+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c017030 con 0x7f6e4c100ea0 2024-04-22T13:24:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.499+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e3c004020 con 0x7f6e4c100ea0 2024-04-22T13:24:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.499+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c1a00a0 con 0x7f6e4c100ea0 2024-04-22T13:24:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.499+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0047a0 con 0x7f6e4c100ea0 2024-04-22T13:24:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.499+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c1a0640 con 0x7f6e4c100ea0 2024-04-22T13:24:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.499+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4c1a0900 con 0x7f6e4c100ea0 2024-04-22T13:24:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.501+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f6e3c009a30 con 0x7f6e4c100ea0 2024-04-22T13:24:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.501+0000 7f6e48ff9640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e340a3c30 0x7f6e340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.502+0000 7f6e4affd640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e340a3c30 0x7f6e340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.502+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f6e3c0d0010 con 0x7f6e4c100ea0 2024-04-22T13:24:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.502+0000 7f6e4affd640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e340a3c30 0x7f6e340a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036040 tx=0x7f6e40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.505+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e3c09cf10 con 0x7f6e4c100ea0 2024-04-22T13:24:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.733+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e4c1a0ad0 con 0x7f6e4c100ea0 2024-04-22T13:24:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.734+0000 7f6e48ff9640 1 -- 172.21.15.5:0/3487047853 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f6e3c09c810 con 0x7f6e4c100ea0 2024-04-22T13:24:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-22T13:24:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-04-22T13:24:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e340a3c30 msgr2=0x7f6e340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e340a3c30 0x7f6e340a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036040 tx=0x7f6e40005fd0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 msgr2=0x7f6e4c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c198190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00d950 tx=0x7f6e3c00de10 comp rx=0 tx=0).stop 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 shutdown_connections 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e340a3c30 0x7f6e340a60f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e4c1029b0 0x7f6e4c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e4c101ff0 0x7f6e4c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 --2- 172.21.15.5:0/3487047853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e4c100ea0 0x7f6e4c198190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 >> 172.21.15.5:0/3487047853 conn(0x7f6e4c0fc410 msgr2=0x7f6e4c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 shutdown_connections 2024-04-22T13:24:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:33.736+0000 7f6e51db2640 1 -- 172.21.15.5:0/3487047853 wait complete. 2024-04-22T13:24:34.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:34 smithi049 ceph-mon[31379]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:34.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:34 smithi049 ceph-mon[31379]: Deploying daemon mds.l on smithi107 2024-04-22T13:24:34.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/283729663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:34.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/492732548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:34.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3487047853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:34.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:34 smithi107 ceph-mon[31859]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:34.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:34 smithi107 ceph-mon[31859]: Deploying daemon mds.l on smithi107 2024-04-22T13:24:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/283729663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/492732548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:34.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3487047853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:34.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:34 smithi005 ceph-mon[28006]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:34.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:34 smithi005 ceph-mon[28006]: Deploying daemon mds.l on smithi107 2024-04-22T13:24:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/283729663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/492732548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:34.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3487047853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:34.785 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:34 smithi107 systemd[1]: Starting Ceph mds.l for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:34 smithi107 podman[62638]: 2024-04-22 13:24:34.794165377 +0000 UTC m=+0.102511245 container create dfc486f84b81a22bb00aa920a851c8e5c162ed35314a769e927f526219871aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:34 smithi107 podman[62638]: 2024-04-22 13:24:34.710230262 +0000 UTC m=+0.018576116 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:34 smithi107 podman[62638]: 2024-04-22 13:24:34.988040687 +0000 UTC m=+0.296386516 container init dfc486f84b81a22bb00aa920a851c8e5c162ed35314a769e927f526219871aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:34 smithi107 podman[62638]: 2024-04-22 13:24:34.990717049 +0000 UTC m=+0.299062877 container start dfc486f84b81a22bb00aa920a851c8e5c162ed35314a769e927f526219871aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:35 smithi107 ceph-mds[62657]: -- 172.21.15.107:0/2232955103 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af39d15a40 con 0x55af39ce8c00 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: starting mds.l at 2024-04-22T13:24:35.076 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:35 smithi107 bash[62638]: dfc486f84b81a22bb00aa920a851c8e5c162ed35314a769e927f526219871aa5 2024-04-22T13:24:35.534 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:24:35 smithi107 systemd[1]: Started Ceph mds.l for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:35.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 -- 172.21.15.5:0/620770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250102a60 msgr2=0x7f7250104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 --2- 172.21.15.5:0/620770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250102a60 0x7f7250104e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7240009900 tx=0x7f724002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 -- 172.21.15.5:0/620770968 shutdown_connections 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 --2- 172.21.15.5:0/620770968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250107cc0 0x7f725010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 --2- 172.21.15.5:0/620770968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7250105390 0x7f7250107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 --2- 172.21.15.5:0/620770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250102a60 0x7f7250104e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 -- 172.21.15.5:0/620770968 >> 172.21.15.5:0/620770968 conn(0x7f72500fc3f0 msgr2=0x7f72500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 -- 172.21.15.5:0/620770968 shutdown_connections 2024-04-22T13:24:35.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.863+0000 7f725852b640 1 -- 172.21.15.5:0/620770968 wait complete. 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 Processor -- start 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 -- start start 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7250102a60 0x7f72501981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250105390 0x7f7250198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 0x7f725019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f725010dac0 con 0x7f7250107cc0 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f725010d940 con 0x7f7250105390 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f725852b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f725010dc40 con 0x7f7250102a60 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 0x7f725019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 0x7f725019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46092/0 (socket says 172.21.15.5:46092) 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 -- 172.21.15.5:0/3139821826 learned_addr learned my addr 172.21.15.5:0/3139821826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f72562c7640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7250102a60 0x7f72501981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7255ac6640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250105390 0x7f7250198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:35.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 -- 172.21.15.5:0/3139821826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7250102a60 msgr2=0x7f72501981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7250102a60 0x7f72501981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 -- 172.21.15.5:0/3139821826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250105390 msgr2=0x7f7250198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250105390 0x7f7250198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7256ac8640 1 -- 172.21.15.5:0/3139821826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725019fe10 con 0x7f7250107cc0 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.864+0000 7f7255ac6640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250105390 0x7f7250198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.865+0000 7f7256ac8640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 0x7f725019f7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f724800efd0 tx=0x7f724800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.865+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248003b70 con 0x7f7250107cc0 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.865+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72501a0100 con 0x7f7250107cc0 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.865+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7248003d10 con 0x7f7250107cc0 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.865+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72480108a0 con 0x7f7250107cc0 2024-04-22T13:24:35.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.865+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72501a0650 con 0x7f7250107cc0 2024-04-22T13:24:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.866+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72500fdee0 con 0x7f7250107cc0 2024-04-22T13:24:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.867+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f7248004b40 con 0x7f7250107cc0 2024-04-22T13:24:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.868+0000 7f72477fe640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f723c0a3bb0 0x7f723c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.868+0000 7f72562c7640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f723c0a3bb0 0x7f723c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.868+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f72480d0950 con 0x7f7250107cc0 2024-04-22T13:24:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.868+0000 7f72562c7640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f723c0a3bb0 0x7f723c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7240000c00 tx=0x7f7240005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:35.871+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f724809d730 con 0x7f7250107cc0 2024-04-22T13:24:36.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6832/1584198825,v1:172.21.15.107:6833/1584198825] up:boot 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: fsmap 1 up:standby 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:36 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:36.067 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:36.067 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6832/1584198825,v1:172.21.15.107:6833/1584198825] up:boot 2024-04-22T13:24:36.067 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: fsmap 1 up:standby 2024-04-22T13:24:36.067 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:24:36.068 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.068 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.068 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.068 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:36.068 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:36.068 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:36 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.099+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7250062b50 con 0x7f7250107cc0 2024-04-22T13:24:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.099+0000 7f72477fe640 1 -- 172.21.15.5:0/3139821826 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f724809d030 con 0x7f7250107cc0 2024-04-22T13:24:36.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:36.096 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T13:24:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f723c0a3bb0 msgr2=0x7f723c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f723c0a3bb0 0x7f723c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7240000c00 tx=0x7f7240005be0 comp rx=0 tx=0).stop 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 msgr2=0x7f725019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 0x7f725019f7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f724800efd0 tx=0x7f724800c6a0 comp rx=0 tx=0).stop 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 shutdown_connections 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f723c0a3bb0 0x7f723c0a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250107cc0 0x7f725019f7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7250105390 0x7f7250198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 --2- 172.21.15.5:0/3139821826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7250102a60 0x7f72501981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 >> 172.21.15.5:0/3139821826 conn(0x7f72500fc3f0 msgr2=0x7f7250103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 shutdown_connections 2024-04-22T13:24:36.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:36.102+0000 7f725852b640 1 -- 172.21.15.5:0/3139821826 wait complete. 2024-04-22T13:24:36.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6832/1584198825,v1:172.21.15.107:6833/1584198825] up:boot 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: fsmap 1 up:standby 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:36.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:36 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:37.216 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:37 smithi049 ceph-mon[31379]: Deploying daemon mds.k on smithi049 2024-04-22T13:24:37.216 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:37 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3139821826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:37 smithi107 ceph-mon[31859]: Deploying daemon mds.k on smithi049 2024-04-22T13:24:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:37 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3139821826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:37 smithi005 ceph-mon[28006]: Deploying daemon mds.k on smithi049 2024-04-22T13:24:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:37 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3139821826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:37.521 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 systemd[1]: Starting Ceph mds.k for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:37.877 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 podman[60378]: 2024-04-22 13:24:37.529504422 +0000 UTC m=+0.018566861 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:37.877 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 podman[60378]: 2024-04-22 13:24:37.660920974 +0000 UTC m=+0.149983401 container create 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T13:24:37.877 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 podman[60378]: 2024-04-22 13:24:37.85459996 +0000 UTC m=+0.343662387 container init 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-22T13:24:37.878 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 podman[60378]: 2024-04-22 13:24:37.857295025 +0000 UTC m=+0.346357455 container start 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-22T13:24:38.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 -- 172.21.15.5:0/2682447884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb80101b70 msgr2=0x7fdb80101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 --2- 172.21.15.5:0/2682447884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb80101b70 0x7fdb80101ff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009ca0 tx=0x7fdb7802c180 comp rx=0 tx=0).stop 2024-04-22T13:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 -- 172.21.15.5:0/2682447884 shutdown_connections 2024-04-22T13:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 --2- 172.21.15.5:0/2682447884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb80101b70 0x7fdb80101ff0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 --2- 172.21.15.5:0/2682447884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb801011d0 0x7fdb80101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 --2- 172.21.15.5:0/2682447884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 0x7fdb80100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 -- 172.21.15.5:0/2682447884 >> 172.21.15.5:0/2682447884 conn(0x7fdb800fc3f0 msgr2=0x7fdb800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.218+0000 7fdb86150640 1 -- 172.21.15.5:0/2682447884 shutdown_connections 2024-04-22T13:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.219+0000 7fdb86150640 1 -- 172.21.15.5:0/2682447884 wait complete. 2024-04-22T13:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.219+0000 7fdb86150640 1 Processor -- start 2024-04-22T13:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 -- start start 2024-04-22T13:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 0x7fdb8019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 0x7fdb8019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb80101b70 0x7fdb801a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010fcc0 con 0x7fdb801011d0 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010fb40 con 0x7fdb80100870 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb86150640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010fe40 con 0x7fdb80101b70 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 0x7fdb8019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 0x7fdb8019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46116/0 (socket says 172.21.15.5:46116) 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 -- 172.21.15.5:0/273961328 learned_addr learned my addr 172.21.15.5:0/273961328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7f7fe640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 0x7fdb8019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7ffff640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb80101b70 0x7fdb801a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 -- 172.21.15.5:0/273961328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb80101b70 msgr2=0x7fdb801a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb80101b70 0x7fdb801a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 -- 172.21.15.5:0/273961328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 msgr2=0x7fdb8019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 0x7fdb8019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7effd640 1 -- 172.21.15.5:0/273961328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb801a1fb0 con 0x7fdb801011d0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.220+0000 7fdb7f7fe640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 0x7fdb8019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.221+0000 7fdb7effd640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 0x7fdb8019a8d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e9f0 tx=0x7fdb7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.221+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400cd80 con 0x7fdb801011d0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.221+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb740054d0 con 0x7fdb801011d0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.221+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74010690 con 0x7fdb801011d0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.221+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801a22a0 con 0x7fdb801011d0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.221+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a27f0 con 0x7fdb801011d0 2024-04-22T13:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.222+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb80068e00 con 0x7fdb801011d0 2024-04-22T13:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.223+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fdb74004e30 con 0x7fdb801011d0 2024-04-22T13:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.224+0000 7fdb7cff9640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb680a3c30 0x7fdb680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.225+0000 7fdb7f7fe640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb680a3c30 0x7fdb680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.225+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fdb740d04b0 con 0x7fdb801011d0 2024-04-22T13:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.225+0000 7fdb7f7fe640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb680a3c30 0x7fdb680a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb70005d00 tx=0x7fdb7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.228+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb74053ff0 con 0x7fdb801011d0 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:38 smithi049 ceph-mon[31379]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:38 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:38 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 ceph-mds[60397]: -- 172.21.15.49:0/3344931003 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e66448fa40 con 0x55e664462c00 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k[60393]: starting mds.k at 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 bash[60378]: 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 2024-04-22T13:24:38.271 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:24:37 smithi049 systemd[1]: Started Ceph mds.k for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:38.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:38 smithi107 ceph-mon[31859]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:38.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:38 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:38.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:38 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:38.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:38 smithi005 ceph-mon[28006]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:38.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:38 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:38.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:38 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.455+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb8010b5c0 con 0x7fdb801011d0 2024-04-22T13:24:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.455+0000 7fdb7cff9640 1 -- 172.21.15.5:0/273961328 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fdb7409d2d0 con 0x7fdb801011d0 2024-04-22T13:24:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:38.453 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T13:24:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-04-22T13:24:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.457+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb680a3c30 msgr2=0x7fdb680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.457+0000 7fdb86150640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb680a3c30 0x7fdb680a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb70005d00 tx=0x7fdb7000b410 comp rx=0 tx=0).stop 2024-04-22T13:24:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.457+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 msgr2=0x7fdb8019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.457+0000 7fdb86150640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 0x7fdb8019a8d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e9f0 tx=0x7fdb7400edf0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 shutdown_connections 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb680a3c30 0x7fdb680a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb801011d0 0x7fdb8019a8d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb80101b70 0x7fdb801a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 --2- 172.21.15.5:0/273961328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb80100870 0x7fdb8019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 >> 172.21.15.5:0/273961328 conn(0x7fdb800fc3f0 msgr2=0x7fdb800fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 shutdown_connections 2024-04-22T13:24:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:38.458+0000 7fdb86150640 1 -- 172.21.15.5:0/273961328 wait complete. 2024-04-22T13:24:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6832/1569295173,v1:172.21.15.49:6833/1569295173] up:boot 2024-04-22T13:24:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: fsmap 2 up:standby 2024-04-22T13:24:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:24:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:39.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:39.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:39.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:39.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:39 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/273961328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6832/1569295173,v1:172.21.15.49:6833/1569295173] up:boot 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: fsmap 2 up:standby 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:39.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:39 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/273961328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6832/1569295173,v1:172.21.15.49:6833/1569295173] up:boot 2024-04-22T13:24:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: fsmap 2 up:standby 2024-04-22T13:24:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:24:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:39.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:39 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/273961328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:40.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:40 smithi049 ceph-mon[31379]: Deploying daemon mds.j on smithi005 2024-04-22T13:24:40.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:40 smithi049 ceph-mon[31379]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:40.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:40 smithi107 ceph-mon[31859]: Deploying daemon mds.j on smithi005 2024-04-22T13:24:40.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:40 smithi107 ceph-mon[31859]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:40.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:40 smithi005 ceph-mon[28006]: Deploying daemon mds.j on smithi005 2024-04-22T13:24:40.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:40 smithi005 ceph-mon[28006]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:40.308 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 systemd[1]: Starting Ceph mds.j for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:40.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:40.559 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 podman[69022]: 2024-04-22 13:24:40.514620294 +0000 UTC m=+0.097108843 container create 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-22T13:24:40.559 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 podman[69022]: 2024-04-22 13:24:40.435812166 +0000 UTC m=+0.018300724 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 -- 172.21.15.5:0/561647826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810101f90 msgr2=0x7fc8101023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 --2- 172.21.15.5:0/561647826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810101f90 0x7fc8101023b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc800009860 tx=0x7fc80002bb70 comp rx=0 tx=0).stop 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 -- 172.21.15.5:0/561647826 shutdown_connections 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 --2- 172.21.15.5:0/561647826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810103b40 0x7fc81010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 --2- 172.21.15.5:0/561647826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 0x7fc810103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 --2- 172.21.15.5:0/561647826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810101f90 0x7fc8101023b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 -- 172.21.15.5:0/561647826 >> 172.21.15.5:0/561647826 conn(0x7fc8100fd520 msgr2=0x7fc8100ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 -- 172.21.15.5:0/561647826 shutdown_connections 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.578+0000 7fc816414640 1 -- 172.21.15.5:0/561647826 wait complete. 2024-04-22T13:24:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 Processor -- start 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 -- start start 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810101f90 0x7fc810198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 0x7fc8101986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 0x7fc81019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc81010da80 con 0x7fc810103b40 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc81010d900 con 0x7fc810103180 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc816414640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc81010dc00 con 0x7fc810101f90 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 0x7fc81019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 0x7fc81019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48484/0 (socket says 172.21.15.5:48484) 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 -- 172.21.15.5:0/1555684539 learned_addr learned my addr 172.21.15.5:0/1555684539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc80f7fe640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 0x7fc8101986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc80ffff640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810101f90 0x7fc810198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 -- 172.21.15.5:0/1555684539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810101f90 msgr2=0x7fc810198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810101f90 0x7fc810198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 -- 172.21.15.5:0/1555684539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 msgr2=0x7fc8101986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 0x7fc8101986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc8149b1640 1 -- 172.21.15.5:0/1555684539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81019fdb0 con 0x7fc810103b40 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.579+0000 7fc80f7fe640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 0x7fc8101986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.580+0000 7fc8149b1640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 0x7fc81019f740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc80800eb10 tx=0x7fc80800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.580+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808004b20 con 0x7fc810103b40 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.580+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8101a00a0 con 0x7fc810103b40 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.580+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc808004cc0 con 0x7fc810103b40 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.580+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808010690 con 0x7fc810103b40 2024-04-22T13:24:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.580+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8101a0640 con 0x7fc810103b40 2024-04-22T13:24:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.581+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc810109380 con 0x7fc810103b40 2024-04-22T13:24:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.582+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fc808005020 con 0x7fc810103b40 2024-04-22T13:24:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.582+0000 7fc80d7fa640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc7f80a3bb0 0x7fc7f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:40.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.583+0000 7fc80ffff640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc7f80a3bb0 0x7fc7f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:40.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.583+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fc8080d04a0 con 0x7fc810103b40 2024-04-22T13:24:40.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.583+0000 7fc80ffff640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc7f80a3bb0 0x7fc7f80a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc800009830 tx=0x7fc8000349b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:40.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.587+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc80809d320 con 0x7fc810103b40 2024-04-22T13:24:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.836+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc810062b50 con 0x7fc810103b40 2024-04-22T13:24:40.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.836+0000 7fc80d7fa640 1 -- 172.21.15.5:0/1555684539 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fc808087090 con 0x7fc810103b40 2024-04-22T13:24:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:40.834 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T13:24:40.834 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc7f80a3bb0 msgr2=0x7fc7f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc7f80a3bb0 0x7fc7f80a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc800009830 tx=0x7fc8000349b0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 msgr2=0x7fc81019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 0x7fc81019f740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc80800eb10 tx=0x7fc80800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 shutdown_connections 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc7f80a3bb0 0x7fc7f80a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc810103b40 0x7fc81019f740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc810103180 0x7fc8101986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 --2- 172.21.15.5:0/1555684539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810101f90 0x7fc810198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 >> 172.21.15.5:0/1555684539 conn(0x7fc8100fd520 msgr2=0x7fc810106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 shutdown_connections 2024-04-22T13:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:40.838+0000 7fc816414640 1 -- 172.21.15.5:0/1555684539 wait complete. 2024-04-22T13:24:41.059 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 podman[69022]: 2024-04-22 13:24:40.700824409 +0000 UTC m=+0.283312968 container init 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T13:24:41.059 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 podman[69022]: 2024-04-22 13:24:40.70431589 +0000 UTC m=+0.286804445 container start 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T13:24:41.059 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 ceph-mds[69080]: -- 172.21.15.5:0/2622926931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8ce7b1a40 con 0x55b8ce784800 2024-04-22T13:24:41.059 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j[69076]: starting mds.j at 2024-04-22T13:24:41.059 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 bash[69022]: 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 2024-04-22T13:24:41.059 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:24:40 smithi005 systemd[1]: Started Ceph mds.j for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:42.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1555684539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:42.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: Deploying daemon mds.d on smithi005 2024-04-22T13:24:42.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6834/219923343,v1:172.21.15.5:6835/219923343] up:boot 2024-04-22T13:24:42.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: fsmap 3 up:standby 2024-04-22T13:24:42.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:41 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:24:42.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1555684539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: Deploying daemon mds.d on smithi005 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6834/219923343,v1:172.21.15.5:6835/219923343] up:boot 2024-04-22T13:24:42.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: fsmap 3 up:standby 2024-04-22T13:24:42.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:41 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:24:42.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1555684539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: Deploying daemon mds.d on smithi005 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6834/219923343,v1:172.21.15.5:6835/219923343] up:boot 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: fsmap 3 up:standby 2024-04-22T13:24:42.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:41 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:24:42.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 -- 172.21.15.5:0/418546196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4107c60 msgr2=0x7fc3a410a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/418546196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4107c60 0x7fc3a410a0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009ca0 tx=0x7fc39c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 -- 172.21.15.5:0/418546196 shutdown_connections 2024-04-22T13:24:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/418546196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4107c60 0x7fc3a410a0b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/418546196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a4105330 0x7fc3a4107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/418546196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 0x7fc3a4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 -- 172.21.15.5:0/418546196 >> 172.21.15.5:0/418546196 conn(0x7fc3a40fc3f0 msgr2=0x7fc3a40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 -- 172.21.15.5:0/418546196 shutdown_connections 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 -- 172.21.15.5:0/418546196 wait complete. 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.977+0000 7fc3a8e74640 1 Processor -- start 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 -- start start 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 0x7fc3a4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 0x7fc3a4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a4107c60 0x7fc3a419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a410dac0 con 0x7fc3a4105330 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a410d940 con 0x7fc3a4102a00 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a410dc40 con 0x7fc3a4107c60 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 0x7fc3a4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 0x7fc3a4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48530/0 (socket says 172.21.15.5:48530) 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 -- 172.21.15.5:0/1655441219 learned_addr learned my addr 172.21.15.5:0/1655441219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a259c640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 0x7fc3a4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a2d9d640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a4107c60 0x7fc3a419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 -- 172.21.15.5:0/1655441219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a4107c60 msgr2=0x7fc3a419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a4107c60 0x7fc3a419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 -- 172.21.15.5:0/1655441219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 msgr2=0x7fc3a4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 0x7fc3a4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 -- 172.21.15.5:0/1655441219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a419fd30 con 0x7fc3a4105330 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a259c640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 0x7fc3a4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a1d9b640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 0x7fc3a4198650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc39800d740 tx=0x7fc39800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398017070 con 0x7fc3a4105330 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc39800de50 con 0x7fc3a4105330 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.978+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a419ffc0 con 0x7fc3a4105330 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.979+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398004ce0 con 0x7fc3a4105330 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.979+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a41a04e0 con 0x7fc3a4105330 2024-04-22T13:24:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.979+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a40fdee0 con 0x7fc3a4105330 2024-04-22T13:24:42.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.981+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fc398004e80 con 0x7fc3a4105330 2024-04-22T13:24:42.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.981+0000 7fc3937fe640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc38c0a3c30 0x7fc38c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:42.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.981+0000 7fc3a259c640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc38c0a3c30 0x7fc38c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:42.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.981+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fc3980cfcd0 con 0x7fc3a4105330 2024-04-22T13:24:42.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.982+0000 7fc3a259c640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc38c0a3c30 0x7fc38c0a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc3940058c0 tx=0x7fc394005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:42.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:42.985+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc39809cbd0 con 0x7fc3a4105330 2024-04-22T13:24:43.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.220+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3a402d5c0 con 0x7fc3a4105330 2024-04-22T13:24:43.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.221+0000 7fc3937fe640 1 -- 172.21.15.5:0/1655441219 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fc39809c4d0 con 0x7fc3a4105330 2024-04-22T13:24:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:43.219 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T13:24:43.219 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc38c0a3c30 msgr2=0x7fc38c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc38c0a3c30 0x7fc38c0a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc3940058c0 tx=0x7fc394005ec0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 msgr2=0x7fc3a4198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 0x7fc3a4198650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc39800d740 tx=0x7fc39800dc00 comp rx=0 tx=0).stop 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 shutdown_connections 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc38c0a3c30 0x7fc38c0a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4105330 0x7fc3a4198650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a4107c60 0x7fc3a419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 --2- 172.21.15.5:0/1655441219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3a4102a00 0x7fc3a4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 >> 172.21.15.5:0/1655441219 conn(0x7fc3a40fc3f0 msgr2=0x7fc3a4103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 shutdown_connections 2024-04-22T13:24:43.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:43.223+0000 7fc3a8e74640 1 -- 172.21.15.5:0/1655441219 wait complete. 2024-04-22T13:24:44.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1655441219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:44.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:43 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:44.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:44.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1655441219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:44.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:44.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:44.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:43 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:44.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:44.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1655441219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:44.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:44.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:44.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:44.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:43 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:45.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:45.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:44 smithi005 ceph-mon[28006]: Deploying daemon mds.e on smithi049 2024-04-22T13:24:45.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:44 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6836/1096407808,v1:172.21.15.5:6837/1096407808] up:boot 2024-04-22T13:24:45.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:44 smithi005 ceph-mon[28006]: fsmap 4 up:standby 2024-04-22T13:24:45.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:44 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:24:45.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:44 smithi049 ceph-mon[31379]: Deploying daemon mds.e on smithi049 2024-04-22T13:24:45.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:44 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6836/1096407808,v1:172.21.15.5:6837/1096407808] up:boot 2024-04-22T13:24:45.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:44 smithi049 ceph-mon[31379]: fsmap 4 up:standby 2024-04-22T13:24:45.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:44 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:24:45.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:44 smithi107 ceph-mon[31859]: Deploying daemon mds.e on smithi049 2024-04-22T13:24:45.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:44 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6836/1096407808,v1:172.21.15.5:6837/1096407808] up:boot 2024-04-22T13:24:45.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:44 smithi107 ceph-mon[31859]: fsmap 4 up:standby 2024-04-22T13:24:45.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:44 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 -- 172.21.15.5:0/1030894021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24102a20 msgr2=0x7fdc24104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/1030894021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24102a20 0x7fdc24104e10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009900 tx=0x7fdc2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 -- 172.21.15.5:0/1030894021 shutdown_connections 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/1030894021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc24107c80 0x7fdc2410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/1030894021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc24105350 0x7fdc24107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/1030894021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24102a20 0x7fdc24104e10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 -- 172.21.15.5:0/1030894021 >> 172.21.15.5:0/1030894021 conn(0x7fdc240fc410 msgr2=0x7fdc240fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 -- 172.21.15.5:0/1030894021 shutdown_connections 2024-04-22T13:24:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.354+0000 7fdc2c22e640 1 -- 172.21.15.5:0/1030894021 wait complete. 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 Processor -- start 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 -- start start 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc24102a20 0x7fdc241981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 0x7fdc24198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc24107c80 0x7fdc2419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410db40 con 0x7fdc24105350 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410d9c0 con 0x7fdc24107c80 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2c22e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410dcc0 con 0x7fdc24102a20 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc297c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 0x7fdc24198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc297c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 0x7fdc24198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48572/0 (socket says 172.21.15.5:48572) 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc297c9640 1 -- 172.21.15.5:0/2809260456 learned_addr learned my addr 172.21.15.5:0/2809260456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc2a7cb640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc24107c80 0x7fdc2419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.355+0000 7fdc29fca640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc24102a20 0x7fdc241981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc297c9640 1 -- 172.21.15.5:0/2809260456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc24102a20 msgr2=0x7fdc241981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc297c9640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc24102a20 0x7fdc241981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc297c9640 1 -- 172.21.15.5:0/2809260456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc24107c80 msgr2=0x7fdc2419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc297c9640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc24107c80 0x7fdc2419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc297c9640 1 -- 172.21.15.5:0/2809260456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2419fe00 con 0x7fdc24105350 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc297c9640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 0x7fdc24198720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b450 tx=0x7fdc1400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14015020 con 0x7fdc24105350 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc241a00f0 con 0x7fdc24105350 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc14011d40 con 0x7fdc24105350 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400f340 con 0x7fdc24105350 2024-04-22T13:24:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.356+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a06c0 con 0x7fdc24105350 2024-04-22T13:24:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.357+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc240fdf00 con 0x7fdc24105350 2024-04-22T13:24:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.358+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fdc14018020 con 0x7fdc24105350 2024-04-22T13:24:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.359+0000 7fdc1affd640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdc100a3bb0 0x7fdc100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.359+0000 7fdc29fca640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdc100a3bb0 0x7fdc100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.359+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fdc140cfaa0 con 0x7fdc24105350 2024-04-22T13:24:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.359+0000 7fdc29fca640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdc100a3bb0 0x7fdc100a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000c00 tx=0x7fdc20005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.364+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc1401f020 con 0x7fdc24105350 2024-04-22T13:24:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.590+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc2402d5c0 con 0x7fdc24105350 2024-04-22T13:24:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.591+0000 7fdc1affd640 1 -- 172.21.15.5:0/2809260456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7fdc1400d040 con 0x7fdc24105350 2024-04-22T13:24:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:45.589 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T13:24:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-04-22T13:24:45.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdc100a3bb0 msgr2=0x7fdc100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdc100a3bb0 0x7fdc100a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000c00 tx=0x7fdc20005be0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 msgr2=0x7fdc24198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 0x7fdc24198720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b450 tx=0x7fdc1400b910 comp rx=0 tx=0).stop 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 shutdown_connections 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdc100a3bb0 0x7fdc100a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc24105350 0x7fdc24198720 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc24107c80 0x7fdc2419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 --2- 172.21.15.5:0/2809260456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc24102a20 0x7fdc241981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 >> 172.21.15.5:0/2809260456 conn(0x7fdc240fc410 msgr2=0x7fdc241034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 shutdown_connections 2024-04-22T13:24:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:45.593+0000 7fdc2c22e640 1 -- 172.21.15.5:0/2809260456 wait complete. 2024-04-22T13:24:46.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:45 smithi049 ceph-mon[31379]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:46.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:45 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2809260456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:46.022 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:45 smithi049 systemd[1]: Starting Ceph mds.e for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:46.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:45 smithi005 ceph-mon[28006]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:46.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:45 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2809260456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:45 smithi107 ceph-mon[31859]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:46.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:45 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2809260456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:46.412 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 podman[60696]: 2024-04-22 13:24:46.061720134 +0000 UTC m=+0.018643032 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:46.413 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 podman[60696]: 2024-04-22 13:24:46.163367017 +0000 UTC m=+0.120289893 container create 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T13:24:46.413 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 podman[60696]: 2024-04-22 13:24:46.391136965 +0000 UTC m=+0.348059839 container init 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-22T13:24:46.413 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 podman[60696]: 2024-04-22 13:24:46.393771642 +0000 UTC m=+0.350694516 container start 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:24:46.771 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 ceph-mds[60716]: -- 172.21.15.49:0/769673717 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5616b1a17a40 con 0x5616b19eb000 2024-04-22T13:24:46.771 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e[60712]: starting mds.e at 2024-04-22T13:24:46.771 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 bash[60696]: 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 2024-04-22T13:24:46.771 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:24:46 smithi049 systemd[1]: Started Ceph mds.e for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:47.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.719+0000 7fc53ae5c640 1 -- 172.21.15.5:0/1183833657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 msgr2=0x7fc5341023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.719+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/1183833657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc5341023b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc524009900 tx=0x7fc52402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- 172.21.15.5:0/1183833657 shutdown_connections 2024-04-22T13:24:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/1183833657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc534103b40 0x7fc53410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/1183833657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc534103180 0x7fc534103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/1183833657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc5341023b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- 172.21.15.5:0/1183833657 >> 172.21.15.5:0/1183833657 conn(0x7fc5340fd520 msgr2=0x7fc5340ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- 172.21.15.5:0/1183833657 shutdown_connections 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- 172.21.15.5:0/1183833657 wait complete. 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 Processor -- start 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- start start 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc534198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc534103180 0x7fc5341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc534103b40 0x7fc53419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410dad0 con 0x7fc534101f90 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410d950 con 0x7fc534103b40 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc53ae5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410dc50 con 0x7fc534103180 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.720+0000 7fc538bf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc534198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc534198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48596/0 (socket says 172.21.15.5:48596) 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 -- 172.21.15.5:0/48021083 learned_addr learned my addr 172.21.15.5:0/48021083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc5393f9640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc534103b40 0x7fc53419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc533fff640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc534103180 0x7fc5341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 -- 172.21.15.5:0/48021083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc534103180 msgr2=0x7fc5341986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc534103180 0x7fc5341986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 -- 172.21.15.5:0/48021083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc534103b40 msgr2=0x7fc53419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc534103b40 0x7fc53419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 -- 172.21.15.5:0/48021083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53419fdb0 con 0x7fc534101f90 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc538bf8640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc534198190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524043020 con 0x7fc534101f90 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc524004aa0 con 0x7fc534101f90 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5341a0040 con 0x7fc534101f90 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.721+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524003d00 con 0x7fc534101f90 2024-04-22T13:24:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.722+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5341a0590 con 0x7fc534101f90 2024-04-22T13:24:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.723+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5341092d0 con 0x7fc534101f90 2024-04-22T13:24:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.724+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fc524046020 con 0x7fc534101f90 2024-04-22T13:24:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.725+0000 7fc531ffb640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.725+0000 7fc533fff640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.725+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fc5240f2380 con 0x7fc534101f90 2024-04-22T13:24:47.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.725+0000 7fc533fff640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc528005fd0 tx=0x7fc528005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:47.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.730+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc52404d020 con 0x7fc534101f90 2024-04-22T13:24:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:47.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: Deploying daemon mds.f on smithi107 2024-04-22T13:24:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] up:boot 2024-04-22T13:24:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: fsmap 5 up:standby 2024-04-22T13:24:47.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:47 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:24:47.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:47.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: Deploying daemon mds.f on smithi107 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] up:boot 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: fsmap 5 up:standby 2024-04-22T13:24:47.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:47 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:24:47.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:47.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: Deploying daemon mds.f on smithi107 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] up:boot 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: fsmap 5 up:standby 2024-04-22T13:24:47.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:47 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:24:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.955+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc53402d5c0 con 0x7fc534101f90 2024-04-22T13:24:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.955+0000 7fc531ffb640 1 -- 172.21.15.5:0/48021083 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4684 (secure 0 0 0) 0x7fc5240bf200 con 0x7fc534101f90 2024-04-22T13:24:47.953 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:47.953 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T13:24:47.953 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc51c0a3bb0 msgr2=0x7fc51c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc528005fd0 tx=0x7fc528005850 comp rx=0 tx=0).stop 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 msgr2=0x7fc534198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc534198190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524004270 comp rx=0 tx=0).stop 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 shutdown_connections 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc534103b40 0x7fc53419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc534103180 0x7fc5341986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 --2- 172.21.15.5:0/48021083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc534101f90 0x7fc534198190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 >> 172.21.15.5:0/48021083 conn(0x7fc5340fd520 msgr2=0x7fc5341064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 shutdown_connections 2024-04-22T13:24:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:47.957+0000 7fc53ae5c640 1 -- 172.21.15.5:0/48021083 wait complete. 2024-04-22T13:24:48.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:48 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/48021083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:48.785 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:48 smithi107 systemd[1]: Starting Ceph mds.f for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:48.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:48 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/48021083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:49.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:48 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/48021083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:49.086 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:48 smithi107 podman[62950]: 2024-04-22 13:24:48.741754661 +0000 UTC m=+0.018383172 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:49.086 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:48 smithi107 podman[62950]: 2024-04-22 13:24:48.845344097 +0000 UTC m=+0.121972604 container create 31f1c4059595dc0e0ebad726dbcd41c3c298e91881fa6278fa2135b6a83f811b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:24:49.086 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:49 smithi107 podman[62950]: 2024-04-22 13:24:49.064578021 +0000 UTC m=+0.341206527 container init 31f1c4059595dc0e0ebad726dbcd41c3c298e91881fa6278fa2135b6a83f811b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:24:49.086 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:49 smithi107 podman[62950]: 2024-04-22 13:24:49.067234188 +0000 UTC m=+0.343862698 container start 31f1c4059595dc0e0ebad726dbcd41c3c298e91881fa6278fa2135b6a83f811b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-22T13:24:49.087 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mds[62969]: -- 172.21.15.107:0/3329239401 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55982f517a40 con 0x55982f4ea800 2024-04-22T13:24:49.531 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: starting mds.f at 2024-04-22T13:24:49.531 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:49 smithi107 bash[62950]: 31f1c4059595dc0e0ebad726dbcd41c3c298e91881fa6278fa2135b6a83f811b 2024-04-22T13:24:49.531 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:24:49 smithi107 systemd[1]: Started Ceph mds.f for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:49.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:49.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:49.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:49 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:49.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:49.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:49.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:49.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:49.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:49.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:49.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:49 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:49.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:50.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:49 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 -- 172.21.15.5:0/3076112654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 msgr2=0x7fa2641012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 --2- 172.21.15.5:0/3076112654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa2641012a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa254009900 tx=0x7fa25402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 -- 172.21.15.5:0/3076112654 shutdown_connections 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 --2- 172.21.15.5:0/3076112654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264102990 0x7fa2641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 --2- 172.21.15.5:0/3076112654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264101fd0 0x7fa264102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 --2- 172.21.15.5:0/3076112654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa2641012a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 -- 172.21.15.5:0/3076112654 >> 172.21.15.5:0/3076112654 conn(0x7fa2640fc3f0 msgr2=0x7fa2640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 -- 172.21.15.5:0/3076112654 shutdown_connections 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 -- 172.21.15.5:0/3076112654 wait complete. 2024-04-22T13:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.086+0000 7fa2697ca640 1 Processor -- start 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 -- start start 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa264069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264101fd0 0x7fa26419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264102990 0x7fa26419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa26410c9d0 con 0x7fa264100e80 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa26410c850 con 0x7fa264101fd0 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2697ca640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa26410cb50 con 0x7fa264102990 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa264069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa264069dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39890/0 (socket says 172.21.15.5:39890) 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 -- 172.21.15.5:0/24626226 learned_addr learned my addr 172.21.15.5:0/24626226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2637fe640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264102990 0x7fa26419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2627fc640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264101fd0 0x7fa26419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 -- 172.21.15.5:0/24626226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264102990 msgr2=0x7fa26419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264102990 0x7fa26419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 -- 172.21.15.5:0/24626226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264101fd0 msgr2=0x7fa26419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264101fd0 0x7fa26419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 -- 172.21.15.5:0/24626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26419fe20 con 0x7fa264100e80 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2637fe640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264102990 0x7fa26419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa2627fc640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264101fd0 0x7fa26419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.087+0000 7fa262ffd640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa264069dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa254006010 tx=0x7fa254005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.088+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254043020 con 0x7fa264100e80 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.088+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2641a0050 con 0x7fa264100e80 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.088+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2540043d0 con 0x7fa264100e80 2024-04-22T13:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.088+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254003b80 con 0x7fa264100e80 2024-04-22T13:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.088+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2641a05f0 con 0x7fa264100e80 2024-04-22T13:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.089+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa264108230 con 0x7fa264100e80 2024-04-22T13:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.090+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fa254046020 con 0x7fa264100e80 2024-04-22T13:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.091+0000 7fa24bfff640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa24c0a3bb0 0x7fa24c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.091+0000 7fa2627fc640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa24c0a3bb0 0x7fa24c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.091+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fa2540f22a0 con 0x7fa264100e80 2024-04-22T13:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.091+0000 7fa2627fc640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa24c0a3bb0 0x7fa24c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa264069b30 tx=0x7fa258005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.096+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa25404d020 con 0x7fa264100e80 2024-04-22T13:24:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.325+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa26402d810 con 0x7fa264100e80 2024-04-22T13:24:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.326+0000 7fa24bfff640 1 -- 172.21.15.5:0/24626226 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5512 (secure 0 0 0) 0x7fa25403c350 con 0x7fa264100e80 2024-04-22T13:24:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:50.322 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T13:24:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-04-22T13:24:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa24c0a3bb0 msgr2=0x7fa24c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa24c0a3bb0 0x7fa24c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa264069b30 tx=0x7fa258005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 msgr2=0x7fa264069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa264069dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa254006010 tx=0x7fa254005e40 comp rx=0 tx=0).stop 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 shutdown_connections 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa24c0a3bb0 0x7fa24c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa264102990 0x7fa26419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa264101fd0 0x7fa26419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 --2- 172.21.15.5:0/24626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa264100e80 0x7fa264069dc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 >> 172.21.15.5:0/24626226 conn(0x7fa2640fc3f0 msgr2=0x7fa264105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 shutdown_connections 2024-04-22T13:24:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:50.328+0000 7fa2697ca640 1 -- 172.21.15.5:0/24626226 wait complete. 2024-04-22T13:24:50.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:50 smithi005 ceph-mon[28006]: Deploying daemon mds.c on smithi107 2024-04-22T13:24:50.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:50 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6834/2603097758,v1:172.21.15.107:6835/2603097758] up:boot 2024-04-22T13:24:50.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:50 smithi005 ceph-mon[28006]: fsmap 6 up:standby 2024-04-22T13:24:50.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:50 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:24:50.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:50 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/24626226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:50.833 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:50 smithi107 ceph-mon[31859]: Deploying daemon mds.c on smithi107 2024-04-22T13:24:50.833 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:50 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6834/2603097758,v1:172.21.15.107:6835/2603097758] up:boot 2024-04-22T13:24:50.833 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:50 smithi107 ceph-mon[31859]: fsmap 6 up:standby 2024-04-22T13:24:50.834 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:50 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:24:50.834 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:50 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/24626226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:50.838 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:50 smithi049 ceph-mon[31379]: Deploying daemon mds.c on smithi107 2024-04-22T13:24:50.838 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:50 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6834/2603097758,v1:172.21.15.107:6835/2603097758] up:boot 2024-04-22T13:24:50.839 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:50 smithi049 ceph-mon[31379]: fsmap 6 up:standby 2024-04-22T13:24:50.839 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:50 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:24:50.839 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:50 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/24626226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:51.502 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 systemd[1]: Starting Ceph mds.c for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:51.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:51 smithi107 ceph-mon[31859]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:51.785 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 podman[63262]: 2024-04-22 13:24:51.408960075 +0000 UTC m=+0.017661120 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:51.785 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 podman[63262]: 2024-04-22 13:24:51.506562655 +0000 UTC m=+0.115263689 container create cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-22T13:24:51.785 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 podman[63262]: 2024-04-22 13:24:51.725642965 +0000 UTC m=+0.334343998 container init cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:24:51.785 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 podman[63262]: 2024-04-22 13:24:51.72819395 +0000 UTC m=+0.336894986 container start cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:24:51.785 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 ceph-mds[63281]: -- 172.21.15.107:0/2413025050 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d98f063a40 con 0x55d98f037000 2024-04-22T13:24:51.785 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c[63277]: starting mds.c at 2024-04-22T13:24:51.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:51 smithi005 ceph-mon[28006]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:52.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:51 smithi049 ceph-mon[31379]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:52.284 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 bash[63262]: cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 2024-04-22T13:24:52.284 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:24:51 smithi107 systemd[1]: Started Ceph mds.c for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:52.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 -- 172.21.15.5:0/1241896449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 msgr2=0x7fe7641023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 --2- 172.21.15.5:0/1241896449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe7641023c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe754009900 tx=0x7fe75402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 -- 172.21.15.5:0/1241896449 shutdown_connections 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 --2- 172.21.15.5:0/1241896449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103b50 0x7fe76410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 --2- 172.21.15.5:0/1241896449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764103190 0x7fe764103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 --2- 172.21.15.5:0/1241896449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe7641023c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 -- 172.21.15.5:0/1241896449 >> 172.21.15.5:0/1241896449 conn(0x7fe7640fd570 msgr2=0x7fe7640ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 -- 172.21.15.5:0/1241896449 shutdown_connections 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 -- 172.21.15.5:0/1241896449 wait complete. 2024-04-22T13:24:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.455+0000 7fe769b7e640 1 Processor -- start 2024-04-22T13:24:52.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 -- start start 2024-04-22T13:24:52.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe764075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:52.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103190 0x7fe764074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:52.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764103b50 0x7fe764074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:52.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe76410daa0 con 0x7fe764101fa0 2024-04-22T13:24:52.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe76410d920 con 0x7fe764103b50 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe769b7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe76410dc20 con 0x7fe764103190 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe764075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe764075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39916/0 (socket says 172.21.15.5:39916) 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 -- 172.21.15.5:0/4204860031 learned_addr learned my addr 172.21.15.5:0/4204860031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe763fff640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764103b50 0x7fe764074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe762ffd640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103190 0x7fe764074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 -- 172.21.15.5:0/4204860031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103190 msgr2=0x7fe764074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103190 0x7fe764074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 -- 172.21.15.5:0/4204860031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764103b50 msgr2=0x7fe764074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764103b50 0x7fe764074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 -- 172.21.15.5:0/4204860031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe764075110 con 0x7fe764101fa0 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe762ffd640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103190 0x7fe764074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.456+0000 7fe7637fe640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe764075d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe754000c00 tx=0x7fe754004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.457+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754043020 con 0x7fe764101fa0 2024-04-22T13:24:52.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.457+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641a2260 con 0x7fe764101fa0 2024-04-22T13:24:52.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.457+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe754004440 con 0x7fe764101fa0 2024-04-22T13:24:52.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.457+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754003c90 con 0x7fe764101fa0 2024-04-22T13:24:52.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.457+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a2800 con 0x7fe764101fa0 2024-04-22T13:24:52.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.458+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7641092e0 con 0x7fe764101fa0 2024-04-22T13:24:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.459+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe754046020 con 0x7fe764101fa0 2024-04-22T13:24:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.459+0000 7fe760ff9640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe74c0a3bb0 0x7fe74c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.460+0000 7fe762ffd640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe74c0a3bb0 0x7fe74c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.460+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fe7540f21b0 con 0x7fe764101fa0 2024-04-22T13:24:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.460+0000 7fe762ffd640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe74c0a3bb0 0x7fe74c0a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe764075a70 tx=0x7fe758005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:52.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.464+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe75404d020 con 0x7fe764101fa0 2024-04-22T13:24:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.692+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe76402d5c0 con 0x7fe764101fa0 2024-04-22T13:24:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.693+0000 7fe760ff9640 1 -- 172.21.15.5:0/4204860031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6337 (secure 0 0 0) 0x7fe7540bf030 con 0x7fe764101fa0 2024-04-22T13:24:52.689 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:52.689 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T13:24:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-22T13:24:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe74c0a3bb0 msgr2=0x7fe74c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe74c0a3bb0 0x7fe74c0a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe764075a70 tx=0x7fe758005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 msgr2=0x7fe764075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe764075d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe754000c00 tx=0x7fe754004040 comp rx=0 tx=0).stop 2024-04-22T13:24:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 shutdown_connections 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe74c0a3bb0 0x7fe74c0a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764103b50 0x7fe764074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe764103190 0x7fe764074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 --2- 172.21.15.5:0/4204860031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe764101fa0 0x7fe764075d00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 >> 172.21.15.5:0/4204860031 conn(0x7fe7640fd570 msgr2=0x7fe7641064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 shutdown_connections 2024-04-22T13:24:52.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:52.695+0000 7fe769b7e640 1 -- 172.21.15.5:0/4204860031 wait complete. 2024-04-22T13:24:53.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: Deploying daemon mds.h on smithi049 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] up:boot 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: fsmap 7 up:standby 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:24:53.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:52 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4204860031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:53.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: Deploying daemon mds.h on smithi049 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] up:boot 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: fsmap 7 up:standby 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:24:53.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:52 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4204860031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:53.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: Deploying daemon mds.h on smithi049 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] up:boot 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: fsmap 7 up:standby 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:24:53.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:52 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4204860031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:54.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:53 smithi049 ceph-mon[31379]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:54.271 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 systemd[1]: Starting Ceph mds.h for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:54.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:53 smithi107 ceph-mon[31859]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:54.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:53 smithi005 ceph-mon[28006]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:54.631 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 podman[61007]: 2024-04-22 13:24:54.349929017 +0000 UTC m=+0.100188082 container create fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-22T13:24:54.632 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 podman[61007]: 2024-04-22 13:24:54.26830124 +0000 UTC m=+0.018560343 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:24:54.632 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 podman[61007]: 2024-04-22 13:24:54.569093235 +0000 UTC m=+0.319352304 container init fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-22T13:24:54.632 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 podman[61007]: 2024-04-22 13:24:54.571791492 +0000 UTC m=+0.322050567 container start fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:24:54.632 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 ceph-mds[61027]: -- 172.21.15.49:0/3243825150 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609b34ada40 con 0x5609b3481000 2024-04-22T13:24:54.632 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h[61023]: starting mds.h at 2024-04-22T13:24:54.632 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 bash[61007]: fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 2024-04-22T13:24:54.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:54.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 -- 172.21.15.5:0/3240146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 msgr2=0x7f776410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 --2- 172.21.15.5:0/3240146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776410a0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f775c009900 tx=0x7f775c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 -- 172.21.15.5:0/3240146685 shutdown_connections 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 --2- 172.21.15.5:0/3240146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776410a0e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 --2- 172.21.15.5:0/3240146685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764105360 0x7f7764107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 --2- 172.21.15.5:0/3240146685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7764102a30 0x7f7764104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 -- 172.21.15.5:0/3240146685 >> 172.21.15.5:0/3240146685 conn(0x7f77640fc460 msgr2=0x7f77640fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 -- 172.21.15.5:0/3240146685 shutdown_connections 2024-04-22T13:24:54.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.821+0000 7f776b806640 1 -- 172.21.15.5:0/3240146685 wait complete. 2024-04-22T13:24:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 Processor -- start 2024-04-22T13:24:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 -- start start 2024-04-22T13:24:54.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764102a30 0x7f7764198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7764105360 0x7f77641986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f776410daf0 con 0x7f7764107c90 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f776410d970 con 0x7f7764102a30 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f776b806640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f776410dc70 con 0x7f7764105360 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39934/0 (socket says 172.21.15.5:39934) 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 -- 172.21.15.5:0/788949407 learned_addr learned my addr 172.21.15.5:0/788949407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7768da1640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7764105360 0x7f77641986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f77695a2640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764102a30 0x7f7764198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 -- 172.21.15.5:0/788949407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7764105360 msgr2=0x7f77641986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7764105360 0x7f77641986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 -- 172.21.15.5:0/788949407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764102a30 msgr2=0x7f7764198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764102a30 0x7f7764198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 -- 172.21.15.5:0/788949407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776419fd60 con 0x7f7764107c90 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.822+0000 7f7769da3640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776419f6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f775c009230 tx=0x7f775c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.823+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c043020 con 0x7f7764107c90 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.823+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f775c0044c0 con 0x7f7764107c90 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.823+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776419fff0 con 0x7f7764107c90 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.823+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c003bc0 con 0x7f7764107c90 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.823+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641a04b0 con 0x7f7764107c90 2024-04-22T13:24:54.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.824+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77640fdf50 con 0x7f7764107c90 2024-04-22T13:24:54.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.825+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f775c046020 con 0x7f7764107c90 2024-04-22T13:24:54.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.825+0000 7f775a7fc640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f77500a3bb0 0x7f77500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.826+0000 7f77695a2640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f77500a3bb0 0x7f77500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.826+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f775c0f22b0 con 0x7f7764107c90 2024-04-22T13:24:54.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.826+0000 7f77695a2640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f77500a3bb0 0x7f77500a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7760005fd0 tx=0x7f7760009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:54.830+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f775c04d020 con 0x7f7764107c90 2024-04-22T13:24:55.021 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:24:54 smithi049 systemd[1]: Started Ceph mds.h for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.059+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f776402d5c0 con 0x7f7764107c90 2024-04-22T13:24:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.059+0000 7f775a7fc640 1 -- 172.21.15.5:0/788949407 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7f775c0bf130 con 0x7f7764107c90 2024-04-22T13:24:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:55.056 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T13:24:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f77500a3bb0 msgr2=0x7f77500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f77500a3bb0 0x7f77500a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7760005fd0 tx=0x7f7760009500 comp rx=0 tx=0).stop 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 msgr2=0x7f776419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776419f6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f775c009230 tx=0x7f775c0040c0 comp rx=0 tx=0).stop 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 shutdown_connections 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f77500a3bb0 0x7f77500a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7764107c90 0x7f776419f6f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7764105360 0x7f77641986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 --2- 172.21.15.5:0/788949407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764102a30 0x7f7764198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 >> 172.21.15.5:0/788949407 conn(0x7f77640fc460 msgr2=0x7f7764103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 shutdown_connections 2024-04-22T13:24:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:55.061+0000 7f776b806640 1 -- 172.21.15.5:0/788949407 wait complete. 2024-04-22T13:24:56.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:56.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: Deploying daemon mds.i on smithi107 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] up:boot 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: fsmap 8 up:standby 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:24:56.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:55 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/788949407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:56.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:56.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: Deploying daemon mds.i on smithi107 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] up:boot 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: fsmap 8 up:standby 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:24:56.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:55 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/788949407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:56.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:56.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: Deploying daemon mds.i on smithi107 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] up:boot 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: fsmap 8 up:standby 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:24:56.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:55 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/788949407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:57.035 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:56 smithi107 podman[63574]: 2024-04-22 13:24:56.951173773 +0000 UTC m=+0.119207599 container create 3d90cf10b3400a4302b84c7fb1523e0f9d2984f9545a4259920d708b9ab3fb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-22T13:24:57.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:57.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 -- 172.21.15.5:0/1263513964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 msgr2=0x7f575010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:57.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 --2- 172.21.15.5:0/1263513964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f575010a3e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5740009900 tx=0x7f574002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:57.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 -- 172.21.15.5:0/1263513964 shutdown_connections 2024-04-22T13:24:57.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 --2- 172.21.15.5:0/1263513964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f575010a3e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 --2- 172.21.15.5:0/1263513964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750103190 0x7f5750103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 --2- 172.21.15.5:0/1263513964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5750101fa0 0x7f57501023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 -- 172.21.15.5:0/1263513964 >> 172.21.15.5:0/1263513964 conn(0x7f57500fd570 msgr2=0x7f57500ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 -- 172.21.15.5:0/1263513964 shutdown_connections 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.191+0000 7f5754af1640 1 -- 172.21.15.5:0/1263513964 wait complete. 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 Processor -- start 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 -- start start 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750101fa0 0x7f5750075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5750103190 0x7f5750074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f5750074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f575010dae0 con 0x7f5750103b50 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f575010d960 con 0x7f5750101fa0 2024-04-22T13:24:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f5754af1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f575010dc60 con 0x7f5750103190 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f574ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f5750074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f574ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f5750074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39954/0 (socket says 172.21.15.5:39954) 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f574ed9d640 1 -- 172.21.15.5:0/714063411 learned_addr learned my addr 172.21.15.5:0/714063411 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.192+0000 7f574dd9b640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5750103190 0x7f5750074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574e59c640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750101fa0 0x7f5750075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574ed9d640 1 -- 172.21.15.5:0/714063411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5750103190 msgr2=0x7f5750074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574ed9d640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5750103190 0x7f5750074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574ed9d640 1 -- 172.21.15.5:0/714063411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750101fa0 msgr2=0x7f5750075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574ed9d640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750101fa0 0x7f5750075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574ed9d640 1 -- 172.21.15.5:0/714063411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5750075110 con 0x7f5750103b50 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574e59c640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750101fa0 0x7f5750075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f574ed9d640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f5750074880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5740009230 tx=0x7f57400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740043020 con 0x7f5750103b50 2024-04-22T13:24:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57501a2200 con 0x7f5750103b50 2024-04-22T13:24:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57400044c0 con 0x7f5750103b50 2024-04-22T13:24:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740003bc0 con 0x7f5750103b50 2024-04-22T13:24:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.193+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a27a0 con 0x7f5750103b50 2024-04-22T13:24:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.194+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57501092c0 con 0x7f5750103b50 2024-04-22T13:24:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.195+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5740046020 con 0x7f5750103b50 2024-04-22T13:24:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.196+0000 7f573f7fe640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57380a3bb0 0x7f57380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.196+0000 7f574e59c640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57380a3bb0 0x7f57380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.196+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f57400f2720 con 0x7f5750103b50 2024-04-22T13:24:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.197+0000 7f574e59c640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57380a3bb0 0x7f57380a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5750102ff0 tx=0x7f5748009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:57.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.200+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f574004d020 con 0x7f5750103b50 2024-04-22T13:24:57.320 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:57 smithi107 podman[63574]: 2024-04-22 13:24:57.195200711 +0000 UTC m=+0.363234495 container init 3d90cf10b3400a4302b84c7fb1523e0f9d2984f9545a4259920d708b9ab3fb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-22T13:24:57.321 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:57 smithi107 podman[63574]: 2024-04-22 13:24:57.198730097 +0000 UTC m=+0.366763881 container start 3d90cf10b3400a4302b84c7fb1523e0f9d2984f9545a4259920d708b9ab3fb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-22T13:24:57.321 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mds[63594]: -- 172.21.15.107:0/929908588 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56014c317a40 con 0x56014c2ea800 2024-04-22T13:24:57.321 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i[63589]: starting mds.i at 2024-04-22T13:24:57.321 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:57 smithi107 bash[63574]: 3d90cf10b3400a4302b84c7fb1523e0f9d2984f9545a4259920d708b9ab3fb0a 2024-04-22T13:24:57.321 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:24:57 smithi107 systemd[1]: Started Ceph mds.i for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:24:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.434+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5750062b50 con 0x7f5750103b50 2024-04-22T13:24:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.435+0000 7f573f7fe640 1 -- 172.21.15.5:0/714063411 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7f57400bf500 con 0x7f5750103b50 2024-04-22T13:24:57.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:57.432 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T13:24:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57380a3bb0 msgr2=0x7f57380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57380a3bb0 0x7f57380a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5750102ff0 tx=0x7f5748009500 comp rx=0 tx=0).stop 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 msgr2=0x7f5750074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f5750074880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5740009230 tx=0x7f57400040c0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 shutdown_connections 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57380a3bb0 0x7f57380a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5750103b50 0x7f5750074880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5750103190 0x7f5750074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 --2- 172.21.15.5:0/714063411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5750101fa0 0x7f5750075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 >> 172.21.15.5:0/714063411 conn(0x7f57500fd570 msgr2=0x7f5750106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 shutdown_connections 2024-04-22T13:24:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:57.437+0000 7f5754af1640 1 -- 172.21.15.5:0/714063411 wait complete. 2024-04-22T13:24:57.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:57.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:57.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:57 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/714063411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:58.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:58.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:58.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:58.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:58.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:58.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:58.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:58.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:57 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/714063411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:58.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:58.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:58.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:58.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:24:58.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:24:58.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:24:58.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:58.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:57 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/714063411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:59.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:58 smithi049 ceph-mon[31379]: Deploying daemon mds.g on smithi005 2024-04-22T13:24:59.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:58 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] up:boot 2024-04-22T13:24:59.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:58 smithi049 ceph-mon[31379]: fsmap 9 up:standby 2024-04-22T13:24:59.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:58 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:24:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:58 smithi107 ceph-mon[31859]: Deploying daemon mds.g on smithi005 2024-04-22T13:24:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:58 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] up:boot 2024-04-22T13:24:59.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:58 smithi107 ceph-mon[31859]: fsmap 9 up:standby 2024-04-22T13:24:59.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:58 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:24:59.053 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:58 smithi005 ceph-mon[28006]: Deploying daemon mds.g on smithi005 2024-04-22T13:24:59.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:58 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] up:boot 2024-04-22T13:24:59.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:58 smithi005 ceph-mon[28006]: fsmap 9 up:standby 2024-04-22T13:24:59.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:58 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:24:59.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:59.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.653+0000 7efefc68c640 1 -- 172.21.15.5:0/1194545627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4107c60 msgr2=0x7efef410a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:59.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.653+0000 7efefc68c640 1 --2- 172.21.15.5:0/1194545627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4107c60 0x7efef410a0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efeec009ca0 tx=0x7efeec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 -- 172.21.15.5:0/1194545627 shutdown_connections 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 --2- 172.21.15.5:0/1194545627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4107c60 0x7efef410a0b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 --2- 172.21.15.5:0/1194545627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef4105330 0x7efef4107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 --2- 172.21.15.5:0/1194545627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef4102a00 0x7efef4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 -- 172.21.15.5:0/1194545627 >> 172.21.15.5:0/1194545627 conn(0x7efef40fc3f0 msgr2=0x7efef40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 -- 172.21.15.5:0/1194545627 shutdown_connections 2024-04-22T13:24:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 -- 172.21.15.5:0/1194545627 wait complete. 2024-04-22T13:24:59.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.654+0000 7efefc68c640 1 Processor -- start 2024-04-22T13:24:59.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 -- start start 2024-04-22T13:24:59.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 0x7efef4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef4105330 0x7efef4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef4107c60 0x7efef419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efef410dac0 con 0x7efef4102a00 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efef410d940 con 0x7efef4107c60 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefc68c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efef410dc40 con 0x7efef4105330 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 0x7efef4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 0x7efef4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44732/0 (socket says 172.21.15.5:44732) 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 -- 172.21.15.5:0/213002496 learned_addr learned my addr 172.21.15.5:0/213002496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefac29640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef4107c60 0x7efef419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efef9c27640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef4105330 0x7efef4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 -- 172.21.15.5:0/213002496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef4105330 msgr2=0x7efef4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef4105330 0x7efef4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 -- 172.21.15.5:0/213002496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef4107c60 msgr2=0x7efef419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef4107c60 0x7efef419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 -- 172.21.15.5:0/213002496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef419fd30 con 0x7efef4102a00 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.655+0000 7efefa428640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 0x7efef4198110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efef000e9f0 tx=0x7efef000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.656+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef000cd80 con 0x7efef4102a00 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.656+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef419ffc0 con 0x7efef4102a00 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.656+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef00108e0 con 0x7efef4102a00 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.656+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41a04d0 con 0x7efef4102a00 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.656+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0018980 con 0x7efef4102a00 2024-04-22T13:24:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.657+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef40fdee0 con 0x7efef4102a00 2024-04-22T13:24:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.658+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7efef0010430 con 0x7efef4102a00 2024-04-22T13:24:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.659+0000 7efeeb7fe640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efee00a3bb0 0x7efee00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.659+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7efef00d06c0 con 0x7efef4102a00 2024-04-22T13:24:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.659+0000 7efef9c27640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efee00a3bb0 0x7efee00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.659+0000 7efef9c27640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efee00a3bb0 0x7efee00a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efef4199620 tx=0x7efee4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.665+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efef009cef0 con 0x7efef4102a00 2024-04-22T13:24:59.694 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:24:59 smithi005 systemd[1]: Starting Ceph mds.g for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:24:59.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.898+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efef402d5c0 con 0x7efef4102a00 2024-04-22T13:24:59.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.899+0000 7efeeb7fe640 1 -- 172.21.15.5:0/213002496 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7efef00a11f0 con 0x7efef4102a00 2024-04-22T13:24:59.896 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:24:59.896 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T13:24:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efee00a3bb0 msgr2=0x7efee00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efee00a3bb0 0x7efee00a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efef4199620 tx=0x7efee4005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 msgr2=0x7efef4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 0x7efef4198110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efef000e9f0 tx=0x7efef000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 shutdown_connections 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efee00a3bb0 0x7efee00a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef4107c60 0x7efef419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef4105330 0x7efef4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 --2- 172.21.15.5:0/213002496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efef4102a00 0x7efef4198110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 >> 172.21.15.5:0/213002496 conn(0x7efef40fc3f0 msgr2=0x7efef4103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 shutdown_connections 2024-04-22T13:24:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:24:59.901+0000 7efefc68c640 1 -- 172.21.15.5:0/213002496 wait complete. 2024-04-22T13:25:00.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:24:59 smithi005 ceph-mon[28006]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:00.019 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:24:59 smithi005 podman[70005]: 2024-04-22 13:24:59.71795996 +0000 UTC m=+0.019035971 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:25:00.019 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:24:59 smithi005 podman[70005]: 2024-04-22 13:24:59.820873516 +0000 UTC m=+0.121949515 container create d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:25:00.019 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:25:00 smithi005 podman[70005]: 2024-04-22 13:25:00.023231045 +0000 UTC m=+0.324307044 container init d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416) 2024-04-22T13:25:00.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:24:59 smithi049 ceph-mon[31379]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:00.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:24:59 smithi107 ceph-mon[31859]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:00.309 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:25:00 smithi005 podman[70005]: 2024-04-22 13:25:00.026204769 +0000 UTC m=+0.327280769 container start d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-22T13:25:00.309 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mds[70026]: -- 172.21.15.5:0/1719827444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9684e7a40 con 0x55d9684bac00 2024-04-22T13:25:00.309 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g[70022]: starting mds.g at 2024-04-22T13:25:00.309 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:25:00 smithi005 bash[70005]: d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea 2024-04-22T13:25:00.309 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:25:00 smithi005 systemd[1]: Started Ceph mds.g for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/213002496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:00 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/213002496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:25:01.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:25:01.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:00 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/213002496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:25:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:00 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:01.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:01.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:01 smithi005 ceph-mon[28006]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:01.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:01 smithi005 ceph-mon[28006]: Deploying daemon mds.a on smithi005 2024-04-22T13:25:01.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:01 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6838/373669901,v1:172.21.15.5:6839/373669901] up:boot 2024-04-22T13:25:01.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:01 smithi005 ceph-mon[28006]: fsmap 10 up:standby 2024-04-22T13:25:01.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:01 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:25:02.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:01 smithi049 ceph-mon[31379]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:02.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:01 smithi049 ceph-mon[31379]: Deploying daemon mds.a on smithi005 2024-04-22T13:25:02.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:01 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6838/373669901,v1:172.21.15.5:6839/373669901] up:boot 2024-04-22T13:25:02.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:01 smithi049 ceph-mon[31379]: fsmap 10 up:standby 2024-04-22T13:25:02.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:01 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:25:02.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:01 smithi107 ceph-mon[31859]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:02.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:01 smithi107 ceph-mon[31859]: Deploying daemon mds.a on smithi005 2024-04-22T13:25:02.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:01 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6838/373669901,v1:172.21.15.5:6839/373669901] up:boot 2024-04-22T13:25:02.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:01 smithi107 ceph-mon[31859]: fsmap 10 up:standby 2024-04-22T13:25:02.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:01 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 -- 172.21.15.5:0/2523345314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 msgr2=0x7f7118107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 --2- 172.21.15.5:0/2523345314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118107780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7104009900 tx=0x7f710402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 -- 172.21.15.5:0/2523345314 shutdown_connections 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 --2- 172.21.15.5:0/2523345314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118107780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 --2- 172.21.15.5:0/2523345314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7118107cc0 0x7f711810a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 --2- 172.21.15.5:0/2523345314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7118102a60 0x7f7118104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 -- 172.21.15.5:0/2523345314 >> 172.21.15.5:0/2523345314 conn(0x7f71180fc3f0 msgr2=0x7f71180fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 -- 172.21.15.5:0/2523345314 shutdown_connections 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.080+0000 7f711edef640 1 -- 172.21.15.5:0/2523345314 wait complete. 2024-04-22T13:25:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 Processor -- start 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 -- start start 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7118102a60 0x7f71181981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7118107cc0 0x7f711819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f711810dab0 con 0x7f7118105390 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f711810d930 con 0x7f7118102a60 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711edef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f711810dc30 con 0x7f7118107cc0 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44762/0 (socket says 172.21.15.5:44762) 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 -- 172.21.15.5:0/1514104893 learned_addr learned my addr 172.21.15.5:0/1514104893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711d38c640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7118107cc0 0x7f711819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f711cb8b640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7118102a60 0x7f71181981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 -- 172.21.15.5:0/1514104893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7118107cc0 msgr2=0x7f711819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7118107cc0 0x7f711819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 -- 172.21.15.5:0/1514104893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7118102a60 msgr2=0x7f71181981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7118102a60 0x7f71181981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.081+0000 7f7117fff640 1 -- 172.21.15.5:0/1514104893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711819fde0 con 0x7f7118105390 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.082+0000 7f7117fff640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118198700 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7104000c00 tx=0x7f7104004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.082+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104043020 con 0x7f7118105390 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.082+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a0070 con 0x7f7118105390 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.082+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7104004aa0 con 0x7f7118105390 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.082+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104003d00 con 0x7f7118105390 2024-04-22T13:25:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.082+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a05c0 con 0x7f7118105390 2024-04-22T13:25:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.083+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71180fdee0 con 0x7f7118105390 2024-04-22T13:25:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.084+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f7104046020 con 0x7f7118105390 2024-04-22T13:25:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.085+0000 7f7115ffb640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71080a3bb0 0x7f71080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.085+0000 7f711cb8b640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71080a3bb0 0x7f71080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.085+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f71040f2310 con 0x7f7118105390 2024-04-22T13:25:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.085+0000 7f711cb8b640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71080a3bb0 0x7f71080a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f710c005fd0 tx=0x7f710c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:02.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.094+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f710404d020 con 0x7f7118105390 2024-04-22T13:25:02.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.328+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7118062b50 con 0x7f7118105390 2024-04-22T13:25:02.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.329+0000 7f7115ffb640 1 -- 172.21.15.5:0/1514104893 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8809 (secure 0 0 0) 0x7f71040bf0f0 con 0x7f7118105390 2024-04-22T13:25:02.326 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:02.326 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T13:25:02.326 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-04-22T13:25:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71080a3bb0 msgr2=0x7f71080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71080a3bb0 0x7f71080a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f710c005fd0 tx=0x7f710c00b390 comp rx=0 tx=0).stop 2024-04-22T13:25:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 msgr2=0x7f7118198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118198700 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7104000c00 tx=0x7f7104004270 comp rx=0 tx=0).stop 2024-04-22T13:25:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 shutdown_connections 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71080a3bb0 0x7f71080a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7118105390 0x7f7118198700 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7118107cc0 0x7f711819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 --2- 172.21.15.5:0/1514104893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7118102a60 0x7f71181981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 >> 172.21.15.5:0/1514104893 conn(0x7f71180fc3f0 msgr2=0x7f71181034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 shutdown_connections 2024-04-22T13:25:02.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:02.332+0000 7f711edef640 1 -- 172.21.15.5:0/1514104893 wait complete. 2024-04-22T13:25:02.478 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 podman[70363]: 2024-04-22 13:25:02.357915752 +0000 UTC m=+0.019254245 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:25:02.478 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 podman[70363]: 2024-04-22 13:25:02.482182972 +0000 UTC m=+0.143521485 container create f8402367009fcfd4298ad57d67bbba31a5df286e3a946bc7e4ec511a7e74dd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-22T13:25:02.740 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:02 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1514104893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:02.740 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 podman[70363]: 2024-04-22 13:25:02.718662242 +0000 UTC m=+0.380000730 container init f8402367009fcfd4298ad57d67bbba31a5df286e3a946bc7e4ec511a7e74dd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-22T13:25:02.740 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 podman[70363]: 2024-04-22 13:25:02.72159052 +0000 UTC m=+0.382928995 container start f8402367009fcfd4298ad57d67bbba31a5df286e3a946bc7e4ec511a7e74dd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:25:02.740 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 ceph-mds[70383]: -- 172.21.15.5:0/2261886746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2c0f45a40 con 0x55c2c0f19000 2024-04-22T13:25:03.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:02 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1514104893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:03.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:02 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1514104893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:03.058 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a[70379]: starting mds.a at 2024-04-22T13:25:03.058 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 bash[70363]: f8402367009fcfd4298ad57d67bbba31a5df286e3a946bc7e4ec511a7e74dd34 2024-04-22T13:25:03.058 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:25:02 smithi005 systemd[1]: Started Ceph mds.a for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:25:04.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:04.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:25:04.255 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:25:04.256 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:04.256 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: Deploying daemon mds.b on smithi049 2024-04-22T13:25:04.256 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6840/4083506255,v1:172.21.15.5:6841/4083506255] up:boot 2024-04-22T13:25:04.256 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: fsmap 11 up:standby 2024-04-22T13:25:04.256 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:03 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: Deploying daemon mds.b on smithi049 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6840/4083506255,v1:172.21.15.5:6841/4083506255] up:boot 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: fsmap 11 up:standby 2024-04-22T13:25:04.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:03 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:25:04.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: Deploying daemon mds.b on smithi049 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6840/4083506255,v1:172.21.15.5:6841/4083506255] up:boot 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: fsmap 11 up:standby 2024-04-22T13:25:04.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:03 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:25:04.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.462+0000 7fe163225640 1 -- 172.21.15.5:0/727180117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 msgr2=0x7fe15c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.462+0000 7fe163225640 1 --2- 172.21.15.5:0/727180117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c1092d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe154009ca0 tx=0x7fe15402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- 172.21.15.5:0/727180117 shutdown_connections 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 --2- 172.21.15.5:0/727180117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c1092d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 --2- 172.21.15.5:0/727180117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe15c101ff0 0x7fe15c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 --2- 172.21.15.5:0/727180117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c100ea0 0x7fe15c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- 172.21.15.5:0/727180117 >> 172.21.15.5:0/727180117 conn(0x7fe15c0fc410 msgr2=0x7fe15c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- 172.21.15.5:0/727180117 shutdown_connections 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- 172.21.15.5:0/727180117 wait complete. 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 Processor -- start 2024-04-22T13:25:04.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- start start 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe15c100ea0 0x7fe15c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c101ff0 0x7fe15c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10c9e0 con 0x7fe15c1029b0 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10c860 con 0x7fe15c100ea0 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.463+0000 7fe163225640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10cb60 con 0x7fe15c101ff0 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe160fc1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe15c100ea0 0x7fe15c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44786/0 (socket says 172.21.15.5:44786) 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 -- 172.21.15.5:0/2717698816 learned_addr learned my addr 172.21.15.5:0/2717698816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe15bfff640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c101ff0 0x7fe15c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 -- 172.21.15.5:0/2717698816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c101ff0 msgr2=0x7fe15c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c101ff0 0x7fe15c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 -- 172.21.15.5:0/2717698816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe15c100ea0 msgr2=0x7fe15c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe15c100ea0 0x7fe15c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 -- 172.21.15.5:0/2717698816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c19fd30 con 0x7fe15c1029b0 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe1617c2640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c19f6c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe15402c700 tx=0x7fe154034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154039040 con 0x7fe15c1029b0 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe154044070 con 0x7fe15c1029b0 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540042a0 con 0x7fe15c1029b0 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.464+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c19ffc0 con 0x7fe15c1029b0 2024-04-22T13:25:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.465+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a0420 con 0x7fe15c1029b0 2024-04-22T13:25:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.466+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c1a0740 con 0x7fe15c1029b0 2024-04-22T13:25:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.467+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe154003970 con 0x7fe15c1029b0 2024-04-22T13:25:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.467+0000 7fe159ffb640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe14c0a3bb0 0x7fe14c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.467+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fe1540f2d50 con 0x7fe15c1029b0 2024-04-22T13:25:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.467+0000 7fe160fc1640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe14c0a3bb0 0x7fe14c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.468+0000 7fe160fc1640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe14c0a3bb0 0x7fe14c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe15000bbe0 tx=0x7fe15000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:04.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.471+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1540bfbd0 con 0x7fe15c1029b0 2024-04-22T13:25:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.697+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe15c1a0a30 con 0x7fe15c1029b0 2024-04-22T13:25:04.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.698+0000 7fe159ffb640 1 -- 172.21.15.5:0/2717698816 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9632 (secure 0 0 0) 0x7fe1540bf4d0 con 0x7fe15c1029b0 2024-04-22T13:25:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:04.694 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T13:25:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-04-22T13:25:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe14c0a3bb0 msgr2=0x7fe14c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe14c0a3bb0 0x7fe14c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe15000bbe0 tx=0x7fe15000b390 comp rx=0 tx=0).stop 2024-04-22T13:25:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 msgr2=0x7fe15c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c19f6c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe15402c700 tx=0x7fe154034760 comp rx=0 tx=0).stop 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 shutdown_connections 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe14c0a3bb0 0x7fe14c0a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15c1029b0 0x7fe15c19f6c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c101ff0 0x7fe15c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 --2- 172.21.15.5:0/2717698816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe15c100ea0 0x7fe15c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 >> 172.21.15.5:0/2717698816 conn(0x7fe15c0fc410 msgr2=0x7fe15c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 shutdown_connections 2024-04-22T13:25:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:04.700+0000 7fe163225640 1 -- 172.21.15.5:0/2717698816 wait complete. 2024-04-22T13:25:04.921 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:04 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2717698816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:04 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2717698816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:05.271 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 systemd[1]: Starting Ceph mds.b for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:25:05.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:04 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2717698816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:05.561 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 podman[61328]: 2024-04-22 13:25:05.312159173 +0000 UTC m=+0.092801045 container create 6271c96a529e910575672b3f3ea56349ad8ddcee11091bf4365f3920659964ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T13:25:05.561 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 podman[61328]: 2024-04-22 13:25:05.237922181 +0000 UTC m=+0.018564050 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:25:05.561 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 podman[61328]: 2024-04-22 13:25:05.539517336 +0000 UTC m=+0.320159205 container init 6271c96a529e910575672b3f3ea56349ad8ddcee11091bf4365f3920659964ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-22T13:25:05.562 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 podman[61328]: 2024-04-22 13:25:05.542306716 +0000 UTC m=+0.322948592 container start 6271c96a529e910575672b3f3ea56349ad8ddcee11091bf4365f3920659964ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T13:25:05.837 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mds[61348]: -- 172.21.15.49:0/2870669674 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d77b4a5a40 con 0x55d77b478800 2024-04-22T13:25:05.837 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: starting mds.b at 2024-04-22T13:25:05.838 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 bash[61328]: 6271c96a529e910575672b3f3ea56349ad8ddcee11091bf4365f3920659964ff 2024-04-22T13:25:05.838 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:05 smithi049 systemd[1]: Started Ceph mds.b for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:25:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mon[31379]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:05 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:05 smithi107 ceph-mon[31859]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:06.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:05 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:05 smithi005 ceph-mon[28006]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:06.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:05 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:06.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 -- 172.21.15.5:0/2236976271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff804102aa0 msgr2=0x7ff804104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 --2- 172.21.15.5:0/2236976271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff804102aa0 0x7ff804104e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009900 tx=0x7ff7ec02bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 -- 172.21.15.5:0/2236976271 shutdown_connections 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 --2- 172.21.15.5:0/2236976271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804107d00 0x7ff80410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 --2- 172.21.15.5:0/2236976271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8041053d0 0x7ff8041077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 --2- 172.21.15.5:0/2236976271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff804102aa0 0x7ff804104e90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 -- 172.21.15.5:0/2236976271 >> 172.21.15.5:0/2236976271 conn(0x7ff8040fc3f0 msgr2=0x7ff8040fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 -- 172.21.15.5:0/2236976271 shutdown_connections 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.819+0000 7ff809f4a640 1 -- 172.21.15.5:0/2236976271 wait complete. 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 Processor -- start 2024-04-22T13:25:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 -- start start 2024-04-22T13:25:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff804102aa0 0x7ff80419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 0x7ff80419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804107d00 0x7ff8041a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff80410dae0 con 0x7ff8041053d0 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff80410d960 con 0x7ff804102aa0 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff809f4a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff80410dc60 con 0x7ff804107d00 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 0x7ff80419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 0x7ff80419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44810/0 (socket says 172.21.15.5:44810) 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 -- 172.21.15.5:0/1285278997 learned_addr learned my addr 172.21.15.5:0/1285278997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff803fff640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804107d00 0x7ff8041a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff8037fe640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff804102aa0 0x7ff80419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 -- 172.21.15.5:0/1285278997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804107d00 msgr2=0x7ff8041a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804107d00 0x7ff8041a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 -- 172.21.15.5:0/1285278997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff804102aa0 msgr2=0x7ff80419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff804102aa0 0x7ff80419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.820+0000 7ff802ffd640 1 -- 172.21.15.5:0/1285278997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8041a1f90 con 0x7ff8041053d0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.821+0000 7ff802ffd640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 0x7ff80419a8b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c970 tx=0x7ff7f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.821+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4014070 con 0x7ff8041053d0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.821+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041a2280 con 0x7ff8041053d0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.821+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f4004440 con 0x7ff8041053d0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.821+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4003e50 con 0x7ff8041053d0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.821+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041a27d0 con 0x7ff8041053d0 2024-04-22T13:25:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.822+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8041097f0 con 0x7ff8041053d0 2024-04-22T13:25:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.824+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff7f4017070 con 0x7ff8041053d0 2024-04-22T13:25:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.825+0000 7ff800ff9640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff7f00a3d50 0x7ff7f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.825+0000 7ff8037fe640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff7f00a3d50 0x7ff7f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.825+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7ff7f40cfaf0 con 0x7ff8041053d0 2024-04-22T13:25:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.825+0000 7ff8037fe640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff7f00a3d50 0x7ff7f00a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009230 tx=0x7ff7ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:06.830+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7f401e030 con 0x7ff8041053d0 2024-04-22T13:25:07.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:06 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] up:boot 2024-04-22T13:25:07.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:06 smithi049 ceph-mon[31379]: fsmap 12 up:standby 2024-04-22T13:25:07.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:06 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:25:07.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:06 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:07.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:06 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] up:boot 2024-04-22T13:25:07.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:06 smithi107 ceph-mon[31859]: fsmap 12 up:standby 2024-04-22T13:25:07.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:06 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:25:07.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:06 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:07.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.062+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80402d5c0 con 0x7ff8041053d0 2024-04-22T13:25:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:06 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] up:boot 2024-04-22T13:25:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:06 smithi005 ceph-mon[28006]: fsmap 12 up:standby 2024-04-22T13:25:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:06 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:25:07.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:06 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:07.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.063+0000 7ff800ff9640 1 -- 172.21.15.5:0/1285278997 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10458 (secure 0 0 0) 0x7ff7f409c8d0 con 0x7ff8041053d0 2024-04-22T13:25:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:07.059 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-22T13:25:07.060 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff7f00a3d50 msgr2=0x7ff7f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff7f00a3d50 0x7ff7f00a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009230 tx=0x7ff7ec005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 msgr2=0x7ff80419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 0x7ff80419a8b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c970 tx=0x7ff7f400ce30 comp rx=0 tx=0).stop 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 shutdown_connections 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff7f00a3d50 0x7ff7f00a6210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8041053d0 0x7ff80419a8b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff804107d00 0x7ff8041a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 --2- 172.21.15.5:0/1285278997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff804102aa0 0x7ff80419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 >> 172.21.15.5:0/1285278997 conn(0x7ff8040fc3f0 msgr2=0x7ff804102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 shutdown_connections 2024-04-22T13:25:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.065+0000 7ff809f4a640 1 -- 172.21.15.5:0/1285278997 wait complete. 2024-04-22T13:25:07.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:25:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 -- 172.21.15.5:0/63951363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 msgr2=0x7fdac0101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 --2- 172.21.15.5:0/63951363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac0101fd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009ca0 tx=0x7fdab802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 -- 172.21.15.5:0/63951363 shutdown_connections 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 --2- 172.21.15.5:0/63951363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac0101fd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 --2- 172.21.15.5:0/63951363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 0x7fdac0101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 --2- 172.21.15.5:0/63951363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0100850 0x7fdac0100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 -- 172.21.15.5:0/63951363 >> 172.21.15.5:0/63951363 conn(0x7fdac00fc3f0 msgr2=0x7fdac00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 -- 172.21.15.5:0/63951363 shutdown_connections 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.184+0000 7fdac85d5640 1 -- 172.21.15.5:0/63951363 wait complete. 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 Processor -- start 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 -- start start 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0100850 0x7fdac019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 0x7fdac019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac01a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010fcd0 con 0x7fdac0101b50 2024-04-22T13:25:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010fb50 con 0x7fdac01011b0 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac85d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010fe50 con 0x7fdac0100850 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac6b72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac01a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac6371640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0100850 0x7fdac019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac6b72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac01a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44832/0 (socket says 172.21.15.5:44832) 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac6b72640 1 -- 172.21.15.5:0/1526419773 learned_addr learned my addr 172.21.15.5:0/1526419773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.185+0000 7fdac5b70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 0x7fdac019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac6b72640 1 -- 172.21.15.5:0/1526419773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0100850 msgr2=0x7fdac019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac6b72640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0100850 0x7fdac019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac6b72640 1 -- 172.21.15.5:0/1526419773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 msgr2=0x7fdac019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac6b72640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 0x7fdac019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac6b72640 1 -- 172.21.15.5:0/1526419773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac01a1f10 con 0x7fdac0101b50 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac5b70640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 0x7fdac019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac6b72640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac01a18a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdab8032520 tx=0x7fdab80347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8039040 con 0x7fdac0101b50 2024-04-22T13:25:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab8044070 con 0x7fdac0101b50 2024-04-22T13:25:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac01a21a0 con 0x7fdac0101b50 2024-04-22T13:25:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80042a0 con 0x7fdac0101b50 2024-04-22T13:25:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.186+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac01a2690 con 0x7fdac0101b50 2024-04-22T13:25:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.187+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac010b750 con 0x7fdac0101b50 2024-04-22T13:25:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.189+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fdab8003970 con 0x7fdac0101b50 2024-04-22T13:25:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.189+0000 7fdab77fe640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdaac0a3bb0 0x7fdaac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.190+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fdab80f2890 con 0x7fdac0101b50 2024-04-22T13:25:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.190+0000 7fdac6371640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdaac0a3bb0 0x7fdaac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.190+0000 7fdac6371640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdaac0a3bb0 0x7fdaac0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdab000bbe0 tx=0x7fdab000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.193+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdab8035d30 con 0x7fdac0101b50 2024-04-22T13:25:07.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.419+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdac0062b50 con 0x7fdac0101b50 2024-04-22T13:25:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.421+0000 7fdab77fe640 1 -- 172.21.15.5:0/1526419773 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18352 (secure 0 0 0) 0x7fdab80bf670 con 0x7fdac0101b50 2024-04-22T13:25:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:07.417 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:24:23.973672+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:25:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdaac0a3bb0 msgr2=0x7fdaac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdaac0a3bb0 0x7fdaac0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdab000bbe0 tx=0x7fdab000b390 comp rx=0 tx=0).stop 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 msgr2=0x7fdac01a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac01a18a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdab8032520 tx=0x7fdab80347b0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 shutdown_connections 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdaac0a3bb0 0x7fdaac0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101b50 0x7fdac01a18a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdac01011b0 0x7fdac019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 --2- 172.21.15.5:0/1526419773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0100850 0x7fdac019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 >> 172.21.15.5:0/1526419773 conn(0x7fdac00fc3f0 msgr2=0x7fdac00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 shutdown_connections 2024-04-22T13:25:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.423+0000 7fdac85d5640 1 -- 172.21.15.5:0/1526419773 wait complete. 2024-04-22T13:25:07.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:25:07.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:25:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 -- 172.21.15.5:0/3173881300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814100e90 msgr2=0x7f58141012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 --2- 172.21.15.5:0/3173881300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814100e90 0x7f58141012b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5808009900 tx=0x7f580802bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 -- 172.21.15.5:0/3173881300 shutdown_connections 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 --2- 172.21.15.5:0/3173881300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58141029a0 0x7f58141092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 --2- 172.21.15.5:0/3173881300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814101fe0 0x7f5814102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 --2- 172.21.15.5:0/3173881300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814100e90 0x7f58141012b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 -- 172.21.15.5:0/3173881300 >> 172.21.15.5:0/3173881300 conn(0x7f58140fc420 msgr2=0x7f58140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 -- 172.21.15.5:0/3173881300 shutdown_connections 2024-04-22T13:25:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.565+0000 7f58199b2640 1 -- 172.21.15.5:0/3173881300 wait complete. 2024-04-22T13:25:07.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 Processor -- start 2024-04-22T13:25:07.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 -- start start 2024-04-22T13:25:07.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814100e90 0x7f5814069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 0x7f581419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58141029a0 0x7f581419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f581410ca60 con 0x7f5814101fe0 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f581410c8e0 con 0x7f58141029a0 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58199b2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f581410cbe0 con 0x7f5814100e90 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58127fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 0x7f581419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58127fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 0x7f581419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44852/0 (socket says 172.21.15.5:44852) 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58127fc640 1 -- 172.21.15.5:0/2064185086 learned_addr learned my addr 172.21.15.5:0/2064185086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814100e90 0x7f5814069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.566+0000 7f58137fe640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58141029a0 0x7f581419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f5812ffd640 1 -- 172.21.15.5:0/2064185086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58141029a0 msgr2=0x7f581419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f5812ffd640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58141029a0 0x7f581419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f5812ffd640 1 -- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 msgr2=0x7f581419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f5812ffd640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 0x7f581419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f5812ffd640 1 -- 172.21.15.5:0/2064185086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581419fea0 con 0x7f5814100e90 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f58127fc640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 0x7f581419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f5812ffd640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814100e90 0x7f5814069d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5808000c00 tx=0x7f5808004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808043020 con 0x7f5814100e90 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5808004440 con 0x7f5814100e90 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58141a0130 con 0x7f5814100e90 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808003c60 con 0x7f5814100e90 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.567+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a0700 con 0x7f5814100e90 2024-04-22T13:25:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.568+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814108310 con 0x7f5814100e90 2024-04-22T13:25:07.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.570+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5808046020 con 0x7f5814100e90 2024-04-22T13:25:07.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.570+0000 7f58189b0640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57fc0a3bb0 0x7f57fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.571+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f58080f26a0 con 0x7f5814100e90 2024-04-22T13:25:07.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.571+0000 7f58127fc640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57fc0a3bb0 0x7f57fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.571+0000 7f58127fc640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57fc0a3bb0 0x7f57fc0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5814069ae0 tx=0x7f5804005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.574+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f580804d020 con 0x7f5814100e90 2024-04-22T13:25:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.767+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f581402d750 con 0x7f5814100e90 2024-04-22T13:25:07.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.767+0000 7f58189b0640 1 -- 172.21.15.5:0/2064185086 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f58080bf520 con 0x7f5814100e90 2024-04-22T13:25:07.764 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57fc0a3bb0 msgr2=0x7f57fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57fc0a3bb0 0x7f57fc0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5814069ae0 tx=0x7f5804005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814100e90 msgr2=0x7f5814069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814100e90 0x7f5814069d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5808000c00 tx=0x7f5808004040 comp rx=0 tx=0).stop 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 shutdown_connections 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f57fc0a3bb0 0x7f57fc0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5814101fe0 0x7f581419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58141029a0 0x7f581419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 --2- 172.21.15.5:0/2064185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5814100e90 0x7f5814069d70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.769+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 >> 172.21.15.5:0/2064185086 conn(0x7f58140fc420 msgr2=0x7f58141054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.770+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 shutdown_connections 2024-04-22T13:25:07.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.770+0000 7f58199b2640 1 -- 172.21.15.5:0/2064185086 wait complete. 2024-04-22T13:25:07.777 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-22T13:25:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 -- 172.21.15.5:0/3696575384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22881053c0 msgr2=0x7f22881077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 --2- 172.21.15.5:0/3696575384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22881053c0 0x7f22881077b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2278009900 tx=0x7f227802bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 -- 172.21.15.5:0/3696575384 shutdown_connections 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 --2- 172.21.15.5:0/3696575384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22881053c0 0x7f22881077b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 --2- 172.21.15.5:0/3696575384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 0x7f228810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 --2- 172.21.15.5:0/3696575384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288102a90 0x7f2288104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 -- 172.21.15.5:0/3696575384 >> 172.21.15.5:0/3696575384 conn(0x7f22880fc420 msgr2=0x7f22880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 -- 172.21.15.5:0/3696575384 shutdown_connections 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.889+0000 7f228f3a4640 1 -- 172.21.15.5:0/3696575384 wait complete. 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 Processor -- start 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 -- start start 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 0x7f2288102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22881053c0 0x7f2288100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 0x7f2288100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f228810dab0 con 0x7f2288102a90 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f228810d930 con 0x7f2288107cf0 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228f3a4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f228810dc30 con 0x7f22881053c0 2024-04-22T13:25:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228d140640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 0x7f2288102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228d140640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 0x7f2288102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44868/0 (socket says 172.21.15.5:44868) 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228d140640 1 -- 172.21.15.5:0/1672315771 learned_addr learned my addr 172.21.15.5:0/1672315771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228c93f640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22881053c0 0x7f2288100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.890+0000 7f228d941640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 0x7f2288100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d140640 1 -- 172.21.15.5:0/1672315771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22881053c0 msgr2=0x7f2288100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d140640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22881053c0 0x7f2288100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d140640 1 -- 172.21.15.5:0/1672315771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 msgr2=0x7f2288100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d140640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 0x7f2288100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d140640 1 -- 172.21.15.5:0/1672315771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288101810 con 0x7f2288102a90 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228c93f640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22881053c0 0x7f2288100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d941640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 0x7f2288100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228d140640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 0x7f2288102400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9f0 tx=0x7f228400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228400cd80 con 0x7f2288102a90 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881a2300 con 0x7f2288102a90 2024-04-22T13:25:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a2870 con 0x7f2288102a90 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22840108e0 con 0x7f2288102a90 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.891+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284026060 con 0x7f2288102a90 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.892+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226c0052c0 con 0x7f2288102a90 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.894+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2284010430 con 0x7f2288102a90 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.895+0000 7f227e7fc640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22740a3bb0 0x7f22740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.895+0000 7f228c93f640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22740a3bb0 0x7f22740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.895+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f22840d0450 con 0x7f2288102a90 2024-04-22T13:25:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.895+0000 7f228c93f640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22740a3bb0 0x7f22740a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2278005ab0 tx=0x7f2278005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:07.900+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f228409cc80 con 0x7f2288102a90 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1285278997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1526419773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:07 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2064185086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:25:08.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:08.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1285278997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1526419773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:07 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2064185086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:25:08.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.102+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f226c005880 con 0x7f2288102a90 2024-04-22T13:25:08.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.102+0000 7f227e7fc640 1 -- 172.21.15.5:0/1672315771 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7f22840a0f80 con 0x7f2288102a90 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:25:08.099 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22740a3bb0 msgr2=0x7f22740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22740a3bb0 0x7f22740a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2278005ab0 tx=0x7f2278005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 msgr2=0x7f2288102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 0x7f2288102400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9f0 tx=0x7f228400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 shutdown_connections 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22740a3bb0 0x7f22740a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2288107cf0 0x7f2288100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22881053c0 0x7f2288100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 --2- 172.21.15.5:0/1672315771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2288102a90 0x7f2288102400 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.104+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 >> 172.21.15.5:0/1672315771 conn(0x7f22880fc420 msgr2=0x7f2288103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.105+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 shutdown_connections 2024-04-22T13:25:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.105+0000 7f228f3a4640 1 -- 172.21.15.5:0/1672315771 wait complete. 2024-04-22T13:25:08.112 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-22T13:25:08.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 -- 172.21.15.5:0/1894850525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c100e90 msgr2=0x7f903c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 --2- 172.21.15.5:0/1894850525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c100e90 0x7f903c1012b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9030009900 tx=0x7f903002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 -- 172.21.15.5:0/1894850525 shutdown_connections 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 --2- 172.21.15.5:0/1894850525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c1029a0 0x7f903c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 --2- 172.21.15.5:0/1894850525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f903c101fe0 0x7f903c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 --2- 172.21.15.5:0/1894850525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c100e90 0x7f903c1012b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 -- 172.21.15.5:0/1894850525 >> 172.21.15.5:0/1894850525 conn(0x7f903c0fc420 msgr2=0x7f903c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 -- 172.21.15.5:0/1894850525 shutdown_connections 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.224+0000 7f90410d5640 1 -- 172.21.15.5:0/1894850525 wait complete. 2024-04-22T13:25:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 Processor -- start 2024-04-22T13:25:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 -- start start 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c100e90 0x7f903c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c101fe0 0x7f903c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f903c1029a0 0x7f903c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10c9e0 con 0x7f903c101fe0 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10c860 con 0x7f903c100e90 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f90410d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10cb60 con 0x7f903c1029a0 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f903a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c101fe0 0x7f903c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f903a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c101fe0 0x7f903c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44874/0 (socket says 172.21.15.5:44874) 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f903a59c640 1 -- 172.21.15.5:0/4124072558 learned_addr learned my addr 172.21.15.5:0/4124072558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f903ad9d640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c100e90 0x7f903c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.225+0000 7f903b59e640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f903c1029a0 0x7f903c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f903ad9d640 1 -- 172.21.15.5:0/4124072558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f903c1029a0 msgr2=0x7f903c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f903ad9d640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f903c1029a0 0x7f903c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f903ad9d640 1 -- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c101fe0 msgr2=0x7f903c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f903ad9d640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c101fe0 0x7f903c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f903ad9d640 1 -- 172.21.15.5:0/4124072558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c19fe30 con 0x7f903c100e90 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f903ad9d640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c100e90 0x7f903c198210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9030000c00 tx=0x7f9030005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030043020 con 0x7f903c100e90 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9030004350 con 0x7f903c100e90 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c1a00c0 con 0x7f903c100e90 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030003b50 con 0x7f903c100e90 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.226+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c1a0610 con 0x7f903c100e90 2024-04-22T13:25:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.227+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903c1a0810 con 0x7f903c100e90 2024-04-22T13:25:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.228+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9030046020 con 0x7f903c100e90 2024-04-22T13:25:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.229+0000 7f9023fff640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f90240a3bb0 0x7f90240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.229+0000 7f903a59c640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f90240a3bb0 0x7f90240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.229+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f90300f1fb0 con 0x7f903c100e90 2024-04-22T13:25:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.229+0000 7f903a59c640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f90240a3bb0 0x7f90240a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f903c199720 tx=0x7f902c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.234+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f903004d020 con 0x7f903c100e90 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1285278997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1526419773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:08.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2064185086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:25:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.426+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f903c1a0b60 con 0x7f903c100e90 2024-04-22T13:25:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.860+0000 7f9023fff640 1 -- 172.21.15.5:0/4124072558 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7f90300bee30 con 0x7f903c100e90 2024-04-22T13:25:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f90240a3bb0 msgr2=0x7f90240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f90240a3bb0 0x7f90240a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f903c199720 tx=0x7f902c005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c100e90 msgr2=0x7f903c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c100e90 0x7f903c198210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9030000c00 tx=0x7f9030005dc0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 shutdown_connections 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f90240a3bb0 0x7f90240a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c101fe0 0x7f903c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f903c1029a0 0x7f903c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 --2- 172.21.15.5:0/4124072558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f903c100e90 0x7f903c198210 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 >> 172.21.15.5:0/4124072558 conn(0x7f903c0fc420 msgr2=0x7f903c069280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.864+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 shutdown_connections 2024-04-22T13:25:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.865+0000 7f90410d5640 1 -- 172.21.15.5:0/4124072558 wait complete. 2024-04-22T13:25:08.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-22T13:25:08.933 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:08 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:08.933 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:08 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1672315771' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T13:25:08.934 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:08 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4124072558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:25:08.934 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:08 smithi049 ceph-mon[31379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:25:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 -- 172.21.15.5:0/2245636652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134105340 msgr2=0x7f9134107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2245636652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134105340 0x7f9134107730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9124009900 tx=0x7f912402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 -- 172.21.15.5:0/2245636652 shutdown_connections 2024-04-22T13:25:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2245636652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134105340 0x7f9134107730 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2245636652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 0x7f913410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2245636652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9134102a10 0x7f9134104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.986+0000 7f913b3ce640 1 -- 172.21.15.5:0/2245636652 >> 172.21.15.5:0/2245636652 conn(0x7f91340fc420 msgr2=0x7f91340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 -- 172.21.15.5:0/2245636652 shutdown_connections 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 -- 172.21.15.5:0/2245636652 wait complete. 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 Processor -- start 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 -- start start 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 0x7f9134198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9134105340 0x7f9134198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 0x7f913419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f913410dab0 con 0x7f9134102a10 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f913410d930 con 0x7f9134105340 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913b3ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f913410dc30 con 0x7f9134107c70 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913916a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 0x7f9134198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913916a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 0x7f9134198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57888/0 (socket says 172.21.15.5:57888) 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913916a640 1 -- 172.21.15.5:0/2982771467 learned_addr learned my addr 172.21.15.5:0/2982771467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f9138969640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9134105340 0x7f9134198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.987+0000 7f913996b640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 0x7f913419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913916a640 1 -- 172.21.15.5:0/2982771467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 msgr2=0x7f913419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913916a640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 0x7f913419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913916a640 1 -- 172.21.15.5:0/2982771467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9134105340 msgr2=0x7f9134198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913916a640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9134105340 0x7f9134198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913916a640 1 -- 172.21.15.5:0/2982771467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913419fe20 con 0x7f9134102a10 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913996b640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 0x7f913419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913916a640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 0x7f9134198250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f913000e9f0 tx=0x7f913000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913000cd80 con 0x7f9134102a10 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91300108e0 con 0x7f9134102a10 2024-04-22T13:25:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91341a0110 con 0x7f9134102a10 2024-04-22T13:25:08.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130018980 con 0x7f9134102a10 2024-04-22T13:25:08.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.988+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91341a0630 con 0x7f9134102a10 2024-04-22T13:25:08.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.989+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9134069250 con 0x7f9134102a10 2024-04-22T13:25:08.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.990+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9130010430 con 0x7f9134102a10 2024-04-22T13:25:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.991+0000 7f912a7fc640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91200a3bb0 0x7f91200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.991+0000 7f9138969640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91200a3bb0 0x7f91200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.991+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7497+0+0 (secure 0 0 0) 0x7f91300d0960 con 0x7f9134102a10 2024-04-22T13:25:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.992+0000 7f9138969640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91200a3bb0 0x7f91200a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9124005ab0 tx=0x7f9124005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:08.995+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f913009d610 con 0x7f9134102a10 2024-04-22T13:25:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:08 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:08 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1672315771' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T13:25:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:08 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4124072558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:25:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:08 smithi107 ceph-mon[31859]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:25:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:08 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:08 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1672315771' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T13:25:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:08 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4124072558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:25:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:08 smithi005 ceph-mon[28006]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:25:09.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.187+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f91340fdf10 con 0x7f9134102a10 2024-04-22T13:25:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.858+0000 7f912a7fc640 1 -- 172.21.15.5:0/2982771467 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7f913009cf10 con 0x7f9134102a10 2024-04-22T13:25:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-22T13:25:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91200a3bb0 msgr2=0x7f91200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91200a3bb0 0x7f91200a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9124005ab0 tx=0x7f9124005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 msgr2=0x7f9134198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 0x7f9134198250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f913000e9f0 tx=0x7f913000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 shutdown_connections 2024-04-22T13:25:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91200a3bb0 0x7f91200a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9134107c70 0x7f913419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9134105340 0x7f9134198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 --2- 172.21.15.5:0/2982771467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9134102a10 0x7f9134198250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.861+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 >> 172.21.15.5:0/2982771467 conn(0x7f91340fc420 msgr2=0x7f91341034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.862+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 shutdown_connections 2024-04-22T13:25:09.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.862+0000 7f913b3ce640 1 -- 172.21.15.5:0/2982771467 wait complete. 2024-04-22T13:25:09.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 -- 172.21.15.5:0/2891748517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16281029b0 msgr2=0x7f16281092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 --2- 172.21.15.5:0/2891748517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16281029b0 0x7f16281092d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1620009ca0 tx=0x7f162002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 -- 172.21.15.5:0/2891748517 shutdown_connections 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 --2- 172.21.15.5:0/2891748517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16281029b0 0x7f16281092d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 --2- 172.21.15.5:0/2891748517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1628101ff0 0x7f1628102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 --2- 172.21.15.5:0/2891748517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1628100ea0 0x7f16281012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 -- 172.21.15.5:0/2891748517 >> 172.21.15.5:0/2891748517 conn(0x7f16280fc430 msgr2=0x7f16280fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 -- 172.21.15.5:0/2891748517 shutdown_connections 2024-04-22T13:25:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.987+0000 7f162ce06640 1 -- 172.21.15.5:0/2891748517 wait complete. 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 Processor -- start 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 -- start start 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 0x7f162806b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1628101ff0 0x7f162819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281029b0 0x7f162819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f162810ca00 con 0x7f1628100ea0 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f162810c880 con 0x7f1628101ff0 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162ce06640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f162810cb80 con 0x7f16281029b0 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 0x7f162806b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 0x7f162806b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57900/0 (socket says 172.21.15.5:57900) 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162659c640 1 -- 172.21.15.5:0/2923074410 learned_addr learned my addr 172.21.15.5:0/2923074410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:09.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f1626d9d640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281029b0 0x7f162819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162659c640 1 -- 172.21.15.5:0/2923074410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281029b0 msgr2=0x7f162819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162659c640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281029b0 0x7f162819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f162659c640 1 -- 172.21.15.5:0/2923074410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1628101ff0 msgr2=0x7f162819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.988+0000 7f1625d9b640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1628101ff0 0x7f162819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f162659c640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1628101ff0 0x7f162819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f162659c640 1 -- 172.21.15.5:0/2923074410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162819fd90 con 0x7f1628100ea0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f1626d9d640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281029b0 0x7f162819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f162659c640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 0x7f162806b1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f161000e9f0 tx=0x7f161000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161000cd80 con 0x7f1628100ea0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16281a0080 con 0x7f1628100ea0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16100054d0 con 0x7f1628100ea0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610010690 con 0x7f1628100ea0 2024-04-22T13:25:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.989+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16281a05d0 con 0x7f1628100ea0 2024-04-22T13:25:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.990+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1628069250 con 0x7f1628100ea0 2024-04-22T13:25:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.991+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f1610004e90 con 0x7f1628100ea0 2024-04-22T13:25:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.992+0000 7f161f7fe640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16140a3bb0 0x7f16140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.992+0000 7f1625d9b640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16140a3bb0 0x7f16140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.992+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f16100d0840 con 0x7f1628100ea0 2024-04-22T13:25:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.992+0000 7f1625d9b640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16140a3bb0 0x7f16140a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:09.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:09.997+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f161009d330 con 0x7f1628100ea0 2024-04-22T13:25:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T13:25:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2982771467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:09 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.225+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f16281081b0 con 0x7f1628100ea0 2024-04-22T13:25:10.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2982771467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:10.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:09 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:10.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T13:25:10.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T13:25:10.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2982771467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:25:10.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:10.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:09 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:25:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.887+0000 7f161f7fe640 1 -- 172.21.15.5:0/2923074410 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f161009cc30 con 0x7f1628100ea0 2024-04-22T13:25:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.893+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16140a3bb0 msgr2=0x7f16140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.893+0000 7f162ce06640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16140a3bb0 0x7f16140a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618005850 comp rx=0 tx=0).stop 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.893+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 msgr2=0x7f162806b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.893+0000 7f162ce06640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 0x7f162806b1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f161000e9f0 tx=0x7f161000edf0 comp rx=0 tx=0).stop 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 shutdown_connections 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16140a3bb0 0x7f16140a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281029b0 0x7f162819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1628101ff0 0x7f162819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 --2- 172.21.15.5:0/2923074410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1628100ea0 0x7f162806b1a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 >> 172.21.15.5:0/2923074410 conn(0x7f16280fc430 msgr2=0x7f1628105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 shutdown_connections 2024-04-22T13:25:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:10.894+0000 7f162ce06640 1 -- 172.21.15.5:0/2923074410 wait complete. 2024-04-22T13:25:10.914 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:25:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 -- 172.21.15.5:0/2984141052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64100810 msgr2=0x7f6c64102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 --2- 172.21.15.5:0/2984141052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64100810 0x7f6c64102c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009900 tx=0x7f6c5402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 -- 172.21.15.5:0/2984141052 shutdown_connections 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 --2- 172.21.15.5:0/2984141052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c64105b30 0x7f6c64105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 --2- 172.21.15.5:0/2984141052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64103180 0x7f6c641055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 --2- 172.21.15.5:0/2984141052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64100810 0x7f6c64102c40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 -- 172.21.15.5:0/2984141052 >> 172.21.15.5:0/2984141052 conn(0x7f6c640fc410 msgr2=0x7f6c640fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 -- 172.21.15.5:0/2984141052 shutdown_connections 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.025+0000 7f6c69b23640 1 -- 172.21.15.5:0/2984141052 wait complete. 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 Processor -- start 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 -- start start 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c64100810 0x7f6c6419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 0x7f6c6419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64105b30 0x7f6c641a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641140c0 con 0x7f6c64103180 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64113f40 con 0x7f6c64105b30 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c69b23640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64114240 con 0x7f6c64100810 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 0x7f6c6419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 0x7f6c6419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57922/0 (socket says 172.21.15.5:57922) 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 -- 172.21.15.5:0/3780058403 learned_addr learned my addr 172.21.15.5:0/3780058403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c637fe640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c64100810 0x7f6c6419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c63fff640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64105b30 0x7f6c641a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 -- 172.21.15.5:0/3780058403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c64100810 msgr2=0x7f6c6419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c64100810 0x7f6c6419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 -- 172.21.15.5:0/3780058403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64105b30 msgr2=0x7f6c641a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64105b30 0x7f6c641a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c62ffd640 1 -- 172.21.15.5:0/3780058403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c641a6380 con 0x7f6c64103180 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.026+0000 7f6c63fff640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64105b30 0x7f6c641a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.027+0000 7f6c62ffd640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 0x7f6c6419eca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c890 tx=0x7f6c5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.027+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004220 con 0x7f6c64103180 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.027+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c580043c0 con 0x7f6c64103180 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.027+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c641a6670 con 0x7f6c64103180 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.027+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004a00 con 0x7f6c64103180 2024-04-22T13:25:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.027+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a6c10 con 0x7f6c64103180 2024-04-22T13:25:11.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.028+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c640fdf00 con 0x7f6c64103180 2024-04-22T13:25:11.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.029+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f6c580079b0 con 0x7f6c64103180 2024-04-22T13:25:11.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.030+0000 7f6c60ff9640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6c4c0a3c30 0x7f6c4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.030+0000 7f6c637fe640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6c4c0a3c30 0x7f6c4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.030+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6c580d0010 con 0x7f6c64103180 2024-04-22T13:25:11.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.030+0000 7f6c637fe640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6c4c0a3c30 0x7f6c4c0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009230 tx=0x7f6c54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:11.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.034+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c58014030 con 0x7f6c64103180 2024-04-22T13:25:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.222+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c64062b50 con 0x7f6c64103180 2024-04-22T13:25:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.223+0000 7f6c60ff9640 1 -- 172.21.15.5:0/3780058403 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22128 (secure 0 0 0) 0x7f6c5809caa0 con 0x7f6c64103180 2024-04-22T13:25:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:11.220 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:25:10.226939+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6c4c0a3c30 msgr2=0x7f6c4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6c4c0a3c30 0x7f6c4c0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009230 tx=0x7f6c54005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 msgr2=0x7f6c6419eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 0x7f6c6419eca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c890 tx=0x7f6c5800cd50 comp rx=0 tx=0).stop 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 shutdown_connections 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6c4c0a3c30 0x7f6c4c0a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c64103180 0x7f6c6419eca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c64105b30 0x7f6c641a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 --2- 172.21.15.5:0/3780058403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c64100810 0x7f6c6419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.225+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 >> 172.21.15.5:0/3780058403 conn(0x7f6c640fc410 msgr2=0x7f6c64101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.226+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 shutdown_connections 2024-04-22T13:25:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.226+0000 7f6c69b23640 1 -- 172.21.15.5:0/3780058403 wait complete. 2024-04-22T13:25:11.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:10 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2982771467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T13:25:11.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:10 smithi005 ceph-mon[28006]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T13:25:11.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:10 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2923074410' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T13:25:11.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:10 smithi005 ceph-mon[28006]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T13:25:11.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:25:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:10 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2982771467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T13:25:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:10 smithi049 ceph-mon[31379]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T13:25:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:10 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2923074410' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T13:25:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:10 smithi049 ceph-mon[31379]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T13:25:11.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:10 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2982771467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T13:25:11.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:10 smithi107 ceph-mon[31859]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T13:25:11.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:10 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2923074410' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T13:25:11.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:10 smithi107 ceph-mon[31859]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T13:25:11.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/2066638056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac107d40 msgr2=0x7f8cac10a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/2066638056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac107d40 0x7f8cac10a130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009900 tx=0x7f8c9402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/2066638056 shutdown_connections 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/2066638056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac107d40 0x7f8cac10a130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/2066638056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cac105410 0x7f8cac107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/2066638056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac102ae0 0x7f8cac104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.338+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/2066638056 >> 172.21.15.5:0/2066638056 conn(0x7f8cac0fc410 msgr2=0x7f8cac0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/2066638056 shutdown_connections 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/2066638056 wait complete. 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 Processor -- start 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 -- start start 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 0x7f8cac1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac105410 0x7f8cac198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cac107d40 0x7f8cac19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10db70 con 0x7f8cac102ae0 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10d9f0 con 0x7f8cac107d40 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb2f2b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10dcf0 con 0x7f8cac105410 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb0cc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 0x7f8cac1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb0cc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 0x7f8cac1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57952/0 (socket says 172.21.15.5:57952) 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.339+0000 7f8cb0cc7640 1 -- 172.21.15.5:0/3453892239 learned_addr learned my addr 172.21.15.5:0/3453892239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb14c8640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cac107d40 0x7f8cac19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb0cc7640 1 -- 172.21.15.5:0/3453892239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac105410 msgr2=0x7f8cac198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb0cc7640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac105410 0x7f8cac198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb0cc7640 1 -- 172.21.15.5:0/3453892239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cac107d40 msgr2=0x7f8cac19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb0cc7640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cac107d40 0x7f8cac19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb0cc7640 1 -- 172.21.15.5:0/3453892239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac19fde0 con 0x7f8cac102ae0 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb0cc7640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 0x7f8cac1981c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c970 tx=0x7f8ca400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4007bb0 con 0x7f8cac102ae0 2024-04-22T13:25:11.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca4007d50 con 0x7f8cac102ae0 2024-04-22T13:25:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac1a00d0 con 0x7f8cac102ae0 2024-04-22T13:25:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400f450 con 0x7f8cac102ae0 2024-04-22T13:25:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.340+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1a0650 con 0x7f8cac102ae0 2024-04-22T13:25:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.341+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac0fdf00 con 0x7f8cac102ae0 2024-04-22T13:25:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.342+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8ca4004f20 con 0x7f8cac102ae0 2024-04-22T13:25:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.343+0000 7f8ca9ffb640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8c980a3bb0 0x7f8c980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.343+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8ca40d0170 con 0x7f8cac102ae0 2024-04-22T13:25:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.343+0000 7f8cabfff640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8c980a3bb0 0x7f8c980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.343+0000 7f8cabfff640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8c980a3bb0 0x7f8c980a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cac1996d0 tx=0x7f8c9c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.347+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ca4014030 con 0x7f8cac102ae0 2024-04-22T13:25:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.533+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cac062b80 con 0x7f8cac102ae0 2024-04-22T13:25:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.535+0000 7f8ca9ffb640 1 -- 172.21.15.5:0/3453892239 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22128 (secure 0 0 0) 0x7f8ca409cc00 con 0x7f8cac102ae0 2024-04-22T13:25:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:11.531 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:25:10.226939+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:25:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8c980a3bb0 msgr2=0x7f8c980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8c980a3bb0 0x7f8c980a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cac1996d0 tx=0x7f8c9c009250 comp rx=0 tx=0).stop 2024-04-22T13:25:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 msgr2=0x7f8cac1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 0x7f8cac1981c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c970 tx=0x7f8ca400ce30 comp rx=0 tx=0).stop 2024-04-22T13:25:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 shutdown_connections 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8c980a3bb0 0x7f8c980a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cac107d40 0x7f8cac19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac105410 0x7f8cac198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 --2- 172.21.15.5:0/3453892239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cac102ae0 0x7f8cac1981c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 >> 172.21.15.5:0/3453892239 conn(0x7f8cac0fc410 msgr2=0x7f8cac103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 shutdown_connections 2024-04-22T13:25:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.537+0000 7f8cb2f2b640 1 -- 172.21.15.5:0/3453892239 wait complete. 2024-04-22T13:25:11.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-22T13:25:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.652+0000 7f82d332c640 1 -- 172.21.15.5:0/3373207413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 msgr2=0x7f82cc10a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.652+0000 7f82d332c640 1 --2- 172.21.15.5:0/3373207413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc10a0d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- 172.21.15.5:0/3373207413 shutdown_connections 2024-04-22T13:25:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 --2- 172.21.15.5:0/3373207413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc10a0d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 --2- 172.21.15.5:0/3373207413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc105350 0x7f82cc107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 --2- 172.21.15.5:0/3373207413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82cc102a20 0x7f82cc104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- 172.21.15.5:0/3373207413 >> 172.21.15.5:0/3373207413 conn(0x7f82cc0fc430 msgr2=0x7f82cc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- 172.21.15.5:0/3373207413 shutdown_connections 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- 172.21.15.5:0/3373207413 wait complete. 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 Processor -- start 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- start start 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82cc102a20 0x7f82cc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc105350 0x7f82cc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10db00 con 0x7f82cc107c80 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10d980 con 0x7f82cc102a20 2024-04-22T13:25:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.653+0000 7f82d332c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10dc80 con 0x7f82cc105350 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57976/0 (socket says 172.21.15.5:57976) 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 -- 172.21.15.5:0/1329998630 learned_addr learned my addr 172.21.15.5:0/1329998630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d10c8640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82cc102a20 0x7f82cc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d08c7640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc105350 0x7f82cc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 -- 172.21.15.5:0/1329998630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc105350 msgr2=0x7f82cc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc105350 0x7f82cc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 -- 172.21.15.5:0/1329998630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82cc102a20 msgr2=0x7f82cc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82cc102a20 0x7f82cc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 -- 172.21.15.5:0/1329998630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc19fdc0 con 0x7f82cc107c80 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d18c9640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc19f750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4043020 con 0x7f82cc107c80 2024-04-22T13:25:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c4004aa0 con 0x7f82cc107c80 2024-04-22T13:25:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc1a0050 con 0x7f82cc107c80 2024-04-22T13:25:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4003c70 con 0x7f82cc107c80 2024-04-22T13:25:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.654+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc1a05f0 con 0x7f82cc107c80 2024-04-22T13:25:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.655+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82cc0fdf20 con 0x7f82cc107c80 2024-04-22T13:25:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.656+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f82c4046020 con 0x7f82cc107c80 2024-04-22T13:25:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.657+0000 7f82c27fc640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f82b80a3c30 0x7f82b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.657+0000 7f82d10c8640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f82b80a3c30 0x7f82b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.657+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f82c40f2960 con 0x7f82cc107c80 2024-04-22T13:25:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.658+0000 7f82d10c8640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f82b80a3c30 0x7f82b80a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005c40 tx=0x7f82c800a570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.661+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82c404d020 con 0x7f82cc107c80 2024-04-22T13:25:11.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:11.887+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f82cc02d5c0 con 0x7f82cc107c80 2024-04-22T13:25:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: pgmap v285: 97 pgs: 16 creating+peering, 1 active+clean, 80 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T13:25:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T13:25:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2923074410' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T13:25:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T13:25:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T13:25:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3780058403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3453892239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: pgmap v285: 97 pgs: 16 creating+peering, 1 active+clean, 80 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2923074410' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3780058403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3453892239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: pgmap v285: 97 pgs: 16 creating+peering, 1 active+clean, 80 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2923074410' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3780058403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3453892239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.899+0000 7f82c27fc640 1 -- 172.21.15.5:0/1329998630 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f82c40bf490 con 0x7f82cc107c80 2024-04-22T13:25:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.901+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f82b80a3c30 msgr2=0x7f82b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.901+0000 7f82d332c640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f82b80a3c30 0x7f82b80a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005c40 tx=0x7f82c800a570 comp rx=0 tx=0).stop 2024-04-22T13:25:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.901+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 msgr2=0x7f82cc19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.901+0000 7f82d332c640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc19f750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c40042f0 comp rx=0 tx=0).stop 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 shutdown_connections 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f82b80a3c30 0x7f82b80a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82cc107c80 0x7f82cc19f750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82cc105350 0x7f82cc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 --2- 172.21.15.5:0/1329998630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82cc102a20 0x7f82cc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 >> 172.21.15.5:0/1329998630 conn(0x7f82cc0fc430 msgr2=0x7f82cc103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 shutdown_connections 2024-04-22T13:25:12.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:12.902+0000 7f82d332c640 1 -- 172.21.15.5:0/1329998630 wait complete. 2024-04-22T13:25:12.912 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-22T13:25:12.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 -- 172.21.15.5:0/2104522102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0102a20 msgr2=0x7f53d0104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 --2- 172.21.15.5:0/2104522102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0102a20 0x7f53d0104e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009900 tx=0x7f53c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 -- 172.21.15.5:0/2104522102 shutdown_connections 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 --2- 172.21.15.5:0/2104522102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d0107c80 0x7f53d010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 --2- 172.21.15.5:0/2104522102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d0105350 0x7f53d0107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 --2- 172.21.15.5:0/2104522102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0102a20 0x7f53d0104e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 -- 172.21.15.5:0/2104522102 >> 172.21.15.5:0/2104522102 conn(0x7f53d00fc430 msgr2=0x7f53d00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 -- 172.21.15.5:0/2104522102 shutdown_connections 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.027+0000 7f53d523f640 1 -- 172.21.15.5:0/2104522102 wait complete. 2024-04-22T13:25:13.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 Processor -- start 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 -- start start 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d0102a20 0x7f53d0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d0105350 0x7f53d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 0x7f53d019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010dae0 con 0x7f53d0107c80 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010d960 con 0x7f53d0102a20 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53d523f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010dc60 con 0x7f53d0105350 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53cf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 0x7f53d019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53cf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 0x7f53d019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58000/0 (socket says 172.21.15.5:58000) 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53cf59e640 1 -- 172.21.15.5:0/2207009830 learned_addr learned my addr 172.21.15.5:0/2207009830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53ce59c640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d0105350 0x7f53d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.028+0000 7f53ced9d640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d0102a20 0x7f53d0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:13.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53cf59e640 1 -- 172.21.15.5:0/2207009830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d0105350 msgr2=0x7f53d0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53cf59e640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d0105350 0x7f53d0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53cf59e640 1 -- 172.21.15.5:0/2207009830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d0102a20 msgr2=0x7f53d0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53cf59e640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d0102a20 0x7f53d0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53cf59e640 1 -- 172.21.15.5:0/2207009830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d019fe20 con 0x7f53d0107c80 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53cf59e640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 0x7f53d019f7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53c800eaa0 tx=0x7f53c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c800cdb0 con 0x7f53d0107c80 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c8005020 con 0x7f53d0107c80 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d01a0110 con 0x7f53d0107c80 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c80116d0 con 0x7f53d0107c80 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.029+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01a0610 con 0x7f53d0107c80 2024-04-22T13:25:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.030+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d00fdf20 con 0x7f53d0107c80 2024-04-22T13:25:13.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.031+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f53c80051d0 con 0x7f53d0107c80 2024-04-22T13:25:13.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.032+0000 7f53b7fff640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53b80a3bb0 0x7f53b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:13.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.032+0000 7f53ced9d640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53b80a3bb0 0x7f53b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:13.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.032+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f53c80d04a0 con 0x7f53d0107c80 2024-04-22T13:25:13.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.032+0000 7f53ced9d640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53b80a3bb0 0x7f53b80a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009230 tx=0x7f53c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.037+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53c8016030 con 0x7f53d0107c80 2024-04-22T13:25:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.264+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f53d0062b50 con 0x7f53d0107c80 2024-04-22T13:25:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:12 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] up:active 2024-04-22T13:25:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:12 smithi049 ceph-mon[31379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:12 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1329998630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T13:25:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:12 smithi049 ceph-mon[31379]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T13:25:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:12 smithi049 ceph-mon[31379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:12 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] up:active 2024-04-22T13:25:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:12 smithi107 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:12 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1329998630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T13:25:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:12 smithi107 ceph-mon[31859]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T13:25:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:12 smithi107 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:12 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] up:active 2024-04-22T13:25:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:12 smithi005 ceph-mon[28006]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:12 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1329998630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T13:25:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:12 smithi005 ceph-mon[28006]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T13:25:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:12 smithi005 ceph-mon[28006]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.905+0000 7f53b7fff640 1 -- 172.21.15.5:0/2207009830 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f53c809c8e0 con 0x7f53d0107c80 2024-04-22T13:25:13.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53b80a3bb0 msgr2=0x7f53b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:13.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53b80a3bb0 0x7f53b80a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009230 tx=0x7f53c0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 msgr2=0x7f53d019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:13.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 0x7f53d019f7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53c800eaa0 tx=0x7f53c800ef60 comp rx=0 tx=0).stop 2024-04-22T13:25:13.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 shutdown_connections 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53b80a3bb0 0x7f53b80a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53d0107c80 0x7f53d019f7b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d0105350 0x7f53d0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 --2- 172.21.15.5:0/2207009830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53d0102a20 0x7f53d0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 >> 172.21.15.5:0/2207009830 conn(0x7f53d00fc430 msgr2=0x7f53d0103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 shutdown_connections 2024-04-22T13:25:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:13.908+0000 7f53d523f640 1 -- 172.21.15.5:0/2207009830 wait complete. 2024-04-22T13:25:13.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-22T13:25:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 -- 172.21.15.5:0/2591369616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c105350 msgr2=0x7f360c107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2591369616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c105350 0x7f360c107740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009900 tx=0x7f35f402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 -- 172.21.15.5:0/2591369616 shutdown_connections 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2591369616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c105350 0x7f360c107740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2591369616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f360c107c80 0x7f360c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2591369616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c102a20 0x7f360c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 -- 172.21.15.5:0/2591369616 >> 172.21.15.5:0/2591369616 conn(0x7f360c0fc430 msgr2=0x7f360c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 -- 172.21.15.5:0/2591369616 shutdown_connections 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.035+0000 7f3612b8f640 1 -- 172.21.15.5:0/2591369616 wait complete. 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 Processor -- start 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 -- start start 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 0x7f360c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f360c105350 0x7f360c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c107c80 0x7f360c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f360c10dba0 con 0x7f360c102a20 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f360c10da20 con 0x7f360c107c80 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f3612b8f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f360c10dd20 con 0x7f360c105350 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 0x7f360c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 0x7f360c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58024/0 (socket says 172.21.15.5:58024) 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 -- 172.21.15.5:0/2770296217 learned_addr learned my addr 172.21.15.5:0/2770296217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f360bfff640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f360c105350 0x7f360c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 -- 172.21.15.5:0/2770296217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f360c105350 msgr2=0x7f360c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f360c105350 0x7f360c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 -- 172.21.15.5:0/2770296217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c107c80 msgr2=0x7f360c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361112c640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c107c80 0x7f360c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c107c80 0x7f360c19f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.036+0000 7f361092b640 1 -- 172.21.15.5:0/2770296217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c19ff80 con 0x7f360c102a20 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.037+0000 7f361092b640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 0x7f360c198360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00b4a0 tx=0x7f35fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.037+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc014070 con 0x7f360c102a20 2024-04-22T13:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.037+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c1a0270 con 0x7f360c102a20 2024-04-22T13:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.037+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35fc010ac0 con 0x7f360c102a20 2024-04-22T13:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.037+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc017c30 con 0x7f360c102a20 2024-04-22T13:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.037+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c1a0860 con 0x7f360c102a20 2024-04-22T13:25:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.038+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360c069250 con 0x7f360c102a20 2024-04-22T13:25:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.039+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f35fc017070 con 0x7f360c102a20 2024-04-22T13:25:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.040+0000 7f3609ffb640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35f80a3bb0 0x7f35f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.040+0000 7f360bfff640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35f80a3bb0 0x7f35f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.040+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f35fc0cfa40 con 0x7f360c102a20 2024-04-22T13:25:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.040+0000 7f360bfff640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35f80a3bb0 0x7f35f80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009230 tx=0x7f35f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:14.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.044+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35fc09c4d0 con 0x7f360c102a20 2024-04-22T13:25:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.274+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f360c062b50 con 0x7f360c102a20 2024-04-22T13:25:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:13 smithi049 ceph-mon[31379]: pgmap v288: 97 pgs: 20 creating+peering, 19 active+clean, 58 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:13 smithi049 ceph-mon[31379]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T13:25:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:13 smithi049 ceph-mon[31379]: Cluster is now healthy 2024-04-22T13:25:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1329998630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T13:25:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:13 smithi049 ceph-mon[31379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2207009830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-22T13:25:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:13 smithi107 ceph-mon[31859]: pgmap v288: 97 pgs: 20 creating+peering, 19 active+clean, 58 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:13 smithi107 ceph-mon[31859]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T13:25:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:13 smithi107 ceph-mon[31859]: Cluster is now healthy 2024-04-22T13:25:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1329998630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T13:25:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:13 smithi107 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2207009830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-22T13:25:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:13 smithi005 ceph-mon[28006]: pgmap v288: 97 pgs: 20 creating+peering, 19 active+clean, 58 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:25:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:13 smithi005 ceph-mon[28006]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T13:25:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:13 smithi005 ceph-mon[28006]: Cluster is now healthy 2024-04-22T13:25:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1329998630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T13:25:14.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:13 smithi005 ceph-mon[28006]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:14.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2207009830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-22T13:25:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.914+0000 7f3609ffb640 1 -- 172.21.15.5:0/2770296217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f35fc09bdd0 con 0x7f360c102a20 2024-04-22T13:25:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35f80a3bb0 msgr2=0x7f35f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35f80a3bb0 0x7f35f80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009230 tx=0x7f35f4005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 msgr2=0x7f360c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 0x7f360c198360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00b4a0 tx=0x7f35fc00b960 comp rx=0 tx=0).stop 2024-04-22T13:25:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 shutdown_connections 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35f80a3bb0 0x7f35f80a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c107c80 0x7f360c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f360c105350 0x7f360c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 --2- 172.21.15.5:0/2770296217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360c102a20 0x7f360c198360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 >> 172.21.15.5:0/2770296217 conn(0x7f360c0fc430 msgr2=0x7f360c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 shutdown_connections 2024-04-22T13:25:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:14.917+0000 7f3612b8f640 1 -- 172.21.15.5:0/2770296217 wait complete. 2024-04-22T13:25:14.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 -- 172.21.15.5:0/1082505629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014100800 msgr2=0x7f0014100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 --2- 172.21.15.5:0/1082505629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014100800 0x7f0014100c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0010009900 tx=0x7f001002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 -- 172.21.15.5:0/1082505629 shutdown_connections 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 --2- 172.21.15.5:0/1082505629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0014101b00 0x7f0014101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 --2- 172.21.15.5:0/1082505629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014101160 0x7f00141015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 --2- 172.21.15.5:0/1082505629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014100800 0x7f0014100c20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.045+0000 7f001c006640 1 -- 172.21.15.5:0/1082505629 >> 172.21.15.5:0/1082505629 conn(0x7f00140fc420 msgr2=0x7f00140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 -- 172.21.15.5:0/1082505629 shutdown_connections 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 -- 172.21.15.5:0/1082505629 wait complete. 2024-04-22T13:25:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 Processor -- start 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 -- start start 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014100800 0x7f001419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0014101160 0x7f001419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 0x7f00141a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f001410fd30 con 0x7f0014101b00 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f001410fbb0 con 0x7f0014101160 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001c006640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f001410feb0 con 0x7f0014100800 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001a5a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 0x7f00141a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001a5a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 0x7f00141a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58042/0 (socket says 172.21.15.5:58042) 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f001a5a3640 1 -- 172.21.15.5:0/974837350 learned_addr learned my addr 172.21.15.5:0/974837350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.046+0000 7f00195a1640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0014101160 0x7f001419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f0019da2640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014100800 0x7f001419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001a5a3640 1 -- 172.21.15.5:0/974837350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014100800 msgr2=0x7f001419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001a5a3640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014100800 0x7f001419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001a5a3640 1 -- 172.21.15.5:0/974837350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0014101160 msgr2=0x7f001419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001a5a3640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0014101160 0x7f001419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001a5a3640 1 -- 172.21.15.5:0/974837350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00141a2040 con 0x7f0014101b00 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f0019da2640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014100800 0x7f001419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001a5a3640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 0x7f00141a19d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f000c00eb10 tx=0x7f000c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c004b20 con 0x7f0014101b00 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c004cc0 con 0x7f0014101b00 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00141a2330 con 0x7f0014101b00 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c010670 con 0x7f0014101b00 2024-04-22T13:25:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.047+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00141a2830 con 0x7f0014101b00 2024-04-22T13:25:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.048+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001410b550 con 0x7f0014101b00 2024-04-22T13:25:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.049+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f000c005020 con 0x7f0014101b00 2024-04-22T13:25:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.050+0000 7f000affd640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00000a3bb0 0x7f00000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.050+0000 7f0019da2640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00000a3bb0 0x7f00000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.050+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f000c0d09b0 con 0x7f0014101b00 2024-04-22T13:25:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.051+0000 7f0019da2640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00000a3bb0 0x7f00000a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0010006010 tx=0x7f001002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.055+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f000c09d440 con 0x7f0014101b00 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2207009830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T13:25:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:14 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2770296217' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T13:25:15.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.282+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f0014062b50 con 0x7f0014101b00 2024-04-22T13:25:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T13:25:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2207009830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-22T13:25:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T13:25:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-22T13:25:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T13:25:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2770296217' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T13:25:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T13:25:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2207009830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-22T13:25:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T13:25:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:25:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-22T13:25:15.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T13:25:15.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2770296217' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T13:25:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.920+0000 7f000affd640 1 -- 172.21.15.5:0/974837350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f000c09cd40 con 0x7f0014101b00 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00000a3bb0 msgr2=0x7f00000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00000a3bb0 0x7f00000a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0010006010 tx=0x7f001002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 msgr2=0x7f00141a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 0x7f00141a19d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f000c00eb10 tx=0x7f000c00c2d0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 shutdown_connections 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00000a3bb0 0x7f00000a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0014101b00 0x7f00141a19d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0014101160 0x7f001419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 --2- 172.21.15.5:0/974837350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0014100800 0x7f001419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 >> 172.21.15.5:0/974837350 conn(0x7f00140fc420 msgr2=0x7f00140fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 shutdown_connections 2024-04-22T13:25:15.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:15.924+0000 7f001c006640 1 -- 172.21.15.5:0/974837350 wait complete. 2024-04-22T13:25:15.938 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-22T13:25:15.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-22T13:25:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.052+0000 7efe68843640 1 -- 172.21.15.5:0/66679109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64100e90 msgr2=0x7efe641012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.052+0000 7efe68843640 1 --2- 172.21.15.5:0/66679109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64100e90 0x7efe641012b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009900 tx=0x7efe5c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- 172.21.15.5:0/66679109 shutdown_connections 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 --2- 172.21.15.5:0/66679109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe641029a0 0x7efe641092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 --2- 172.21.15.5:0/66679109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe64101fe0 0x7efe64102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 --2- 172.21.15.5:0/66679109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64100e90 0x7efe641012b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- 172.21.15.5:0/66679109 >> 172.21.15.5:0/66679109 conn(0x7efe640fc420 msgr2=0x7efe640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- 172.21.15.5:0/66679109 shutdown_connections 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- 172.21.15.5:0/66679109 wait complete. 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 Processor -- start 2024-04-22T13:25:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- start start 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 0x7efe64198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64101fe0 0x7efe641987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe641029a0 0x7efe6419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe6410c9e0 con 0x7efe64101fe0 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efe6410c860 con 0x7efe64100e90 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.053+0000 7efe68843640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe6410cb60 con 0x7efe641029a0 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64101fe0 0x7efe641987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 0x7efe64198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 0x7efe64198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59682/0 (socket says 172.21.15.5:59682) 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64101fe0 0x7efe641987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58066/0 (socket says 172.21.15.5:58066) 2024-04-22T13:25:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 -- 172.21.15.5:0/4045896599 learned_addr learned my addr 172.21.15.5:0/4045896599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe62d9d640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe641029a0 0x7efe6419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 -- 172.21.15.5:0/4045896599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe641029a0 msgr2=0x7efe6419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe641029a0 0x7efe6419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 -- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64101fe0 msgr2=0x7efe641987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64101fe0 0x7efe641987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 -- 172.21.15.5:0/4045896599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6419fe30 con 0x7efe64100e90 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe62d9d640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe641029a0 0x7efe6419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe6259c640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 0x7efe64198260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe5c006010 tx=0x7efe5c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c043020 con 0x7efe64100e90 2024-04-22T13:25:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641a00c0 con 0x7efe64100e90 2024-04-22T13:25:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5c004440 con 0x7efe64100e90 2024-04-22T13:25:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003bf0 con 0x7efe64100e90 2024-04-22T13:25:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.054+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641a0610 con 0x7efe64100e90 2024-04-22T13:25:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.055+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe641a0810 con 0x7efe64100e90 2024-04-22T13:25:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.056+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7efe5c046020 con 0x7efe64100e90 2024-04-22T13:25:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.057+0000 7efe5b7fe640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe500a3bb0 0x7efe500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.057+0000 7efe61d9b640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe500a3bb0 0x7efe500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.057+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7efe5c0f2c10 con 0x7efe64100e90 2024-04-22T13:25:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.058+0000 7efe61d9b640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe500a3bb0 0x7efe500a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe64199720 tx=0x7efe4c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.061+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe5c04d020 con 0x7efe64100e90 2024-04-22T13:25:16.271 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: 2024-04-22T13:25:15.850+0000 7f77e4cf6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T13:25:16.271 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: 2024-04-22T13:25:15.850+0000 7f77e4cf6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-22T13:25:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: pgmap v289: 97 pgs: 7 creating+peering, 88 active+clean, 2 unknown; 580 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-04-22T13:25:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2770296217' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T13:25:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] up:active 2024-04-22T13:25:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T13:25:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-22T13:25:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-22T13:25:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T13:25:16.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/974837350' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T13:25:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.276+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efe64069960 con 0x7efe500a3bb0 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: pgmap v289: 97 pgs: 7 creating+peering, 88 active+clean, 2 unknown; 580 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2770296217' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] up:active 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T13:25:16.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/974837350' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T13:25:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: pgmap v289: 97 pgs: 7 creating+peering, 88 active+clean, 2 unknown; 580 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2770296217' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] up:active 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T13:25:16.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/974837350' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T13:25:16.558 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:25:16.281+0000 7f31b1725700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:16.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:25:16.281+0000 7f31b1725700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:16.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:25:16.281+0000 7f31b1725700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:16.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:25:16.281+0000 7f31b1725700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:16.559 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:25:16.281+0000 7f31b1725700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.954+0000 7efe5b7fe640 1 -- 172.21.15.5:0/4045896599 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe64069960 con 0x7efe500a3bb0 2024-04-22T13:25:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe500a3bb0 msgr2=0x7efe500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe500a3bb0 0x7efe500a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe64199720 tx=0x7efe4c00a2d0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 msgr2=0x7efe64198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 0x7efe64198260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe5c006010 tx=0x7efe5c004040 comp rx=0 tx=0).stop 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 shutdown_connections 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe500a3bb0 0x7efe500a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe64101fe0 0x7efe641987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe641029a0 0x7efe6419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 --2- 172.21.15.5:0/4045896599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe64100e90 0x7efe64198260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 >> 172.21.15.5:0/4045896599 conn(0x7efe640fc420 msgr2=0x7efe64069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 shutdown_connections 2024-04-22T13:25:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:16.957+0000 7efe68843640 1 -- 172.21.15.5:0/4045896599 wait complete. 2024-04-22T13:25:16.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-22T13:25:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 -- 172.21.15.5:0/3589991679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4100810 msgr2=0x7fe5e4100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 --2- 172.21.15.5:0/3589991679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4100810 0x7fe5e4100c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009900 tx=0x7fe5d402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 -- 172.21.15.5:0/3589991679 shutdown_connections 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 --2- 172.21.15.5:0/3589991679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4101b10 0x7fe5e4101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 --2- 172.21.15.5:0/3589991679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e4101170 0x7fe5e41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 --2- 172.21.15.5:0/3589991679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4100810 0x7fe5e4100c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 -- 172.21.15.5:0/3589991679 >> 172.21.15.5:0/3589991679 conn(0x7fe5e40fc430 msgr2=0x7fe5e40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 -- 172.21.15.5:0/3589991679 shutdown_connections 2024-04-22T13:25:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.085+0000 7fe5e9152640 1 -- 172.21.15.5:0/3589991679 wait complete. 2024-04-22T13:25:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.086+0000 7fe5e9152640 1 Processor -- start 2024-04-22T13:25:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.086+0000 7fe5e9152640 1 -- start start 2024-04-22T13:25:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e9152640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4100810 0x7fe5e419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e9152640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 0x7fe5e419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e9152640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e4101b10 0x7fe5e41a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e9152640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410fcf0 con 0x7fe5e4100810 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e9152640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410fb70 con 0x7fe5e4101170 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e9152640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410fe70 con 0x7fe5e4101b10 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4100810 0x7fe5e419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 0x7fe5e419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4100810 0x7fe5e419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58108/0 (socket says 172.21.15.5:58108) 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 0x7fe5e419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:59726/0 (socket says 172.21.15.5:59726) 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e2d9d640 1 -- 172.21.15.5:0/4196619993 learned_addr learned my addr 172.21.15.5:0/4196619993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e359e640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e4101b10 0x7fe5e41a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 -- 172.21.15.5:0/4196619993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e4101b10 msgr2=0x7fe5e41a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e4101b10 0x7fe5e41a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 -- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4100810 msgr2=0x7fe5e419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4100810 0x7fe5e419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 -- 172.21.15.5:0/4196619993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e41a1fe0 con 0x7fe5e4101170 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5e259c640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 0x7fe5e419a900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800d6d0 tx=0x7fe5d800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.087+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8017070 con 0x7fe5e4101170 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.088+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d8009820 con 0x7fe5e4101170 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.088+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e41a22d0 con 0x7fe5e4101170 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.088+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8004c40 con 0x7fe5e4101170 2024-04-22T13:25:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.088+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a2870 con 0x7fe5e4101170 2024-04-22T13:25:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.089+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e410b560 con 0x7fe5e4101170 2024-04-22T13:25:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.090+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe5d8005220 con 0x7fe5e4101170 2024-04-22T13:25:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.091+0000 7fe5cbfff640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe5cc0a3c30 0x7fe5cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.091+0000 7fe5e2d9d640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe5cc0a3c30 0x7fe5cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.091+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fe5d80cfdf0 con 0x7fe5e4101170 2024-04-22T13:25:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.091+0000 7fe5e2d9d640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe5cc0a3c30 0x7fe5cc0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40096c0 tx=0x7fe5d402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.095+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5d809c880 con 0x7fe5e4101170 2024-04-22T13:25:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/974837350' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T13:25:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:16 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] up:active 2024-04-22T13:25:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:16 smithi049 ceph-mon[31379]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-22T13:25:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:16 smithi049 ceph-mon[31379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-22T13:25:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:16 smithi049 ceph-mon[31379]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-22T13:25:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:16 smithi049 ceph-mon[31379]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-22T13:25:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/974837350' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T13:25:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:16 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] up:active 2024-04-22T13:25:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:16 smithi107 ceph-mon[31859]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-22T13:25:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:16 smithi107 ceph-mon[31859]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-22T13:25:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:16 smithi107 ceph-mon[31859]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-22T13:25:17.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:16 smithi107 ceph-mon[31859]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-22T13:25:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/974837350' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T13:25:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] up:active 2024-04-22T13:25:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-mon[28006]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-22T13:25:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-mon[28006]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-22T13:25:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-mon[28006]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-22T13:25:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:16 smithi005 ceph-mon[28006]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-22T13:25:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.335+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe5e4062420 con 0x7fe5cc0a3c30 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.358+0000 7fe5cbfff640 1 -- 172.21.15.5:0/4196619993 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5e4062420 con 0x7fe5cc0a3c30 2024-04-22T13:25:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe5cc0a3c30 msgr2=0x7fe5cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe5cc0a3c30 0x7fe5cc0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40096c0 tx=0x7fe5d402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 msgr2=0x7fe5e419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 0x7fe5e419a900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800d6d0 tx=0x7fe5d800db90 comp rx=0 tx=0).stop 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 shutdown_connections 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe5cc0a3c30 0x7fe5cc0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e4101b10 0x7fe5e41a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5e4101170 0x7fe5e419a900 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 --2- 172.21.15.5:0/4196619993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4100810 0x7fe5e419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 >> 172.21.15.5:0/4196619993 conn(0x7fe5e40fc430 msgr2=0x7fe5e40ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 shutdown_connections 2024-04-22T13:25:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.360+0000 7fe5e9152640 1 -- 172.21.15.5:0/4196619993 wait complete. 2024-04-22T13:25:17.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-22T13:25:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 -- 172.21.15.5:0/1244740004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 msgr2=0x7f148410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 --2- 172.21.15.5:0/1244740004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148410a0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f147c009ca0 tx=0x7f147c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 -- 172.21.15.5:0/1244740004 shutdown_connections 2024-04-22T13:25:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 --2- 172.21.15.5:0/1244740004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148410a0f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 --2- 172.21.15.5:0/1244740004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484105370 0x7f1484107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 --2- 172.21.15.5:0/1244740004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484102a40 0x7f1484104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 -- 172.21.15.5:0/1244740004 >> 172.21.15.5:0/1244740004 conn(0x7f14840fc430 msgr2=0x7f14840fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 -- 172.21.15.5:0/1244740004 shutdown_connections 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.478+0000 7f148a332640 1 -- 172.21.15.5:0/1244740004 wait complete. 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 Processor -- start 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 -- start start 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484102a40 0x7f14841981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484105370 0x7f14841986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f148410db00 con 0x7f1484107ca0 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f148410d980 con 0x7f1484105370 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f148a332640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f148410dc80 con 0x7f1484102a40 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f14888cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f14888cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58116/0 (socket says 172.21.15.5:58116) 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f14888cf640 1 -- 172.21.15.5:0/3134914689 learned_addr learned my addr 172.21.15.5:0/3134914689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.479+0000 7f14837fe640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484105370 0x7f14841986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f1483fff640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484102a40 0x7f14841981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14888cf640 1 -- 172.21.15.5:0/3134914689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484102a40 msgr2=0x7f14841981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14888cf640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484102a40 0x7f14841981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14888cf640 1 -- 172.21.15.5:0/3134914689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484105370 msgr2=0x7f14841986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14888cf640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484105370 0x7f14841986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14888cf640 1 -- 172.21.15.5:0/3134914689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148419fdc0 con 0x7f1484107ca0 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f1483fff640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484102a40 0x7f14841981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14837fe640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484105370 0x7f14841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14888cf640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148419f750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f147c02c700 tx=0x7f147c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c041020 con 0x7f1484107ca0 2024-04-22T13:25:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841a0050 con 0x7f1484107ca0 2024-04-22T13:25:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c034d60 con 0x7f1484107ca0 2024-04-22T13:25:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c039040 con 0x7f1484107ca0 2024-04-22T13:25:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.480+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841a0540 con 0x7f1484107ca0 2024-04-22T13:25:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.481+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14840fdf20 con 0x7f1484107ca0 2024-04-22T13:25:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.482+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f147c044020 con 0x7f1484107ca0 2024-04-22T13:25:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.483+0000 7f14817fa640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f14700a3bb0 0x7f14700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.483+0000 7f1483fff640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f14700a3bb0 0x7f14700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.483+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f147c0f2690 con 0x7f1484107ca0 2024-04-22T13:25:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.484+0000 7f1483fff640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f14700a3bb0 0x7f14700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c20 tx=0x7f146c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.487+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f147c04b020 con 0x7f1484107ca0 2024-04-22T13:25:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.702+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1484062420 con 0x7f14700a3bb0 2024-04-22T13:25:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.707+0000 7f14817fa640 1 -- 172.21.15.5:0/3134914689 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1484062420 con 0x7f14700a3bb0 2024-04-22T13:25:17.704 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/13f308ef-5569-4598-b230-dfbceab5b293 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.709+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f14700a3bb0 msgr2=0x7f14700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.709+0000 7f148a332640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f14700a3bb0 0x7f14700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c20 tx=0x7f146c005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.709+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 msgr2=0x7f148419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.709+0000 7f148a332640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148419f750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f147c02c700 tx=0x7f147c0093f0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 shutdown_connections 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f14700a3bb0 0x7f14700a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484107ca0 0x7f148419f750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1484105370 0x7f14841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 --2- 172.21.15.5:0/3134914689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1484102a40 0x7f14841981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 >> 172.21.15.5:0/3134914689 conn(0x7f14840fc430 msgr2=0x7f14841033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 shutdown_connections 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.710+0000 7f148a332640 1 -- 172.21.15.5:0/3134914689 wait complete. 2024-04-22T13:25:17.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 -- 172.21.15.5:0/1564295251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 msgr2=0x7ff1501023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 --2- 172.21.15.5:0/1564295251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff1501023d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff140009ca0 tx=0x7ff14002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 -- 172.21.15.5:0/1564295251 shutdown_connections 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 --2- 172.21.15.5:0/1564295251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff150103b60 0x7ff15010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 --2- 172.21.15.5:0/1564295251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1501031a0 0x7ff150103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 --2- 172.21.15.5:0/1564295251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff1501023d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 -- 172.21.15.5:0/1564295251 >> 172.21.15.5:0/1564295251 conn(0x7ff1500fd520 msgr2=0x7ff1500ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 -- 172.21.15.5:0/1564295251 shutdown_connections 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.827+0000 7ff154da4640 1 -- 172.21.15.5:0/1564295251 wait complete. 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 Processor -- start 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 -- start start 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff150075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1501031a0 0x7ff150074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff150103b60 0x7ff150074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff15010daf0 con 0x7ff150101fb0 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff15010d970 con 0x7ff150103b60 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff154da4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff15010dc70 con 0x7ff1501031a0 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff14e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff150075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff14e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff150075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58142/0 (socket says 172.21.15.5:58142) 2024-04-22T13:25:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.828+0000 7ff14e59c640 1 -- 172.21.15.5:0/3497831521 learned_addr learned my addr 172.21.15.5:0/3497831521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14dd9b640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1501031a0 0x7ff150074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14ed9d640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff150103b60 0x7ff150074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14e59c640 1 -- 172.21.15.5:0/3497831521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1501031a0 msgr2=0x7ff150074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14e59c640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1501031a0 0x7ff150074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14e59c640 1 -- 172.21.15.5:0/3497831521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff150103b60 msgr2=0x7ff150074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14e59c640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff150103b60 0x7ff150074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14e59c640 1 -- 172.21.15.5:0/3497831521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff150075110 con 0x7ff150101fb0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff14e59c640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff150075d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff140000f80 tx=0x7ff1400346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140041020 con 0x7ff150101fb0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff140034ca0 con 0x7ff150101fb0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1501a22e0 con 0x7ff150101fb0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140039040 con 0x7ff150101fb0 2024-04-22T13:25:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.829+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1501a2880 con 0x7ff150101fb0 2024-04-22T13:25:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.830+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1501093a0 con 0x7ff150101fb0 2024-04-22T13:25:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.831+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff140044020 con 0x7ff150101fb0 2024-04-22T13:25:17.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.832+0000 7ff13f7fe640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff1380a3c30 0x7ff1380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.832+0000 7ff14dd9b640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff1380a3c30 0x7ff1380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.832+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7ff1400f2060 con 0x7ff150101fb0 2024-04-22T13:25:17.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.832+0000 7ff14dd9b640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff1380a3c30 0x7ff1380a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1440095f0 tx=0x7ff144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:17.836+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff14004b020 con 0x7ff150101fb0 2024-04-22T13:25:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.053+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff150062420 con 0x7ff1380a3c30 2024-04-22T13:25:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: pgmap v290: 97 pgs: 97 active+clean; 581 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 11 op/s 2024-04-22T13:25:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] up:active 2024-04-22T13:25:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: Cluster is now healthy 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] up:active 2024-04-22T13:25:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:17 smithi005 ceph-mon[28006]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:25:18.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.077+0000 7ff13f7fe640 1 -- 172.21.15.5:0/3497831521 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff150062420 con 0x7ff1380a3c30 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff1380a3c30 msgr2=0x7ff1380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff1380a3c30 0x7ff1380a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1440095f0 tx=0x7ff144005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 msgr2=0x7ff150075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff150075d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff140000f80 tx=0x7ff1400346f0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 shutdown_connections 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff1380a3c30 0x7ff1380a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff150103b60 0x7ff150074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1501031a0 0x7ff150074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 --2- 172.21.15.5:0/3497831521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff150101fb0 0x7ff150075d00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 >> 172.21.15.5:0/3497831521 conn(0x7ff1500fd520 msgr2=0x7ff150106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 shutdown_connections 2024-04-22T13:25:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.079+0000 7ff154da4640 1 -- 172.21.15.5:0/3497831521 wait complete. 2024-04-22T13:25:18.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.198+0000 7f168bd8f640 1 -- 172.21.15.5:0/2830674515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16841031a0 msgr2=0x7f1684103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.198+0000 7f168bd8f640 1 --2- 172.21.15.5:0/2830674515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16841031a0 0x7f1684103620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1674009900 tx=0x7f167402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- 172.21.15.5:0/2830674515 shutdown_connections 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 --2- 172.21.15.5:0/2830674515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16841031a0 0x7f1684103620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 --2- 172.21.15.5:0/2830674515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1684103b60 0x7f168410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 --2- 172.21.15.5:0/2830674515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1684101fb0 0x7f16841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- 172.21.15.5:0/2830674515 >> 172.21.15.5:0/2830674515 conn(0x7f16840fd520 msgr2=0x7f16840ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- 172.21.15.5:0/2830674515 shutdown_connections 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- 172.21.15.5:0/2830674515 wait complete. 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 Processor -- start 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- start start 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1684101fb0 0x7f1684075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16841031a0 0x7f1684074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 0x7f1684074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f168410daf0 con 0x7f1684103b60 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f168410d970 con 0x7f16841031a0 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168bd8f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f168410dc70 con 0x7f1684101fb0 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.199+0000 7f168a32c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 0x7f1684074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 0x7f1684074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58154/0 (socket says 172.21.15.5:58154) 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 -- 172.21.15.5:0/3059377730 learned_addr learned my addr 172.21.15.5:0/3059377730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f1689b2b640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1684101fb0 0x7f1684075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168932a640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16841031a0 0x7f1684074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 -- 172.21.15.5:0/3059377730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1684101fb0 msgr2=0x7f1684075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1684101fb0 0x7f1684075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 -- 172.21.15.5:0/3059377730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16841031a0 msgr2=0x7f1684074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16841031a0 0x7f1684074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 -- 172.21.15.5:0/3059377730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1684075110 con 0x7f1684103b60 2024-04-22T13:25:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168a32c640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 0x7f1684074880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f167c009fd0 tx=0x7f167c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c017070 con 0x7f1684103b60 2024-04-22T13:25:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c003e20 con 0x7f1684103b60 2024-04-22T13:25:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c0106c0 con 0x7f1684103b60 2024-04-22T13:25:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16841a22f0 con 0x7f1684103b60 2024-04-22T13:25:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.200+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16841a2840 con 0x7f1684103b60 2024-04-22T13:25:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.201+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1684109350 con 0x7f1684103b60 2024-04-22T13:25:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.202+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f167c004140 con 0x7f1684103b60 2024-04-22T13:25:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.203+0000 7f167affd640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16700a3c30 0x7f16700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.203+0000 7f1689b2b640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16700a3c30 0x7f16700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.203+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f167c0cff10 con 0x7f1684103b60 2024-04-22T13:25:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.204+0000 7f1689b2b640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16700a3c30 0x7f16700a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1684103000 tx=0x7f1680005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.208+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f167c0047e0 con 0x7f1684103b60 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: pgmap v290: 97 pgs: 97 active+clean; 581 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 11 op/s 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] up:active 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: Cluster is now healthy 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] up:active 2024-04-22T13:25:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:17 smithi049 ceph-mon[31379]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:25:18.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: pgmap v290: 97 pgs: 97 active+clean; 581 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 11 op/s 2024-04-22T13:25:18.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] up:active 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: Cluster is now healthy 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] up:active 2024-04-22T13:25:18.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:17 smithi107 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:25:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.425+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1684062420 con 0x7f16700a3c30 2024-04-22T13:25:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.430+0000 7f167affd640 1 -- 172.21.15.5:0/3059377730 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1684062420 con 0x7f16700a3c30 2024-04-22T13:25:18.427 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/9f193bc0-536a-473d-81e1-fda6e7626db4 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16700a3c30 msgr2=0x7f16700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16700a3c30 0x7f16700a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1684103000 tx=0x7f1680005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 msgr2=0x7f1684074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 0x7f1684074880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f167c009fd0 tx=0x7f167c00b260 comp rx=0 tx=0).stop 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 shutdown_connections 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f16700a3c30 0x7f16700a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1684103b60 0x7f1684074880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16841031a0 0x7f1684074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 --2- 172.21.15.5:0/3059377730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1684101fb0 0x7f1684075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 >> 172.21.15.5:0/3059377730 conn(0x7f16840fd520 msgr2=0x7f1684106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 shutdown_connections 2024-04-22T13:25:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.432+0000 7f168bd8f640 1 -- 172.21.15.5:0/3059377730 wait complete. 2024-04-22T13:25:18.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 -- 172.21.15.5:0/463577539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4102a00 msgr2=0x7fc0a4104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 --2- 172.21.15.5:0/463577539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4102a00 0x7fc0a4104df0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009900 tx=0x7fc09c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 -- 172.21.15.5:0/463577539 shutdown_connections 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 --2- 172.21.15.5:0/463577539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4107c60 0x7fc0a410a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 --2- 172.21.15.5:0/463577539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0a4105330 0x7fc0a4107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 --2- 172.21.15.5:0/463577539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4102a00 0x7fc0a4104df0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 -- 172.21.15.5:0/463577539 >> 172.21.15.5:0/463577539 conn(0x7fc0a40fc3f0 msgr2=0x7fc0a40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.544+0000 7fc0ab785640 1 -- 172.21.15.5:0/463577539 shutdown_connections 2024-04-22T13:25:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 -- 172.21.15.5:0/463577539 wait complete. 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 Processor -- start 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 -- start start 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0a4102a00 0x7fc0a4102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 0x7fc0a41009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4107c60 0x7fc0a4100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a410dad0 con 0x7fc0a4105330 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a410d950 con 0x7fc0a4102a00 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0ab785640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a410dc50 con 0x7fc0a4107c60 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0a8d20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 0x7fc0a41009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0a8d20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 0x7fc0a41009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58164/0 (socket says 172.21.15.5:58164) 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0a8d20640 1 -- 172.21.15.5:0/2496283014 learned_addr learned my addr 172.21.15.5:0/2496283014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0a9521640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0a4102a00 0x7fc0a4102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.545+0000 7fc0a9d22640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4107c60 0x7fc0a4100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a8d20640 1 -- 172.21.15.5:0/2496283014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4107c60 msgr2=0x7fc0a4100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a8d20640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4107c60 0x7fc0a4100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a8d20640 1 -- 172.21.15.5:0/2496283014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0a4102a00 msgr2=0x7fc0a4102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a8d20640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0a4102a00 0x7fc0a4102370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a8d20640 1 -- 172.21.15.5:0/2496283014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a4101780 con 0x7fc0a4105330 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a8d20640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 0x7fc0a41009b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc098005fd0 tx=0x7fc09800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098017070 con 0x7fc0a4105330 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc098009c30 con 0x7fc0a4105330 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a41a22c0 con 0x7fc0a4105330 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098003d30 con 0x7fc0a4105330 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.546+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a41a2860 con 0x7fc0a4105330 2024-04-22T13:25:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.547+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a40fdee0 con 0x7fc0a4105330 2024-04-22T13:25:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.548+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fc098003ed0 con 0x7fc0a4105330 2024-04-22T13:25:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.549+0000 7fc0a27fc640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc08c0a3c30 0x7fc08c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.549+0000 7fc0a9521640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc08c0a3c30 0x7fc08c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.549+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fc0980d0780 con 0x7fc0a4105330 2024-04-22T13:25:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.549+0000 7fc0a9521640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc08c0a3c30 0x7fc08c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009230 tx=0x7fc09c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.552+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc098020030 con 0x7fc0a4105330 2024-04-22T13:25:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.781+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0a402d810 con 0x7fc0a4105330 2024-04-22T13:25:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.782+0000 7fc0a27fc640 1 -- 172.21.15.5:0/2496283014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11905 (secure 0 0 0) 0x7fc098009de0 con 0x7fc0a4105330 2024-04-22T13:25:18.779 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:25:17.229740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24485,24469,24496,24557,24512]},"id":1}]} 2024-04-22T13:25:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:25:18.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc08c0a3c30 msgr2=0x7fc08c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc08c0a3c30 0x7fc08c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009230 tx=0x7fc09c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 msgr2=0x7fc0a41009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 0x7fc0a41009b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc098005fd0 tx=0x7fc09800dde0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 shutdown_connections 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc08c0a3c30 0x7fc08c0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0a4105330 0x7fc0a41009b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4107c60 0x7fc0a4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 --2- 172.21.15.5:0/2496283014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0a4102a00 0x7fc0a4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 >> 172.21.15.5:0/2496283014 conn(0x7fc0a40fc3f0 msgr2=0x7fc0a41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 shutdown_connections 2024-04-22T13:25:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.784+0000 7fc0ab785640 1 -- 172.21.15.5:0/2496283014 wait complete. 2024-04-22T13:25:18.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 -- 172.21.15.5:0/2447761320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0101fd0 msgr2=0x7facd0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 --2- 172.21.15.5:0/2447761320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0101fd0 0x7facd0102450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7facc0009900 tx=0x7facc002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 -- 172.21.15.5:0/2447761320 shutdown_connections 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 --2- 172.21.15.5:0/2447761320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facd0102990 0x7facd01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 --2- 172.21.15.5:0/2447761320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0101fd0 0x7facd0102450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 --2- 172.21.15.5:0/2447761320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facd0100e80 0x7facd01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.897+0000 7facd738e640 1 -- 172.21.15.5:0/2447761320 >> 172.21.15.5:0/2447761320 conn(0x7facd00fc3f0 msgr2=0x7facd00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 -- 172.21.15.5:0/2447761320 shutdown_connections 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 -- 172.21.15.5:0/2447761320 wait complete. 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 Processor -- start 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 -- start start 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 0x7facd019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facd0101fd0 0x7facd019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facd0102990 0x7facd01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facd010c9e0 con 0x7facd0102990 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7facd010c860 con 0x7facd0100e80 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.898+0000 7facd738e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7facd010cb60 con 0x7facd0101fd0 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 0x7facd019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 0x7facd019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32838/0 (socket says 172.21.15.5:32838) 2024-04-22T13:25:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd4929640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facd0101fd0 0x7facd019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 -- 172.21.15.5:0/1118967833 learned_addr learned my addr 172.21.15.5:0/1118967833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:18.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd592b640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facd0102990 0x7facd01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 -- 172.21.15.5:0/1118967833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facd0101fd0 msgr2=0x7facd019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facd0101fd0 0x7facd019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 -- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facd0102990 msgr2=0x7facd01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facd0102990 0x7facd01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 -- 172.21.15.5:0/1118967833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd01a20d0 con 0x7facd0100e80 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.899+0000 7facd512a640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 0x7facd019a500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faccc00e9f0 tx=0x7faccc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.900+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc00ccc0 con 0x7facd0100e80 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.900+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faccc00ce60 con 0x7facd0100e80 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.900+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd01a23c0 con 0x7facd0100e80 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.900+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc010430 con 0x7facd0100e80 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.900+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd01a2890 con 0x7facd0100e80 2024-04-22T13:25:18.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.901+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd01a2b50 con 0x7facd0100e80 2024-04-22T13:25:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.902+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7faccc0046c0 con 0x7facd0100e80 2024-04-22T13:25:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.903+0000 7facc67fc640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7facbc0a3bb0 0x7facbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.903+0000 7facd4929640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7facbc0a3bb0 0x7facbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.903+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7faccc0d0900 con 0x7facd0100e80 2024-04-22T13:25:18.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.904+0000 7facd4929640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7facbc0a3bb0 0x7facbc0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facc0009230 tx=0x7facc002c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.908+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faccc053ff0 con 0x7facd0100e80 2024-04-22T13:25:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:18.948+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7faccc099300 con 0x7facd0100e80 2024-04-22T13:25:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:18 smithi005 ceph-mon[28006]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:18 smithi005 ceph-mon[28006]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:18 smithi005 ceph-mon[28006]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:18 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2496283014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.102+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facd01a2d20 con 0x7facd0100e80 2024-04-22T13:25:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.105+0000 7facc67fc640 1 -- 172.21.15.5:0/1118967833 <== mon.2 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22115 (secure 0 0 0) 0x7faccc0564c0 con 0x7facd0100e80 2024-04-22T13:25:19.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:19.102 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:25:11.887809+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:25:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7facbc0a3bb0 msgr2=0x7facbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7facbc0a3bb0 0x7facbc0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facc0009230 tx=0x7facc002c740 comp rx=0 tx=0).stop 2024-04-22T13:25:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 msgr2=0x7facd019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 0x7facd019a500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faccc00e9f0 tx=0x7faccc00edf0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 shutdown_connections 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7facbc0a3bb0 0x7facbc0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facd0102990 0x7facd01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facd0101fd0 0x7facd019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 --2- 172.21.15.5:0/1118967833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facd0100e80 0x7facd019a500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 >> 172.21.15.5:0/1118967833 conn(0x7facd00fc3f0 msgr2=0x7facd00692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 shutdown_connections 2024-04-22T13:25:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.107+0000 7facd738e640 1 -- 172.21.15.5:0/1118967833 wait complete. 2024-04-22T13:25:19.115 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-22T13:25:19.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:25:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 -- 172.21.15.5:0/2468708955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170101fa0 msgr2=0x7f51701023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 --2- 172.21.15.5:0/2468708955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170101fa0 0x7f51701023c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5164009900 tx=0x7f516402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 -- 172.21.15.5:0/2468708955 shutdown_connections 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 --2- 172.21.15.5:0/2468708955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5170103b50 0x7f517010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 --2- 172.21.15.5:0/2468708955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5170103190 0x7f5170103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 --2- 172.21.15.5:0/2468708955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170101fa0 0x7f51701023c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 -- 172.21.15.5:0/2468708955 >> 172.21.15.5:0/2468708955 conn(0x7f51700fd570 msgr2=0x7f51700ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 -- 172.21.15.5:0/2468708955 shutdown_connections 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.220+0000 7f5176aa7640 1 -- 172.21.15.5:0/2468708955 wait complete. 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 Processor -- start 2024-04-22T13:25:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 -- start start 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5170101fa0 0x7f51701981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5170103190 0x7f5170198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 0x7f517019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f517010db30 con 0x7f5170103b50 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f517010d9b0 con 0x7f5170101fa0 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5176aa7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f517010dcb0 con 0x7f5170103190 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 0x7f517019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 0x7f517019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51830/0 (socket says 172.21.15.5:51830) 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 -- 172.21.15.5:0/1798148490 learned_addr learned my addr 172.21.15.5:0/1798148490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5174843640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5170101fa0 0x7f51701981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f516ffff640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5170103190 0x7f5170198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 -- 172.21.15.5:0/1798148490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5170103190 msgr2=0x7f5170198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5170103190 0x7f5170198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 -- 172.21.15.5:0/1798148490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5170101fa0 msgr2=0x7f51701981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5170101fa0 0x7f51701981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 -- 172.21.15.5:0/1798148490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517019fdf0 con 0x7f5170103b50 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.221+0000 7f5175044640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 0x7f517019f780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f516800cad0 tx=0x7f516800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.222+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800ae20 con 0x7f5170103b50 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.222+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5168005020 con 0x7f5170103b50 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.222+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51701a00e0 con 0x7f5170103b50 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.222+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800f6b0 con 0x7f5170103b50 2024-04-22T13:25:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.222+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a0630 con 0x7f5170103b50 2024-04-22T13:25:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.223+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51701092e0 con 0x7f5170103b50 2024-04-22T13:25:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.224+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f51680051d0 con 0x7f5170103b50 2024-04-22T13:25:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.224+0000 7f516dffb640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51600a3cc0 0x7f51600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.224+0000 7f5174843640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51600a3cc0 0x7f51600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.224+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f51680d0420 con 0x7f5170103b50 2024-04-22T13:25:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.225+0000 7f5174843640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51600a3cc0 0x7f51600a6180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f516402c240 tx=0x7f516402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.229+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5168014030 con 0x7f5170103b50 2024-04-22T13:25:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:18 smithi049 ceph-mon[31379]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:18 smithi049 ceph-mon[31379]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:18 smithi049 ceph-mon[31379]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:18 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2496283014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:18 smithi107 ceph-mon[31859]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:18 smithi107 ceph-mon[31859]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:18 smithi107 ceph-mon[31859]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:19.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:18 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2496283014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.411+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5170062390 con 0x7f51600a3cc0 2024-04-22T13:25:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.419+0000 7f516dffb640 1 -- 172.21.15.5:0/1798148490 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258728 (secure 0 0 0) 0x7f5170062390 con 0x7f51600a3cc0 2024-04-22T13:25:19.419 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:19.420 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:25:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51600a3cc0 msgr2=0x7f51600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51600a3cc0 0x7f51600a6180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f516402c240 tx=0x7f516402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 msgr2=0x7f517019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 0x7f517019f780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f516800cad0 tx=0x7f516800cf90 comp rx=0 tx=0).stop 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 shutdown_connections 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51600a3cc0 0x7f51600a6180 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5170103b50 0x7f517019f780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5170103190 0x7f5170198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 --2- 172.21.15.5:0/1798148490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5170101fa0 0x7f51701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 >> 172.21.15.5:0/1798148490 conn(0x7f51700fd570 msgr2=0x7f51701064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 shutdown_connections 2024-04-22T13:25:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.422+0000 7f5176aa7640 1 -- 172.21.15.5:0/1798148490 wait complete. 2024-04-22T13:25:19.429 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-04-22T13:25:18.179601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594856,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":140,"num_read_kb":123,"num_write":289,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399780,"kb_used_data":13144,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542044,"statfs":{"total":1151940427776,"available":1151531053056,"internally_reserved":0,"allocated":13459456,"data_stored":7260282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4488,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":108,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001595"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789603+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:25:13.789603+0000","last_peered":"2024-04-22T13:25:13.789603+0000","last_clean":"2024-04-22T13:25:13.789603+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:25:13.789603+0000","last_undegraded":"2024-04-22T13:25:13.789603+0000","last_fullsized":"2024-04-22T13:25:13.789603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375819+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:25:12.375819+0000","last_peered":"2024-04-22T13:25:12.375819+0000","last_clean":"2024-04-22T13:25:12.375819+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:25:12.375819+0000","last_undegraded":"2024-04-22T13:25:12.375819+0000","last_fullsized":"2024-04-22T13:25:12.375819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375616+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:25:12.375616+0000","last_peered":"2024-04-22T13:25:12.375616+0000","last_clean":"2024-04-22T13:25:12.375616+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:25:12.375616+0000","last_undegraded":"2024-04-22T13:25:12.375616+0000","last_fullsized":"2024-04-22T13:25:12.375616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376632+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:25:12.376632+0000","last_peered":"2024-04-22T13:25:12.376632+0000","last_clean":"2024-04-22T13:25:12.376632+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:25:12.376632+0000","last_undegraded":"2024-04-22T13:25:12.376632+0000","last_fullsized":"2024-04-22T13:25:12.376632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899988+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:25:11.899988+0000","last_peered":"2024-04-22T13:25:11.899988+0000","last_clean":"2024-04-22T13:25:11.899988+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:25:11.899988+0000","last_undegraded":"2024-04-22T13:25:11.899988+0000","last_fullsized":"2024-04-22T13:25:11.899988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377789+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:25:12.377789+0000","last_peered":"2024-04-22T13:25:12.377789+0000","last_clean":"2024-04-22T13:25:12.377789+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:25:12.377789+0000","last_undegraded":"2024-04-22T13:25:12.377789+0000","last_fullsized":"2024-04-22T13:25:12.377789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896682+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:25:11.896682+0000","last_peered":"2024-04-22T13:25:11.896682+0000","last_clean":"2024-04-22T13:25:11.896682+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:25:11.896682+0000","last_undegraded":"2024-04-22T13:25:11.896682+0000","last_fullsized":"2024-04-22T13:25:11.896682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899591+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:25:11.899591+0000","last_peered":"2024-04-22T13:25:11.899591+0000","last_clean":"2024-04-22T13:25:11.899591+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:25:11.899591+0000","last_undegraded":"2024-04-22T13:25:11.899591+0000","last_fullsized":"2024-04-22T13:25:11.899591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377506+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:25:12.377506+0000","last_peered":"2024-04-22T13:25:12.377506+0000","last_clean":"2024-04-22T13:25:12.377506+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:25:12.377506+0000","last_undegraded":"2024-04-22T13:25:12.377506+0000","last_fullsized":"2024-04-22T13:25:12.377506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377923+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:25:12.377923+0000","last_peered":"2024-04-22T13:25:12.377923+0000","last_clean":"2024-04-22T13:25:12.377923+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:25:12.377923+0000","last_undegraded":"2024-04-22T13:25:12.377923+0000","last_fullsized":"2024-04-22T13:25:12.377923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.897150+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:25:11.897150+0000","last_peered":"2024-04-22T13:25:11.897150+0000","last_clean":"2024-04-22T13:25:11.897150+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:25:11.897150+0000","last_undegraded":"2024-04-22T13:25:11.897150+0000","last_fullsized":"2024-04-22T13:25:11.897150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376847+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:25:12.376847+0000","last_peered":"2024-04-22T13:25:12.376847+0000","last_clean":"2024-04-22T13:25:12.376847+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:25:12.376847+0000","last_undegraded":"2024-04-22T13:25:12.376847+0000","last_fullsized":"2024-04-22T13:25:12.376847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376023+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:25:12.376023+0000","last_peered":"2024-04-22T13:25:12.376023+0000","last_clean":"2024-04-22T13:25:12.376023+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:25:12.376023+0000","last_undegraded":"2024-04-22T13:25:12.376023+0000","last_fullsized":"2024-04-22T13:25:12.376023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376036+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:25:12.376036+0000","last_peered":"2024-04-22T13:25:12.376036+0000","last_clean":"2024-04-22T13:25:12.376036+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:25:12.376036+0000","last_undegraded":"2024-04-22T13:25:12.376036+0000","last_fullsized":"2024-04-22T13:25:12.376036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375251+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:25:12.375251+0000","last_peered":"2024-04-22T13:25:12.375251+0000","last_clean":"2024-04-22T13:25:12.375251+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:25:12.375251+0000","last_undegraded":"2024-04-22T13:25:12.375251+0000","last_fullsized":"2024-04-22T13:25:12.375251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376604+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:25:12.376604+0000","last_peered":"2024-04-22T13:25:12.376604+0000","last_clean":"2024-04-22T13:25:12.376604+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:25:12.376604+0000","last_undegraded":"2024-04-22T13:25:12.376604+0000","last_fullsized":"2024-04-22T13:25:12.376604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789733+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:25:13.789733+0000","last_peered":"2024-04-22T13:25:13.789733+0000","last_clean":"2024-04-22T13:25:13.789733+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:25:13.789733+0000","last_undegraded":"2024-04-22T13:25:13.789733+0000","last_fullsized":"2024-04-22T13:25:13.789733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.895671+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:25:11.895671+0000","last_peered":"2024-04-22T13:25:11.895671+0000","last_clean":"2024-04-22T13:25:11.895671+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:25:11.895671+0000","last_undegraded":"2024-04-22T13:25:11.895671+0000","last_fullsized":"2024-04-22T13:25:11.895671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896791+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:25:11.896791+0000","last_peered":"2024-04-22T13:25:11.896791+0000","last_clean":"2024-04-22T13:25:11.896791+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:25:11.896791+0000","last_undegraded":"2024-04-22T13:25:11.896791+0000","last_fullsized":"2024-04-22T13:25:11.896791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:17.237330+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:25:17.237330+0000","last_peered":"2024-04-22T13:25:17.237330+0000","last_clean":"2024-04-22T13:25:17.237330+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:25:17.237330+0000","last_undegraded":"2024-04-22T13:25:17.237330+0000","last_fullsized":"2024-04-22T13:25:17.237330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900044+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:25:11.900044+0000","last_peered":"2024-04-22T13:25:11.900044+0000","last_clean":"2024-04-22T13:25:11.900044+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:25:11.900044+0000","last_undegraded":"2024-04-22T13:25:11.900044+0000","last_fullsized":"2024-04-22T13:25:11.900044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900099+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:25:11.900099+0000","last_peered":"2024-04-22T13:25:11.900099+0000","last_clean":"2024-04-22T13:25:11.900099+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:25:11.900099+0000","last_undegraded":"2024-04-22T13:25:11.900099+0000","last_fullsized":"2024-04-22T13:25:11.900099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376047+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:25:12.376047+0000","last_peered":"2024-04-22T13:25:12.376047+0000","last_clean":"2024-04-22T13:25:12.376047+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:25:12.376047+0000","last_undegraded":"2024-04-22T13:25:12.376047+0000","last_fullsized":"2024-04-22T13:25:12.376047+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900404+0000","last_change":"2024-04-22T13:25:09.872420+0000","last_active":"2024-04-22T13:25:11.900404+0000","last_peered":"2024-04-22T13:25:11.900404+0000","last_clean":"2024-04-22T13:25:11.900404+0000","last_became_active":"2024-04-22T13:25:09.871665+0000","last_became_peered":"2024-04-22T13:25:09.871665+0000","last_unstale":"2024-04-22T13:25:11.900404+0000","last_undegraded":"2024-04-22T13:25:11.900404+0000","last_fullsized":"2024-04-22T13:25:11.900404+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:19:01.983486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377225+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:25:12.377225+0000","last_peered":"2024-04-22T13:25:12.377225+0000","last_clean":"2024-04-22T13:25:12.377225+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:25:12.377225+0000","last_undegraded":"2024-04-22T13:25:12.377225+0000","last_fullsized":"2024-04-22T13:25:12.377225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900174+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:25:11.900174+0000","last_peered":"2024-04-22T13:25:11.900174+0000","last_clean":"2024-04-22T13:25:11.900174+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:25:11.900174+0000","last_undegraded":"2024-04-22T13:25:11.900174+0000","last_fullsized":"2024-04-22T13:25:11.900174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376607+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:25:12.376607+0000","last_peered":"2024-04-22T13:25:12.376607+0000","last_clean":"2024-04-22T13:25:12.376607+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:25:12.376607+0000","last_undegraded":"2024-04-22T13:25:12.376607+0000","last_fullsized":"2024-04-22T13:25:12.376607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.922991+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:25:13.922991+0000","last_peered":"2024-04-22T13:25:13.922991+0000","last_clean":"2024-04-22T13:25:13.922991+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:25:13.922991+0000","last_undegraded":"2024-04-22T13:25:13.922991+0000","last_fullsized":"2024-04-22T13:25:13.922991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900218+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:25:11.900218+0000","last_peered":"2024-04-22T13:25:11.900218+0000","last_clean":"2024-04-22T13:25:11.900218+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:25:11.900218+0000","last_undegraded":"2024-04-22T13:25:11.900218+0000","last_fullsized":"2024-04-22T13:25:11.900218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377513+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:25:12.377513+0000","last_peered":"2024-04-22T13:25:12.377513+0000","last_clean":"2024-04-22T13:25:12.377513+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:25:12.377513+0000","last_undegraded":"2024-04-22T13:25:12.377513+0000","last_fullsized":"2024-04-22T13:25:12.377513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376461+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:25:12.376461+0000","last_peered":"2024-04-22T13:25:12.376461+0000","last_clean":"2024-04-22T13:25:12.376461+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:25:12.376461+0000","last_undegraded":"2024-04-22T13:25:12.376461+0000","last_fullsized":"2024-04-22T13:25:12.376461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:14.930563+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:25:14.930563+0000","last_peered":"2024-04-22T13:25:14.930563+0000","last_clean":"2024-04-22T13:25:14.930563+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:25:14.930563+0000","last_undegraded":"2024-04-22T13:25:14.930563+0000","last_fullsized":"2024-04-22T13:25:14.930563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899760+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:25:11.899760+0000","last_peered":"2024-04-22T13:25:11.899760+0000","last_clean":"2024-04-22T13:25:11.899760+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:25:11.899760+0000","last_undegraded":"2024-04-22T13:25:11.899760+0000","last_fullsized":"2024-04-22T13:25:11.899760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376032+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:25:12.376032+0000","last_peered":"2024-04-22T13:25:12.376032+0000","last_clean":"2024-04-22T13:25:12.376032+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:25:12.376032+0000","last_undegraded":"2024-04-22T13:25:12.376032+0000","last_fullsized":"2024-04-22T13:25:12.376032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.895461+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:25:11.895461+0000","last_peered":"2024-04-22T13:25:11.895461+0000","last_clean":"2024-04-22T13:25:11.895461+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:25:11.895461+0000","last_undegraded":"2024-04-22T13:25:11.895461+0000","last_fullsized":"2024-04-22T13:25:11.895461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377729+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:25:12.377729+0000","last_peered":"2024-04-22T13:25:12.377729+0000","last_clean":"2024-04-22T13:25:12.377729+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:25:12.377729+0000","last_undegraded":"2024-04-22T13:25:12.377729+0000","last_fullsized":"2024-04-22T13:25:12.377729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377291+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:25:12.377291+0000","last_peered":"2024-04-22T13:25:12.377291+0000","last_clean":"2024-04-22T13:25:12.377291+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:25:12.377291+0000","last_undegraded":"2024-04-22T13:25:12.377291+0000","last_fullsized":"2024-04-22T13:25:12.377291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896792+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:25:11.896792+0000","last_peered":"2024-04-22T13:25:11.896792+0000","last_clean":"2024-04-22T13:25:11.896792+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:25:11.896792+0000","last_undegraded":"2024-04-22T13:25:11.896792+0000","last_fullsized":"2024-04-22T13:25:11.896792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376270+0000","last_change":"2024-04-22T13:25:10.911708+0000","last_active":"2024-04-22T13:25:12.376270+0000","last_peered":"2024-04-22T13:25:12.376270+0000","last_clean":"2024-04-22T13:25:12.376270+0000","last_became_active":"2024-04-22T13:25:10.911350+0000","last_became_peered":"2024-04-22T13:25:10.911350+0000","last_unstale":"2024-04-22T13:25:12.376270+0000","last_undegraded":"2024-04-22T13:25:12.376270+0000","last_fullsized":"2024-04-22T13:25:12.376270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:33:01.058505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376572+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:25:12.376572+0000","last_peered":"2024-04-22T13:25:12.376572+0000","last_clean":"2024-04-22T13:25:12.376572+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:25:12.376572+0000","last_undegraded":"2024-04-22T13:25:12.376572+0000","last_fullsized":"2024-04-22T13:25:12.376572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377178+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:25:12.377178+0000","last_peered":"2024-04-22T13:25:12.377178+0000","last_clean":"2024-04-22T13:25:12.377178+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:25:12.377178+0000","last_undegraded":"2024-04-22T13:25:12.377178+0000","last_fullsized":"2024-04-22T13:25:12.377178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.378063+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:25:12.378063+0000","last_peered":"2024-04-22T13:25:12.378063+0000","last_clean":"2024-04-22T13:25:12.378063+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:25:12.378063+0000","last_undegraded":"2024-04-22T13:25:12.378063+0000","last_fullsized":"2024-04-22T13:25:12.378063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900355+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:25:11.900355+0000","last_peered":"2024-04-22T13:25:11.900355+0000","last_clean":"2024-04-22T13:25:11.900355+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:25:11.900355+0000","last_undegraded":"2024-04-22T13:25:11.900355+0000","last_fullsized":"2024-04-22T13:25:11.900355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:14.928756+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:25:14.928756+0000","last_peered":"2024-04-22T13:25:14.928756+0000","last_clean":"2024-04-22T13:25:14.928756+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:25:14.928756+0000","last_undegraded":"2024-04-22T13:25:14.928756+0000","last_fullsized":"2024-04-22T13:25:14.928756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375844+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:25:12.375844+0000","last_peered":"2024-04-22T13:25:12.375844+0000","last_clean":"2024-04-22T13:25:12.375844+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:25:12.375844+0000","last_undegraded":"2024-04-22T13:25:12.375844+0000","last_fullsized":"2024-04-22T13:25:12.375844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.922554+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:25:13.922554+0000","last_peered":"2024-04-22T13:25:13.922554+0000","last_clean":"2024-04-22T13:25:13.922554+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:25:13.922554+0000","last_undegraded":"2024-04-22T13:25:13.922554+0000","last_fullsized":"2024-04-22T13:25:13.922554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.897136+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:25:11.897136+0000","last_peered":"2024-04-22T13:25:11.897136+0000","last_clean":"2024-04-22T13:25:11.897136+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:25:11.897136+0000","last_undegraded":"2024-04-22T13:25:11.897136+0000","last_fullsized":"2024-04-22T13:25:11.897136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376263+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:25:12.376263+0000","last_peered":"2024-04-22T13:25:12.376263+0000","last_clean":"2024-04-22T13:25:12.376263+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:25:12.376263+0000","last_undegraded":"2024-04-22T13:25:12.376263+0000","last_fullsized":"2024-04-22T13:25:12.376263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896540+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:25:11.896540+0000","last_peered":"2024-04-22T13:25:11.896540+0000","last_clean":"2024-04-22T13:25:11.896540+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:25:11.896540+0000","last_undegraded":"2024-04-22T13:25:11.896540+0000","last_fullsized":"2024-04-22T13:25:11.896540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899774+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:25:11.899774+0000","last_peered":"2024-04-22T13:25:11.899774+0000","last_clean":"2024-04-22T13:25:11.899774+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:25:11.899774+0000","last_undegraded":"2024-04-22T13:25:11.899774+0000","last_fullsized":"2024-04-22T13:25:11.899774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376347+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:25:12.376347+0000","last_peered":"2024-04-22T13:25:12.376347+0000","last_clean":"2024-04-22T13:25:12.376347+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:25:12.376347+0000","last_undegraded":"2024-04-22T13:25:12.376347+0000","last_fullsized":"2024-04-22T13:25:12.376347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900210+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:25:11.900210+0000","last_peered":"2024-04-22T13:25:11.900210+0000","last_clean":"2024-04-22T13:25:11.900210+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:25:11.900210+0000","last_undegraded":"2024-04-22T13:25:11.900210+0000","last_fullsized":"2024-04-22T13:25:11.900210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789542+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:25:13.789542+0000","last_peered":"2024-04-22T13:25:13.789542+0000","last_clean":"2024-04-22T13:25:13.789542+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:25:13.789542+0000","last_undegraded":"2024-04-22T13:25:13.789542+0000","last_fullsized":"2024-04-22T13:25:13.789542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.956796+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:25:12.956796+0000","last_peered":"2024-04-22T13:25:12.956796+0000","last_clean":"2024-04-22T13:25:12.956796+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:25:12.956796+0000","last_undegraded":"2024-04-22T13:25:12.956796+0000","last_fullsized":"2024-04-22T13:25:12.956796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377563+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:25:12.377563+0000","last_peered":"2024-04-22T13:25:12.377563+0000","last_clean":"2024-04-22T13:25:12.377563+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:25:12.377563+0000","last_undegraded":"2024-04-22T13:25:12.377563+0000","last_fullsized":"2024-04-22T13:25:12.377563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896590+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:25:11.896590+0000","last_peered":"2024-04-22T13:25:11.896590+0000","last_clean":"2024-04-22T13:25:11.896590+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:25:11.896590+0000","last_undegraded":"2024-04-22T13:25:11.896590+0000","last_fullsized":"2024-04-22T13:25:11.896590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377939+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:25:12.377939+0000","last_peered":"2024-04-22T13:25:12.377939+0000","last_clean":"2024-04-22T13:25:12.377939+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:25:12.377939+0000","last_undegraded":"2024-04-22T13:25:12.377939+0000","last_fullsized":"2024-04-22T13:25:12.377939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376381+0000","last_change":"2024-04-22T13:25:10.905094+0000","last_active":"2024-04-22T13:25:12.376381+0000","last_peered":"2024-04-22T13:25:12.376381+0000","last_clean":"2024-04-22T13:25:12.376381+0000","last_became_active":"2024-04-22T13:25:10.904706+0000","last_became_peered":"2024-04-22T13:25:10.904706+0000","last_unstale":"2024-04-22T13:25:12.376381+0000","last_undegraded":"2024-04-22T13:25:12.376381+0000","last_fullsized":"2024-04-22T13:25:12.376381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:33:10.550067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:17.236278+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:25:17.236278+0000","last_peered":"2024-04-22T13:25:17.236278+0000","last_clean":"2024-04-22T13:25:17.236278+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:25:17.236278+0000","last_undegraded":"2024-04-22T13:25:17.236278+0000","last_fullsized":"2024-04-22T13:25:17.236278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899976+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:25:11.899976+0000","last_peered":"2024-04-22T13:25:11.899976+0000","last_clean":"2024-04-22T13:25:11.899976+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:25:11.899976+0000","last_undegraded":"2024-04-22T13:25:11.899976+0000","last_fullsized":"2024-04-22T13:25:11.899976+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.922019+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:25:13.922019+0000","last_peered":"2024-04-22T13:25:13.922019+0000","last_clean":"2024-04-22T13:25:13.922019+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:25:13.922019+0000","last_undegraded":"2024-04-22T13:25:13.922019+0000","last_fullsized":"2024-04-22T13:25:13.922019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896377+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:25:11.896377+0000","last_peered":"2024-04-22T13:25:11.896377+0000","last_clean":"2024-04-22T13:25:11.896377+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:25:11.896377+0000","last_undegraded":"2024-04-22T13:25:11.896377+0000","last_fullsized":"2024-04-22T13:25:11.896377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375498+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:25:12.375498+0000","last_peered":"2024-04-22T13:25:12.375498+0000","last_clean":"2024-04-22T13:25:12.375498+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:25:12.375498+0000","last_undegraded":"2024-04-22T13:25:12.375498+0000","last_fullsized":"2024-04-22T13:25:12.375498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896755+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:25:11.896755+0000","last_peered":"2024-04-22T13:25:11.896755+0000","last_clean":"2024-04-22T13:25:11.896755+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:25:11.896755+0000","last_undegraded":"2024-04-22T13:25:11.896755+0000","last_fullsized":"2024-04-22T13:25:11.896755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900315+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:25:11.900315+0000","last_peered":"2024-04-22T13:25:11.900315+0000","last_clean":"2024-04-22T13:25:11.900315+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:25:11.900315+0000","last_undegraded":"2024-04-22T13:25:11.900315+0000","last_fullsized":"2024-04-22T13:25:11.900315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375924+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:25:12.375924+0000","last_peered":"2024-04-22T13:25:12.375924+0000","last_clean":"2024-04-22T13:25:12.375924+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:25:12.375924+0000","last_undegraded":"2024-04-22T13:25:12.375924+0000","last_fullsized":"2024-04-22T13:25:12.375924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:15.939255+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:25:15.939255+0000","last_peered":"2024-04-22T13:25:15.939255+0000","last_clean":"2024-04-22T13:25:15.939255+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:25:15.939255+0000","last_undegraded":"2024-04-22T13:25:15.939255+0000","last_fullsized":"2024-04-22T13:25:15.939255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900116+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:25:11.900116+0000","last_peered":"2024-04-22T13:25:11.900116+0000","last_clean":"2024-04-22T13:25:11.900116+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:25:11.900116+0000","last_undegraded":"2024-04-22T13:25:11.900116+0000","last_fullsized":"2024-04-22T13:25:11.900116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.920072+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:25:13.920072+0000","last_peered":"2024-04-22T13:25:13.920072+0000","last_clean":"2024-04-22T13:25:13.920072+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:25:13.920072+0000","last_undegraded":"2024-04-22T13:25:13.920072+0000","last_fullsized":"2024-04-22T13:25:13.920072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376002+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:25:12.376002+0000","last_peered":"2024-04-22T13:25:12.376002+0000","last_clean":"2024-04-22T13:25:12.376002+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:25:12.376002+0000","last_undegraded":"2024-04-22T13:25:12.376002+0000","last_fullsized":"2024-04-22T13:25:12.376002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.920444+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:25:13.920444+0000","last_peered":"2024-04-22T13:25:13.920444+0000","last_clean":"2024-04-22T13:25:13.920444+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:25:13.920444+0000","last_undegraded":"2024-04-22T13:25:13.920444+0000","last_fullsized":"2024-04-22T13:25:13.920444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899955+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:25:11.899955+0000","last_peered":"2024-04-22T13:25:11.899955+0000","last_clean":"2024-04-22T13:25:11.899955+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:25:11.899955+0000","last_undegraded":"2024-04-22T13:25:11.899955+0000","last_fullsized":"2024-04-22T13:25:11.899955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.920689+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:25:13.920689+0000","last_peered":"2024-04-22T13:25:13.920689+0000","last_clean":"2024-04-22T13:25:13.920689+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:25:13.920689+0000","last_undegraded":"2024-04-22T13:25:13.920689+0000","last_fullsized":"2024-04-22T13:25:13.920689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789394+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:25:13.789394+0000","last_peered":"2024-04-22T13:25:13.789394+0000","last_clean":"2024-04-22T13:25:13.789394+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:25:13.789394+0000","last_undegraded":"2024-04-22T13:25:13.789394+0000","last_fullsized":"2024-04-22T13:25:13.789394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896340+0000","last_change":"2024-04-22T13:25:09.878548+0000","last_active":"2024-04-22T13:25:11.896340+0000","last_peered":"2024-04-22T13:25:11.896340+0000","last_clean":"2024-04-22T13:25:11.896340+0000","last_became_active":"2024-04-22T13:25:09.878171+0000","last_became_peered":"2024-04-22T13:25:09.878171+0000","last_unstale":"2024-04-22T13:25:11.896340+0000","last_undegraded":"2024-04-22T13:25:11.896340+0000","last_fullsized":"2024-04-22T13:25:11.896340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:22:05.985576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899978+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:25:11.899978+0000","last_peered":"2024-04-22T13:25:11.899978+0000","last_clean":"2024-04-22T13:25:11.899978+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:25:11.899978+0000","last_undegraded":"2024-04-22T13:25:11.899978+0000","last_fullsized":"2024-04-22T13:25:11.899978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899947+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:25:11.899947+0000","last_peered":"2024-04-22T13:25:11.899947+0000","last_clean":"2024-04-22T13:25:11.899947+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:25:11.899947+0000","last_undegraded":"2024-04-22T13:25:11.899947+0000","last_fullsized":"2024-04-22T13:25:11.899947+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896974+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:25:11.896974+0000","last_peered":"2024-04-22T13:25:11.896974+0000","last_clean":"2024-04-22T13:25:11.896974+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:25:11.896974+0000","last_undegraded":"2024-04-22T13:25:11.896974+0000","last_fullsized":"2024-04-22T13:25:11.896974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:14.928198+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:25:14.928198+0000","last_peered":"2024-04-22T13:25:14.928198+0000","last_clean":"2024-04-22T13:25:14.928198+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:25:14.928198+0000","last_undegraded":"2024-04-22T13:25:14.928198+0000","last_fullsized":"2024-04-22T13:25:14.928198+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.897102+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:25:11.897102+0000","last_peered":"2024-04-22T13:25:11.897102+0000","last_clean":"2024-04-22T13:25:11.897102+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:25:11.897102+0000","last_undegraded":"2024-04-22T13:25:11.897102+0000","last_fullsized":"2024-04-22T13:25:11.897102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376374+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:25:12.376374+0000","last_peered":"2024-04-22T13:25:12.376374+0000","last_clean":"2024-04-22T13:25:12.376374+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:25:12.376374+0000","last_undegraded":"2024-04-22T13:25:12.376374+0000","last_fullsized":"2024-04-22T13:25:12.376374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896738+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:25:11.896738+0000","last_peered":"2024-04-22T13:25:11.896738+0000","last_clean":"2024-04-22T13:25:11.896738+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:25:11.896738+0000","last_undegraded":"2024-04-22T13:25:11.896738+0000","last_fullsized":"2024-04-22T13:25:11.896738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:16.945734+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:25:16.945734+0000","last_peered":"2024-04-22T13:25:16.945734+0000","last_clean":"2024-04-22T13:25:16.945734+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:25:16.945734+0000","last_undegraded":"2024-04-22T13:25:16.945734+0000","last_fullsized":"2024-04-22T13:25:16.945734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896961+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:25:11.896961+0000","last_peered":"2024-04-22T13:25:11.896961+0000","last_clean":"2024-04-22T13:25:11.896961+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:25:11.896961+0000","last_undegraded":"2024-04-22T13:25:11.896961+0000","last_fullsized":"2024-04-22T13:25:11.896961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789562+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:25:13.789562+0000","last_peered":"2024-04-22T13:25:13.789562+0000","last_clean":"2024-04-22T13:25:13.789562+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:25:13.789562+0000","last_undegraded":"2024-04-22T13:25:13.789562+0000","last_fullsized":"2024-04-22T13:25:13.789562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896393+0000","last_change":"2024-04-22T13:25:10.911776+0000","last_active":"2024-04-22T13:25:11.896393+0000","last_peered":"2024-04-22T13:25:11.896393+0000","last_clean":"2024-04-22T13:25:11.896393+0000","last_became_active":"2024-04-22T13:25:10.911441+0000","last_became_peered":"2024-04-22T13:25:10.911441+0000","last_unstale":"2024-04-22T13:25:11.896393+0000","last_undegraded":"2024-04-22T13:25:11.896393+0000","last_fullsized":"2024-04-22T13:25:11.896393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:47.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896322+0000","last_change":"2024-04-22T13:25:10.903772+0000","last_active":"2024-04-22T13:25:11.896322+0000","last_peered":"2024-04-22T13:25:11.896322+0000","last_clean":"2024-04-22T13:25:11.896322+0000","last_became_active":"2024-04-22T13:25:10.903378+0000","last_became_peered":"2024-04-22T13:25:10.903378+0000","last_unstale":"2024-04-22T13:25:11.896322+0000","last_undegraded":"2024-04-22T13:25:11.896322+0000","last_fullsized":"2024-04-22T13:25:11.896322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:37.626324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375824+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:25:12.375824+0000","last_peered":"2024-04-22T13:25:12.375824+0000","last_clean":"2024-04-22T13:25:12.375824+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:25:12.375824+0000","last_undegraded":"2024-04-22T13:25:12.375824+0000","last_fullsized":"2024-04-22T13:25:12.375824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376004+0000","last_change":"2024-04-22T13:25:10.900335+0000","last_active":"2024-04-22T13:25:12.376004+0000","last_peered":"2024-04-22T13:25:12.376004+0000","last_clean":"2024-04-22T13:25:12.376004+0000","last_became_active":"2024-04-22T13:25:10.899884+0000","last_became_peered":"2024-04-22T13:25:10.899884+0000","last_unstale":"2024-04-22T13:25:12.376004+0000","last_undegraded":"2024-04-22T13:25:12.376004+0000","last_fullsized":"2024-04-22T13:25:12.376004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:21:07.454212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789619+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:25:13.789619+0000","last_peered":"2024-04-22T13:25:13.789619+0000","last_clean":"2024-04-22T13:25:13.789619+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:25:13.789619+0000","last_undegraded":"2024-04-22T13:25:13.789619+0000","last_fullsized":"2024-04-22T13:25:13.789619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896776+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:25:11.896776+0000","last_peered":"2024-04-22T13:25:11.896776+0000","last_clean":"2024-04-22T13:25:11.896776+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:25:11.896776+0000","last_undegraded":"2024-04-22T13:25:11.896776+0000","last_fullsized":"2024-04-22T13:25:11.896776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900068+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:25:11.900068+0000","last_peered":"2024-04-22T13:25:11.900068+0000","last_clean":"2024-04-22T13:25:11.900068+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:25:11.900068+0000","last_undegraded":"2024-04-22T13:25:11.900068+0000","last_fullsized":"2024-04-22T13:25:11.900068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376249+0000","last_change":"2024-04-22T13:25:10.902634+0000","last_active":"2024-04-22T13:25:12.376249+0000","last_peered":"2024-04-22T13:25:12.376249+0000","last_clean":"2024-04-22T13:25:12.376249+0000","last_became_active":"2024-04-22T13:25:10.902065+0000","last_became_peered":"2024-04-22T13:25:10.902065+0000","last_unstale":"2024-04-22T13:25:12.376249+0000","last_undegraded":"2024-04-22T13:25:12.376249+0000","last_fullsized":"2024-04-22T13:25:12.376249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:45.291008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899666+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:25:11.899666+0000","last_peered":"2024-04-22T13:25:11.899666+0000","last_clean":"2024-04-22T13:25:11.899666+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:25:11.899666+0000","last_undegraded":"2024-04-22T13:25:11.899666+0000","last_fullsized":"2024-04-22T13:25:11.899666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376996+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:25:12.376996+0000","last_peered":"2024-04-22T13:25:12.376996+0000","last_clean":"2024-04-22T13:25:12.376996+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:25:12.376996+0000","last_undegraded":"2024-04-22T13:25:12.376996+0000","last_fullsized":"2024-04-22T13:25:12.376996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376183+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:25:12.376183+0000","last_peered":"2024-04-22T13:25:12.376183+0000","last_clean":"2024-04-22T13:25:12.376183+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:25:12.376183+0000","last_undegraded":"2024-04-22T13:25:12.376183+0000","last_fullsized":"2024-04-22T13:25:12.376183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789525+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:25:13.789525+0000","last_peered":"2024-04-22T13:25:13.789525+0000","last_clean":"2024-04-22T13:25:13.789525+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:25:13.789525+0000","last_undegraded":"2024-04-22T13:25:13.789525+0000","last_fullsized":"2024-04-22T13:25:13.789525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4488,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":108,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":13168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252876,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":457214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514513,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1564672,"data_stored":1040818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678040,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":458336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939677,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":457818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41028,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704124,"statfs":{"total":95995035648,"available":95953022976,"internally_reserved":0,"allocated":1556480,"data_stored":1048164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":458826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":459482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":457408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":457236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712440,"statfs":{"total":95995035648,"available":95961538560,"internally_reserved":0,"allocated":1564672,"data_stored":1047764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:25:19.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:25:19.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.575+0000 7f51c8474640 1 -- 172.21.15.5:0/3768967556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c0102aa0 msgr2=0x7f51c0104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.575+0000 7f51c8474640 1 --2- 172.21.15.5:0/3768967556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c0102aa0 0x7f51c0104e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009900 tx=0x7f51b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- 172.21.15.5:0/3768967556 shutdown_connections 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 --2- 172.21.15.5:0/3768967556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51c0107d00 0x7f51c010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 --2- 172.21.15.5:0/3768967556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c01053d0 0x7f51c01077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 --2- 172.21.15.5:0/3768967556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c0102aa0 0x7f51c0104e90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- 172.21.15.5:0/3768967556 >> 172.21.15.5:0/3768967556 conn(0x7f51c00fc3f0 msgr2=0x7f51c00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- 172.21.15.5:0/3768967556 shutdown_connections 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- 172.21.15.5:0/3768967556 wait complete. 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 Processor -- start 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- start start 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51c0102aa0 0x7f51c01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 0x7f51c01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0107d00 0x7f51c019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c010dae0 con 0x7f51c01053d0 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f51c010d960 con 0x7f51c0107d00 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c8474640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51c010dc60 con 0x7f51c0102aa0 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.576+0000 7f51c5a0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 0x7f51c01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 0x7f51c01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51838/0 (socket says 172.21.15.5:51838) 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 -- 172.21.15.5:0/3063907422 learned_addr learned my addr 172.21.15.5:0/3063907422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c6a11640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0107d00 0x7f51c019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c6210640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51c0102aa0 0x7f51c01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 -- 172.21.15.5:0/3063907422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51c0102aa0 msgr2=0x7f51c01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51c0102aa0 0x7f51c01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 -- 172.21.15.5:0/3063907422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0107d00 msgr2=0x7f51c019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0107d00 0x7f51c019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 -- 172.21.15.5:0/3063907422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c019fdc0 con 0x7f51c01053d0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c5a0f640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 0x7f51c01986e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51b400d770 tx=0x7f51b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4017070 con 0x7f51c01053d0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51b4009c30 con 0x7f51c01053d0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c01a00b0 con 0x7f51c01053d0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b400f620 con 0x7f51c01053d0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.577+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01a0600 con 0x7f51c01053d0 2024-04-22T13:25:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.578+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c00fdee0 con 0x7f51c01053d0 2024-04-22T13:25:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.579+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f51b401a070 con 0x7f51c01053d0 2024-04-22T13:25:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.580+0000 7f51bf7fe640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51a80a3c40 0x7f51a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.580+0000 7f51c6210640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51a80a3c40 0x7f51a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.580+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f51b40cfe90 con 0x7f51c01053d0 2024-04-22T13:25:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.580+0000 7f51c6210640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51a80a3c40 0x7f51a80a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009230 tx=0x7f51b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.584+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51b4009de0 con 0x7f51c01053d0 2024-04-22T13:25:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.767+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51c0062390 con 0x7f51a80a3c40 2024-04-22T13:25:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.774+0000 7f51bf7fe640 1 -- 172.21.15.5:0/3063907422 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258728 (secure 0 0 0) 0x7f51c0062390 con 0x7f51a80a3c40 2024-04-22T13:25:19.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:19.775 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51a80a3c40 msgr2=0x7f51a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51a80a3c40 0x7f51a80a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009230 tx=0x7f51b8005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 msgr2=0x7f51c01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 0x7f51c01986e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51b400d770 tx=0x7f51b400dc30 comp rx=0 tx=0).stop 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 shutdown_connections 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f51a80a3c40 0x7f51a80a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c01053d0 0x7f51c01986e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c0107d00 0x7f51c019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 --2- 172.21.15.5:0/3063907422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51c0102aa0 0x7f51c01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 >> 172.21.15.5:0/3063907422 conn(0x7f51c00fc3f0 msgr2=0x7f51c01034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 shutdown_connections 2024-04-22T13:25:19.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:19.778+0000 7f51c8474640 1 -- 172.21.15.5:0/3063907422 wait complete. 2024-04-22T13:25:19.785 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-04-22T13:25:18.179601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594856,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":140,"num_read_kb":123,"num_write":289,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399780,"kb_used_data":13144,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542044,"statfs":{"total":1151940427776,"available":1151531053056,"internally_reserved":0,"allocated":13459456,"data_stored":7260282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4488,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":108,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001595"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789603+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:25:13.789603+0000","last_peered":"2024-04-22T13:25:13.789603+0000","last_clean":"2024-04-22T13:25:13.789603+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:25:13.789603+0000","last_undegraded":"2024-04-22T13:25:13.789603+0000","last_fullsized":"2024-04-22T13:25:13.789603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375819+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:25:12.375819+0000","last_peered":"2024-04-22T13:25:12.375819+0000","last_clean":"2024-04-22T13:25:12.375819+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:25:12.375819+0000","last_undegraded":"2024-04-22T13:25:12.375819+0000","last_fullsized":"2024-04-22T13:25:12.375819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375616+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:25:12.375616+0000","last_peered":"2024-04-22T13:25:12.375616+0000","last_clean":"2024-04-22T13:25:12.375616+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:25:12.375616+0000","last_undegraded":"2024-04-22T13:25:12.375616+0000","last_fullsized":"2024-04-22T13:25:12.375616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376632+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:25:12.376632+0000","last_peered":"2024-04-22T13:25:12.376632+0000","last_clean":"2024-04-22T13:25:12.376632+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:25:12.376632+0000","last_undegraded":"2024-04-22T13:25:12.376632+0000","last_fullsized":"2024-04-22T13:25:12.376632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899988+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:25:11.899988+0000","last_peered":"2024-04-22T13:25:11.899988+0000","last_clean":"2024-04-22T13:25:11.899988+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:25:11.899988+0000","last_undegraded":"2024-04-22T13:25:11.899988+0000","last_fullsized":"2024-04-22T13:25:11.899988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377789+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:25:12.377789+0000","last_peered":"2024-04-22T13:25:12.377789+0000","last_clean":"2024-04-22T13:25:12.377789+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:25:12.377789+0000","last_undegraded":"2024-04-22T13:25:12.377789+0000","last_fullsized":"2024-04-22T13:25:12.377789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896682+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:25:11.896682+0000","last_peered":"2024-04-22T13:25:11.896682+0000","last_clean":"2024-04-22T13:25:11.896682+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:25:11.896682+0000","last_undegraded":"2024-04-22T13:25:11.896682+0000","last_fullsized":"2024-04-22T13:25:11.896682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899591+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:25:11.899591+0000","last_peered":"2024-04-22T13:25:11.899591+0000","last_clean":"2024-04-22T13:25:11.899591+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:25:11.899591+0000","last_undegraded":"2024-04-22T13:25:11.899591+0000","last_fullsized":"2024-04-22T13:25:11.899591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377506+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:25:12.377506+0000","last_peered":"2024-04-22T13:25:12.377506+0000","last_clean":"2024-04-22T13:25:12.377506+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:25:12.377506+0000","last_undegraded":"2024-04-22T13:25:12.377506+0000","last_fullsized":"2024-04-22T13:25:12.377506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377923+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:25:12.377923+0000","last_peered":"2024-04-22T13:25:12.377923+0000","last_clean":"2024-04-22T13:25:12.377923+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:25:12.377923+0000","last_undegraded":"2024-04-22T13:25:12.377923+0000","last_fullsized":"2024-04-22T13:25:12.377923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.897150+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:25:11.897150+0000","last_peered":"2024-04-22T13:25:11.897150+0000","last_clean":"2024-04-22T13:25:11.897150+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:25:11.897150+0000","last_undegraded":"2024-04-22T13:25:11.897150+0000","last_fullsized":"2024-04-22T13:25:11.897150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376847+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:25:12.376847+0000","last_peered":"2024-04-22T13:25:12.376847+0000","last_clean":"2024-04-22T13:25:12.376847+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:25:12.376847+0000","last_undegraded":"2024-04-22T13:25:12.376847+0000","last_fullsized":"2024-04-22T13:25:12.376847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376023+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:25:12.376023+0000","last_peered":"2024-04-22T13:25:12.376023+0000","last_clean":"2024-04-22T13:25:12.376023+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:25:12.376023+0000","last_undegraded":"2024-04-22T13:25:12.376023+0000","last_fullsized":"2024-04-22T13:25:12.376023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376036+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:25:12.376036+0000","last_peered":"2024-04-22T13:25:12.376036+0000","last_clean":"2024-04-22T13:25:12.376036+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:25:12.376036+0000","last_undegraded":"2024-04-22T13:25:12.376036+0000","last_fullsized":"2024-04-22T13:25:12.376036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375251+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:25:12.375251+0000","last_peered":"2024-04-22T13:25:12.375251+0000","last_clean":"2024-04-22T13:25:12.375251+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:25:12.375251+0000","last_undegraded":"2024-04-22T13:25:12.375251+0000","last_fullsized":"2024-04-22T13:25:12.375251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376604+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:25:12.376604+0000","last_peered":"2024-04-22T13:25:12.376604+0000","last_clean":"2024-04-22T13:25:12.376604+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:25:12.376604+0000","last_undegraded":"2024-04-22T13:25:12.376604+0000","last_fullsized":"2024-04-22T13:25:12.376604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789733+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:25:13.789733+0000","last_peered":"2024-04-22T13:25:13.789733+0000","last_clean":"2024-04-22T13:25:13.789733+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:25:13.789733+0000","last_undegraded":"2024-04-22T13:25:13.789733+0000","last_fullsized":"2024-04-22T13:25:13.789733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.895671+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:25:11.895671+0000","last_peered":"2024-04-22T13:25:11.895671+0000","last_clean":"2024-04-22T13:25:11.895671+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:25:11.895671+0000","last_undegraded":"2024-04-22T13:25:11.895671+0000","last_fullsized":"2024-04-22T13:25:11.895671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896791+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:25:11.896791+0000","last_peered":"2024-04-22T13:25:11.896791+0000","last_clean":"2024-04-22T13:25:11.896791+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:25:11.896791+0000","last_undegraded":"2024-04-22T13:25:11.896791+0000","last_fullsized":"2024-04-22T13:25:11.896791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:17.237330+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:25:17.237330+0000","last_peered":"2024-04-22T13:25:17.237330+0000","last_clean":"2024-04-22T13:25:17.237330+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:25:17.237330+0000","last_undegraded":"2024-04-22T13:25:17.237330+0000","last_fullsized":"2024-04-22T13:25:17.237330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900044+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:25:11.900044+0000","last_peered":"2024-04-22T13:25:11.900044+0000","last_clean":"2024-04-22T13:25:11.900044+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:25:11.900044+0000","last_undegraded":"2024-04-22T13:25:11.900044+0000","last_fullsized":"2024-04-22T13:25:11.900044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900099+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:25:11.900099+0000","last_peered":"2024-04-22T13:25:11.900099+0000","last_clean":"2024-04-22T13:25:11.900099+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:25:11.900099+0000","last_undegraded":"2024-04-22T13:25:11.900099+0000","last_fullsized":"2024-04-22T13:25:11.900099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376047+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:25:12.376047+0000","last_peered":"2024-04-22T13:25:12.376047+0000","last_clean":"2024-04-22T13:25:12.376047+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:25:12.376047+0000","last_undegraded":"2024-04-22T13:25:12.376047+0000","last_fullsized":"2024-04-22T13:25:12.376047+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900404+0000","last_change":"2024-04-22T13:25:09.872420+0000","last_active":"2024-04-22T13:25:11.900404+0000","last_peered":"2024-04-22T13:25:11.900404+0000","last_clean":"2024-04-22T13:25:11.900404+0000","last_became_active":"2024-04-22T13:25:09.871665+0000","last_became_peered":"2024-04-22T13:25:09.871665+0000","last_unstale":"2024-04-22T13:25:11.900404+0000","last_undegraded":"2024-04-22T13:25:11.900404+0000","last_fullsized":"2024-04-22T13:25:11.900404+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:19:01.983486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377225+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:25:12.377225+0000","last_peered":"2024-04-22T13:25:12.377225+0000","last_clean":"2024-04-22T13:25:12.377225+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:25:12.377225+0000","last_undegraded":"2024-04-22T13:25:12.377225+0000","last_fullsized":"2024-04-22T13:25:12.377225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900174+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:25:11.900174+0000","last_peered":"2024-04-22T13:25:11.900174+0000","last_clean":"2024-04-22T13:25:11.900174+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:25:11.900174+0000","last_undegraded":"2024-04-22T13:25:11.900174+0000","last_fullsized":"2024-04-22T13:25:11.900174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376607+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:25:12.376607+0000","last_peered":"2024-04-22T13:25:12.376607+0000","last_clean":"2024-04-22T13:25:12.376607+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:25:12.376607+0000","last_undegraded":"2024-04-22T13:25:12.376607+0000","last_fullsized":"2024-04-22T13:25:12.376607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.922991+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:25:13.922991+0000","last_peered":"2024-04-22T13:25:13.922991+0000","last_clean":"2024-04-22T13:25:13.922991+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:25:13.922991+0000","last_undegraded":"2024-04-22T13:25:13.922991+0000","last_fullsized":"2024-04-22T13:25:13.922991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900218+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:25:11.900218+0000","last_peered":"2024-04-22T13:25:11.900218+0000","last_clean":"2024-04-22T13:25:11.900218+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:25:11.900218+0000","last_undegraded":"2024-04-22T13:25:11.900218+0000","last_fullsized":"2024-04-22T13:25:11.900218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377513+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:25:12.377513+0000","last_peered":"2024-04-22T13:25:12.377513+0000","last_clean":"2024-04-22T13:25:12.377513+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:25:12.377513+0000","last_undegraded":"2024-04-22T13:25:12.377513+0000","last_fullsized":"2024-04-22T13:25:12.377513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376461+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:25:12.376461+0000","last_peered":"2024-04-22T13:25:12.376461+0000","last_clean":"2024-04-22T13:25:12.376461+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:25:12.376461+0000","last_undegraded":"2024-04-22T13:25:12.376461+0000","last_fullsized":"2024-04-22T13:25:12.376461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:14.930563+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:25:14.930563+0000","last_peered":"2024-04-22T13:25:14.930563+0000","last_clean":"2024-04-22T13:25:14.930563+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:25:14.930563+0000","last_undegraded":"2024-04-22T13:25:14.930563+0000","last_fullsized":"2024-04-22T13:25:14.930563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899760+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:25:11.899760+0000","last_peered":"2024-04-22T13:25:11.899760+0000","last_clean":"2024-04-22T13:25:11.899760+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:25:11.899760+0000","last_undegraded":"2024-04-22T13:25:11.899760+0000","last_fullsized":"2024-04-22T13:25:11.899760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376032+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:25:12.376032+0000","last_peered":"2024-04-22T13:25:12.376032+0000","last_clean":"2024-04-22T13:25:12.376032+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:25:12.376032+0000","last_undegraded":"2024-04-22T13:25:12.376032+0000","last_fullsized":"2024-04-22T13:25:12.376032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.895461+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:25:11.895461+0000","last_peered":"2024-04-22T13:25:11.895461+0000","last_clean":"2024-04-22T13:25:11.895461+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:25:11.895461+0000","last_undegraded":"2024-04-22T13:25:11.895461+0000","last_fullsized":"2024-04-22T13:25:11.895461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377729+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:25:12.377729+0000","last_peered":"2024-04-22T13:25:12.377729+0000","last_clean":"2024-04-22T13:25:12.377729+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:25:12.377729+0000","last_undegraded":"2024-04-22T13:25:12.377729+0000","last_fullsized":"2024-04-22T13:25:12.377729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377291+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:25:12.377291+0000","last_peered":"2024-04-22T13:25:12.377291+0000","last_clean":"2024-04-22T13:25:12.377291+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:25:12.377291+0000","last_undegraded":"2024-04-22T13:25:12.377291+0000","last_fullsized":"2024-04-22T13:25:12.377291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896792+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:25:11.896792+0000","last_peered":"2024-04-22T13:25:11.896792+0000","last_clean":"2024-04-22T13:25:11.896792+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:25:11.896792+0000","last_undegraded":"2024-04-22T13:25:11.896792+0000","last_fullsized":"2024-04-22T13:25:11.896792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376270+0000","last_change":"2024-04-22T13:25:10.911708+0000","last_active":"2024-04-22T13:25:12.376270+0000","last_peered":"2024-04-22T13:25:12.376270+0000","last_clean":"2024-04-22T13:25:12.376270+0000","last_became_active":"2024-04-22T13:25:10.911350+0000","last_became_peered":"2024-04-22T13:25:10.911350+0000","last_unstale":"2024-04-22T13:25:12.376270+0000","last_undegraded":"2024-04-22T13:25:12.376270+0000","last_fullsized":"2024-04-22T13:25:12.376270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:33:01.058505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376572+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:25:12.376572+0000","last_peered":"2024-04-22T13:25:12.376572+0000","last_clean":"2024-04-22T13:25:12.376572+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:25:12.376572+0000","last_undegraded":"2024-04-22T13:25:12.376572+0000","last_fullsized":"2024-04-22T13:25:12.376572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377178+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:25:12.377178+0000","last_peered":"2024-04-22T13:25:12.377178+0000","last_clean":"2024-04-22T13:25:12.377178+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:25:12.377178+0000","last_undegraded":"2024-04-22T13:25:12.377178+0000","last_fullsized":"2024-04-22T13:25:12.377178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.378063+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:25:12.378063+0000","last_peered":"2024-04-22T13:25:12.378063+0000","last_clean":"2024-04-22T13:25:12.378063+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:25:12.378063+0000","last_undegraded":"2024-04-22T13:25:12.378063+0000","last_fullsized":"2024-04-22T13:25:12.378063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900355+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:25:11.900355+0000","last_peered":"2024-04-22T13:25:11.900355+0000","last_clean":"2024-04-22T13:25:11.900355+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:25:11.900355+0000","last_undegraded":"2024-04-22T13:25:11.900355+0000","last_fullsized":"2024-04-22T13:25:11.900355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:14.928756+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:25:14.928756+0000","last_peered":"2024-04-22T13:25:14.928756+0000","last_clean":"2024-04-22T13:25:14.928756+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:25:14.928756+0000","last_undegraded":"2024-04-22T13:25:14.928756+0000","last_fullsized":"2024-04-22T13:25:14.928756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375844+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:25:12.375844+0000","last_peered":"2024-04-22T13:25:12.375844+0000","last_clean":"2024-04-22T13:25:12.375844+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:25:12.375844+0000","last_undegraded":"2024-04-22T13:25:12.375844+0000","last_fullsized":"2024-04-22T13:25:12.375844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.922554+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:25:13.922554+0000","last_peered":"2024-04-22T13:25:13.922554+0000","last_clean":"2024-04-22T13:25:13.922554+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:25:13.922554+0000","last_undegraded":"2024-04-22T13:25:13.922554+0000","last_fullsized":"2024-04-22T13:25:13.922554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.897136+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:25:11.897136+0000","last_peered":"2024-04-22T13:25:11.897136+0000","last_clean":"2024-04-22T13:25:11.897136+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:25:11.897136+0000","last_undegraded":"2024-04-22T13:25:11.897136+0000","last_fullsized":"2024-04-22T13:25:11.897136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376263+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:25:12.376263+0000","last_peered":"2024-04-22T13:25:12.376263+0000","last_clean":"2024-04-22T13:25:12.376263+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:25:12.376263+0000","last_undegraded":"2024-04-22T13:25:12.376263+0000","last_fullsized":"2024-04-22T13:25:12.376263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896540+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:25:11.896540+0000","last_peered":"2024-04-22T13:25:11.896540+0000","last_clean":"2024-04-22T13:25:11.896540+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:25:11.896540+0000","last_undegraded":"2024-04-22T13:25:11.896540+0000","last_fullsized":"2024-04-22T13:25:11.896540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899774+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:25:11.899774+0000","last_peered":"2024-04-22T13:25:11.899774+0000","last_clean":"2024-04-22T13:25:11.899774+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:25:11.899774+0000","last_undegraded":"2024-04-22T13:25:11.899774+0000","last_fullsized":"2024-04-22T13:25:11.899774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376347+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:25:12.376347+0000","last_peered":"2024-04-22T13:25:12.376347+0000","last_clean":"2024-04-22T13:25:12.376347+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:25:12.376347+0000","last_undegraded":"2024-04-22T13:25:12.376347+0000","last_fullsized":"2024-04-22T13:25:12.376347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900210+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:25:11.900210+0000","last_peered":"2024-04-22T13:25:11.900210+0000","last_clean":"2024-04-22T13:25:11.900210+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:25:11.900210+0000","last_undegraded":"2024-04-22T13:25:11.900210+0000","last_fullsized":"2024-04-22T13:25:11.900210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789542+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:25:13.789542+0000","last_peered":"2024-04-22T13:25:13.789542+0000","last_clean":"2024-04-22T13:25:13.789542+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:25:13.789542+0000","last_undegraded":"2024-04-22T13:25:13.789542+0000","last_fullsized":"2024-04-22T13:25:13.789542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.956796+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:25:12.956796+0000","last_peered":"2024-04-22T13:25:12.956796+0000","last_clean":"2024-04-22T13:25:12.956796+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:25:12.956796+0000","last_undegraded":"2024-04-22T13:25:12.956796+0000","last_fullsized":"2024-04-22T13:25:12.956796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377563+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:25:12.377563+0000","last_peered":"2024-04-22T13:25:12.377563+0000","last_clean":"2024-04-22T13:25:12.377563+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:25:12.377563+0000","last_undegraded":"2024-04-22T13:25:12.377563+0000","last_fullsized":"2024-04-22T13:25:12.377563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896590+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:25:11.896590+0000","last_peered":"2024-04-22T13:25:11.896590+0000","last_clean":"2024-04-22T13:25:11.896590+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:25:11.896590+0000","last_undegraded":"2024-04-22T13:25:11.896590+0000","last_fullsized":"2024-04-22T13:25:11.896590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.377939+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:25:12.377939+0000","last_peered":"2024-04-22T13:25:12.377939+0000","last_clean":"2024-04-22T13:25:12.377939+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:25:12.377939+0000","last_undegraded":"2024-04-22T13:25:12.377939+0000","last_fullsized":"2024-04-22T13:25:12.377939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376381+0000","last_change":"2024-04-22T13:25:10.905094+0000","last_active":"2024-04-22T13:25:12.376381+0000","last_peered":"2024-04-22T13:25:12.376381+0000","last_clean":"2024-04-22T13:25:12.376381+0000","last_became_active":"2024-04-22T13:25:10.904706+0000","last_became_peered":"2024-04-22T13:25:10.904706+0000","last_unstale":"2024-04-22T13:25:12.376381+0000","last_undegraded":"2024-04-22T13:25:12.376381+0000","last_fullsized":"2024-04-22T13:25:12.376381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:33:10.550067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:17.236278+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:25:17.236278+0000","last_peered":"2024-04-22T13:25:17.236278+0000","last_clean":"2024-04-22T13:25:17.236278+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:25:17.236278+0000","last_undegraded":"2024-04-22T13:25:17.236278+0000","last_fullsized":"2024-04-22T13:25:17.236278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899976+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:25:11.899976+0000","last_peered":"2024-04-22T13:25:11.899976+0000","last_clean":"2024-04-22T13:25:11.899976+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:25:11.899976+0000","last_undegraded":"2024-04-22T13:25:11.899976+0000","last_fullsized":"2024-04-22T13:25:11.899976+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.922019+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:25:13.922019+0000","last_peered":"2024-04-22T13:25:13.922019+0000","last_clean":"2024-04-22T13:25:13.922019+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:25:13.922019+0000","last_undegraded":"2024-04-22T13:25:13.922019+0000","last_fullsized":"2024-04-22T13:25:13.922019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896377+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:25:11.896377+0000","last_peered":"2024-04-22T13:25:11.896377+0000","last_clean":"2024-04-22T13:25:11.896377+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:25:11.896377+0000","last_undegraded":"2024-04-22T13:25:11.896377+0000","last_fullsized":"2024-04-22T13:25:11.896377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375498+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:25:12.375498+0000","last_peered":"2024-04-22T13:25:12.375498+0000","last_clean":"2024-04-22T13:25:12.375498+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:25:12.375498+0000","last_undegraded":"2024-04-22T13:25:12.375498+0000","last_fullsized":"2024-04-22T13:25:12.375498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896755+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:25:11.896755+0000","last_peered":"2024-04-22T13:25:11.896755+0000","last_clean":"2024-04-22T13:25:11.896755+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:25:11.896755+0000","last_undegraded":"2024-04-22T13:25:11.896755+0000","last_fullsized":"2024-04-22T13:25:11.896755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900315+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:25:11.900315+0000","last_peered":"2024-04-22T13:25:11.900315+0000","last_clean":"2024-04-22T13:25:11.900315+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:25:11.900315+0000","last_undegraded":"2024-04-22T13:25:11.900315+0000","last_fullsized":"2024-04-22T13:25:11.900315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375924+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:25:12.375924+0000","last_peered":"2024-04-22T13:25:12.375924+0000","last_clean":"2024-04-22T13:25:12.375924+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:25:12.375924+0000","last_undegraded":"2024-04-22T13:25:12.375924+0000","last_fullsized":"2024-04-22T13:25:12.375924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:15.939255+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:25:15.939255+0000","last_peered":"2024-04-22T13:25:15.939255+0000","last_clean":"2024-04-22T13:25:15.939255+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:25:15.939255+0000","last_undegraded":"2024-04-22T13:25:15.939255+0000","last_fullsized":"2024-04-22T13:25:15.939255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900116+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:25:11.900116+0000","last_peered":"2024-04-22T13:25:11.900116+0000","last_clean":"2024-04-22T13:25:11.900116+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:25:11.900116+0000","last_undegraded":"2024-04-22T13:25:11.900116+0000","last_fullsized":"2024-04-22T13:25:11.900116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.920072+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:25:13.920072+0000","last_peered":"2024-04-22T13:25:13.920072+0000","last_clean":"2024-04-22T13:25:13.920072+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:25:13.920072+0000","last_undegraded":"2024-04-22T13:25:13.920072+0000","last_fullsized":"2024-04-22T13:25:13.920072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376002+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:25:12.376002+0000","last_peered":"2024-04-22T13:25:12.376002+0000","last_clean":"2024-04-22T13:25:12.376002+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:25:12.376002+0000","last_undegraded":"2024-04-22T13:25:12.376002+0000","last_fullsized":"2024-04-22T13:25:12.376002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.920444+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:25:13.920444+0000","last_peered":"2024-04-22T13:25:13.920444+0000","last_clean":"2024-04-22T13:25:13.920444+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:25:13.920444+0000","last_undegraded":"2024-04-22T13:25:13.920444+0000","last_fullsized":"2024-04-22T13:25:13.920444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899955+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:25:11.899955+0000","last_peered":"2024-04-22T13:25:11.899955+0000","last_clean":"2024-04-22T13:25:11.899955+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:25:11.899955+0000","last_undegraded":"2024-04-22T13:25:11.899955+0000","last_fullsized":"2024-04-22T13:25:11.899955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.920689+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:25:13.920689+0000","last_peered":"2024-04-22T13:25:13.920689+0000","last_clean":"2024-04-22T13:25:13.920689+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:25:13.920689+0000","last_undegraded":"2024-04-22T13:25:13.920689+0000","last_fullsized":"2024-04-22T13:25:13.920689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789394+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:25:13.789394+0000","last_peered":"2024-04-22T13:25:13.789394+0000","last_clean":"2024-04-22T13:25:13.789394+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:25:13.789394+0000","last_undegraded":"2024-04-22T13:25:13.789394+0000","last_fullsized":"2024-04-22T13:25:13.789394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896340+0000","last_change":"2024-04-22T13:25:09.878548+0000","last_active":"2024-04-22T13:25:11.896340+0000","last_peered":"2024-04-22T13:25:11.896340+0000","last_clean":"2024-04-22T13:25:11.896340+0000","last_became_active":"2024-04-22T13:25:09.878171+0000","last_became_peered":"2024-04-22T13:25:09.878171+0000","last_unstale":"2024-04-22T13:25:11.896340+0000","last_undegraded":"2024-04-22T13:25:11.896340+0000","last_fullsized":"2024-04-22T13:25:11.896340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:22:05.985576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899978+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:25:11.899978+0000","last_peered":"2024-04-22T13:25:11.899978+0000","last_clean":"2024-04-22T13:25:11.899978+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:25:11.899978+0000","last_undegraded":"2024-04-22T13:25:11.899978+0000","last_fullsized":"2024-04-22T13:25:11.899978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899947+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:25:11.899947+0000","last_peered":"2024-04-22T13:25:11.899947+0000","last_clean":"2024-04-22T13:25:11.899947+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:25:11.899947+0000","last_undegraded":"2024-04-22T13:25:11.899947+0000","last_fullsized":"2024-04-22T13:25:11.899947+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896974+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:25:11.896974+0000","last_peered":"2024-04-22T13:25:11.896974+0000","last_clean":"2024-04-22T13:25:11.896974+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:25:11.896974+0000","last_undegraded":"2024-04-22T13:25:11.896974+0000","last_fullsized":"2024-04-22T13:25:11.896974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:14.928198+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:25:14.928198+0000","last_peered":"2024-04-22T13:25:14.928198+0000","last_clean":"2024-04-22T13:25:14.928198+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:25:14.928198+0000","last_undegraded":"2024-04-22T13:25:14.928198+0000","last_fullsized":"2024-04-22T13:25:14.928198+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.897102+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:25:11.897102+0000","last_peered":"2024-04-22T13:25:11.897102+0000","last_clean":"2024-04-22T13:25:11.897102+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:25:11.897102+0000","last_undegraded":"2024-04-22T13:25:11.897102+0000","last_fullsized":"2024-04-22T13:25:11.897102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376374+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:25:12.376374+0000","last_peered":"2024-04-22T13:25:12.376374+0000","last_clean":"2024-04-22T13:25:12.376374+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:25:12.376374+0000","last_undegraded":"2024-04-22T13:25:12.376374+0000","last_fullsized":"2024-04-22T13:25:12.376374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896738+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:25:11.896738+0000","last_peered":"2024-04-22T13:25:11.896738+0000","last_clean":"2024-04-22T13:25:11.896738+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:25:11.896738+0000","last_undegraded":"2024-04-22T13:25:11.896738+0000","last_fullsized":"2024-04-22T13:25:11.896738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:16.945734+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:25:16.945734+0000","last_peered":"2024-04-22T13:25:16.945734+0000","last_clean":"2024-04-22T13:25:16.945734+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:25:16.945734+0000","last_undegraded":"2024-04-22T13:25:16.945734+0000","last_fullsized":"2024-04-22T13:25:16.945734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896961+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:25:11.896961+0000","last_peered":"2024-04-22T13:25:11.896961+0000","last_clean":"2024-04-22T13:25:11.896961+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:25:11.896961+0000","last_undegraded":"2024-04-22T13:25:11.896961+0000","last_fullsized":"2024-04-22T13:25:11.896961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789562+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:25:13.789562+0000","last_peered":"2024-04-22T13:25:13.789562+0000","last_clean":"2024-04-22T13:25:13.789562+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:25:13.789562+0000","last_undegraded":"2024-04-22T13:25:13.789562+0000","last_fullsized":"2024-04-22T13:25:13.789562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896393+0000","last_change":"2024-04-22T13:25:10.911776+0000","last_active":"2024-04-22T13:25:11.896393+0000","last_peered":"2024-04-22T13:25:11.896393+0000","last_clean":"2024-04-22T13:25:11.896393+0000","last_became_active":"2024-04-22T13:25:10.911441+0000","last_became_peered":"2024-04-22T13:25:10.911441+0000","last_unstale":"2024-04-22T13:25:11.896393+0000","last_undegraded":"2024-04-22T13:25:11.896393+0000","last_fullsized":"2024-04-22T13:25:11.896393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:47.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896322+0000","last_change":"2024-04-22T13:25:10.903772+0000","last_active":"2024-04-22T13:25:11.896322+0000","last_peered":"2024-04-22T13:25:11.896322+0000","last_clean":"2024-04-22T13:25:11.896322+0000","last_became_active":"2024-04-22T13:25:10.903378+0000","last_became_peered":"2024-04-22T13:25:10.903378+0000","last_unstale":"2024-04-22T13:25:11.896322+0000","last_undegraded":"2024-04-22T13:25:11.896322+0000","last_fullsized":"2024-04-22T13:25:11.896322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:37.626324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.375824+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:25:12.375824+0000","last_peered":"2024-04-22T13:25:12.375824+0000","last_clean":"2024-04-22T13:25:12.375824+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:25:12.375824+0000","last_undegraded":"2024-04-22T13:25:12.375824+0000","last_fullsized":"2024-04-22T13:25:12.375824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376004+0000","last_change":"2024-04-22T13:25:10.900335+0000","last_active":"2024-04-22T13:25:12.376004+0000","last_peered":"2024-04-22T13:25:12.376004+0000","last_clean":"2024-04-22T13:25:12.376004+0000","last_became_active":"2024-04-22T13:25:10.899884+0000","last_became_peered":"2024-04-22T13:25:10.899884+0000","last_unstale":"2024-04-22T13:25:12.376004+0000","last_undegraded":"2024-04-22T13:25:12.376004+0000","last_fullsized":"2024-04-22T13:25:12.376004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:21:07.454212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789619+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:25:13.789619+0000","last_peered":"2024-04-22T13:25:13.789619+0000","last_clean":"2024-04-22T13:25:13.789619+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:25:13.789619+0000","last_undegraded":"2024-04-22T13:25:13.789619+0000","last_fullsized":"2024-04-22T13:25:13.789619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.896776+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:25:11.896776+0000","last_peered":"2024-04-22T13:25:11.896776+0000","last_clean":"2024-04-22T13:25:11.896776+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:25:11.896776+0000","last_undegraded":"2024-04-22T13:25:11.896776+0000","last_fullsized":"2024-04-22T13:25:11.896776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.900068+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:25:11.900068+0000","last_peered":"2024-04-22T13:25:11.900068+0000","last_clean":"2024-04-22T13:25:11.900068+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:25:11.900068+0000","last_undegraded":"2024-04-22T13:25:11.900068+0000","last_fullsized":"2024-04-22T13:25:11.900068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376249+0000","last_change":"2024-04-22T13:25:10.902634+0000","last_active":"2024-04-22T13:25:12.376249+0000","last_peered":"2024-04-22T13:25:12.376249+0000","last_clean":"2024-04-22T13:25:12.376249+0000","last_became_active":"2024-04-22T13:25:10.902065+0000","last_became_peered":"2024-04-22T13:25:10.902065+0000","last_unstale":"2024-04-22T13:25:12.376249+0000","last_undegraded":"2024-04-22T13:25:12.376249+0000","last_fullsized":"2024-04-22T13:25:12.376249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:45.291008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:11.899666+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:25:11.899666+0000","last_peered":"2024-04-22T13:25:11.899666+0000","last_clean":"2024-04-22T13:25:11.899666+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:25:11.899666+0000","last_undegraded":"2024-04-22T13:25:11.899666+0000","last_fullsized":"2024-04-22T13:25:11.899666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376996+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:25:12.376996+0000","last_peered":"2024-04-22T13:25:12.376996+0000","last_clean":"2024-04-22T13:25:12.376996+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:25:12.376996+0000","last_undegraded":"2024-04-22T13:25:12.376996+0000","last_fullsized":"2024-04-22T13:25:12.376996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:12.376183+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:25:12.376183+0000","last_peered":"2024-04-22T13:25:12.376183+0000","last_clean":"2024-04-22T13:25:12.376183+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:25:12.376183+0000","last_undegraded":"2024-04-22T13:25:12.376183+0000","last_fullsized":"2024-04-22T13:25:12.376183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:25:13.789525+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:25:13.789525+0000","last_peered":"2024-04-22T13:25:13.789525+0000","last_clean":"2024-04-22T13:25:13.789525+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:25:13.789525+0000","last_undegraded":"2024-04-22T13:25:13.789525+0000","last_fullsized":"2024-04-22T13:25:13.789525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4488,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":108,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":13168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252876,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":457214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514513,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1564672,"data_stored":1040818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678040,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":458336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939677,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":457818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41028,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704124,"statfs":{"total":95995035648,"available":95953022976,"internally_reserved":0,"allocated":1556480,"data_stored":1048164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":458826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":459482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":457408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":457236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712440,"statfs":{"total":95995035648,"available":95961538560,"internally_reserved":0,"allocated":1564672,"data_stored":1047764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:25:19.789 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-22T13:25:19.789 INFO:tasks.cephadm:Setting up client nodes... 2024-04-22T13:25:19.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T13:25:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:19 smithi005 ceph-mon[28006]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-22T13:25:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:19 smithi005 ceph-mon[28006]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:19 smithi005 ceph-mon[28006]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T13:25:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:19 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1118967833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:20.070 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:19 smithi049 ceph-mon[31379]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-22T13:25:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:19 smithi049 ceph-mon[31379]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:19 smithi049 ceph-mon[31379]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T13:25:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:19 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1118967833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:19 smithi107 ceph-mon[31859]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-22T13:25:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:19 smithi107 ceph-mon[31859]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:25:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:19 smithi107 ceph-mon[31859]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T13:25:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:19 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1118967833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:21.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:20 smithi005 ceph-mon[28006]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:25:21.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:20 smithi005 ceph-mon[28006]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:25:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:20 smithi049 ceph-mon[31379]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:25:21.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:20 smithi049 ceph-mon[31379]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:25:21.272 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: 2024-04-22T13:25:20.851+0000 7f77e4cf6700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-22T13:25:21.272 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:25:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: 2024-04-22T13:25:20.851+0000 7f77e4cf6700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-22T13:25:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:20 smithi107 ceph-mon[31859]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:25:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:20 smithi107 ceph-mon[31859]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 -- 172.21.15.5:0/1863073146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c102af0 msgr2=0x7f5e1c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 --2- 172.21.15.5:0/1863073146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c102af0 0x7f5e1c102f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5e180099c0 tx=0x7f5e18009cd0 comp rx=0 tx=0).stop 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 -- 172.21.15.5:0/1863073146 shutdown_connections 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 --2- 172.21.15.5:0/1863073146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c104690 0x7f5e1c10af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 --2- 172.21.15.5:0/1863073146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c102af0 0x7f5e1c102f10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 --2- 172.21.15.5:0/1863073146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 0x7f5e1c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.720+0000 7f5e23b23700 1 -- 172.21.15.5:0/1863073146 >> 172.21.15.5:0/1863073146 conn(0x7f5e1c0fe090 msgr2=0x7f5e1c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 -- 172.21.15.5:0/1863073146 shutdown_connections 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 -- 172.21.15.5:0/1863073146 wait complete. 2024-04-22T13:25:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 Processor -- start 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 -- start start 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c102af0 0x7f5e1c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 0x7f5e1c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 0x7f5e1c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c10e580 con 0x7f5e1c104690 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c10e400 con 0x7f5e1c102af0 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.721+0000 7f5e23b23700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c10e700 con 0x7f5e1c103cf0 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 0x7f5e1c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e218bf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c102af0 0x7f5e1c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 0x7f5e1c1a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51848/0 (socket says 172.21.15.5:51848) 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e218bf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c102af0 0x7f5e1c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:32904/0 (socket says 172.21.15.5:32904) 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 -- 172.21.15.5:0/636329203 learned_addr learned my addr 172.21.15.5:0/636329203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e210be700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 0x7f5e1c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 -- 172.21.15.5:0/636329203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 msgr2=0x7f5e1c19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 0x7f5e1c19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 -- 172.21.15.5:0/636329203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c102af0 msgr2=0x7f5e1c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c102af0 0x7f5e1c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 -- 172.21.15.5:0/636329203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c1a29a0 con 0x7f5e1c104690 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e210be700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 0x7f5e1c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e220c0700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 0x7f5e1c1a2300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400add0 tx=0x7f5e1400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14017020 con 0x7f5e1c104690 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c1a2c90 con 0x7f5e1c104690 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1400ae00 con 0x7f5e1c104690 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e140053e0 con 0x7f5e1c104690 2024-04-22T13:25:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.722+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c1a3180 con 0x7f5e1c104690 2024-04-22T13:25:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.723+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c050f80 con 0x7f5e1c104690 2024-04-22T13:25:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.725+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f5e14004ad0 con 0x7f5e1c104690 2024-04-22T13:25:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.725+0000 7f5e12ffd700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5e080a3d80 0x7f5e080a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.725+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f5e140d00c0 con 0x7f5e1c104690 2024-04-22T13:25:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.725+0000 7f5e218bf700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5e080a3d80 0x7f5e080a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.726+0000 7f5e218bf700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5e080a3d80 0x7f5e080a6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e180038c0 tx=0x7f5e1800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:21.729+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e14021020 con 0x7f5e1c104690 2024-04-22T13:25:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.027+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f5e1c1a35e0 con 0x7f5e1c104690 2024-04-22T13:25:22.024 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:21 smithi005 ceph-mon[28006]: pgmap v292: 97 pgs: 97 active+clean; 620 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-04-22T13:25:22.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.029+0000 7f5e12ffd700 1 -- 172.21.15.5:0/636329203 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f5e1409cb80 con 0x7f5e1c104690 2024-04-22T13:25:22.026 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-22T13:25:22.026 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBCZSZmMme7ARAAP5HgLiov7pqNtussV/K42g== 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5e080a3d80 msgr2=0x7f5e080a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5e080a3d80 0x7f5e080a6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e180038c0 tx=0x7f5e1800b540 comp rx=0 tx=0).stop 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 msgr2=0x7f5e1c1a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 0x7f5e1c1a2300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400add0 tx=0x7f5e1400c330 comp rx=0 tx=0).stop 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 shutdown_connections 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5e080a3d80 0x7f5e080a6240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e1c102af0 0x7f5e1c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e1c104690 0x7f5e1c1a2300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 --2- 172.21.15.5:0/636329203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e1c103cf0 0x7f5e1c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 >> 172.21.15.5:0/636329203 conn(0x7f5e1c0fe090 msgr2=0x7f5e1c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 shutdown_connections 2024-04-22T13:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:22.032+0000 7f5e23b23700 1 -- 172.21.15.5:0/636329203 wait complete. 2024-04-22T13:25:22.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:21 smithi049 ceph-mon[31379]: pgmap v292: 97 pgs: 97 active+clean; 620 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-04-22T13:25:22.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:21 smithi107 ceph-mon[31859]: pgmap v292: 97 pgs: 97 active+clean; 620 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-04-22T13:25:22.673 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:25:22.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-22T13:25:22.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T13:25:22.712 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T13:25:22.712 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-22T13:25:22.712 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-22T13:25:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:23.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:22 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/636329203' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T13:25:23.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:22 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/636329203' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T13:25:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:22 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/636329203' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T13:25:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:22 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/636329203' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T13:25:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:22 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/636329203' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T13:25:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:22 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/636329203' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T13:25:24.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:24 smithi005 ceph-mon[28006]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-22T13:25:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:24 smithi107 ceph-mon[31859]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 -- 172.21.15.5:0/1567891779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244102ac0 msgr2=0x7f8244102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 --2- 172.21.15.5:0/1567891779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244102ac0 0x7f8244102f40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8238009990 tx=0x7f8238009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 -- 172.21.15.5:0/1567891779 shutdown_connections 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 --2- 172.21.15.5:0/1567891779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8244108a70 0x7f8244108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 --2- 172.21.15.5:0/1567891779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244103480 0x7f824410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 --2- 172.21.15.5:0/1567891779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244102ac0 0x7f8244102f40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 -- 172.21.15.5:0/1567891779 >> 172.21.15.5:0/1567891779 conn(0x7f8244079bb0 msgr2=0x7f8244079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 -- 172.21.15.5:0/1567891779 shutdown_connections 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.505+0000 7f8249915700 1 -- 172.21.15.5:0/1567891779 wait complete. 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.506+0000 7f8249915700 1 Processor -- start 2024-04-22T13:25:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.506+0000 7f8249915700 1 -- start start 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8244102ac0 0x7f8244199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244103480 0x7f82441996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 0x7f824419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112c40 con 0x7f8244108a70 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112ac0 con 0x7f8244102ac0 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112dc0 con 0x7f8244103480 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 0x7f824419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 0x7f824419da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51860/0 (socket says 172.21.15.5:51860) 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 -- 172.21.15.5:0/2705323280 learned_addr learned my addr 172.21.15.5:0/2705323280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82427fc700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244103480 0x7f82441996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8242ffd700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8244102ac0 0x7f8244199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 -- 172.21.15.5:0/2705323280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244103480 msgr2=0x7f82441996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244103480 0x7f82441996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 -- 172.21.15.5:0/2705323280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8244102ac0 msgr2=0x7f8244199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8244102ac0 0x7f8244199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 -- 172.21.15.5:0/2705323280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824419e180 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f82437fe700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 0x7f824419da40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f823c00d980 tx=0x7f823c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c017070 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823c005020 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.507+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824419e470 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.508+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0099b0 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.508+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a5cf0 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.509+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244069730 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.510+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f823c004a30 con 0x7f8244108a70 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.510+0000 7f8248913700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f822c0a3cd0 0x7f822c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.511+0000 7f8242ffd700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f822c0a3cd0 0x7f822c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.511+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f823c0d0260 con 0x7f8244108a70 2024-04-22T13:25:24.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.511+0000 7f8242ffd700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f822c0a3cd0 0x7f822c0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8234009c90 tx=0x7f8234009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:24.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.515+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f823c020020 con 0x7f8244108a70 2024-04-22T13:25:24.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:24 smithi049 ceph-mon[31379]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-22T13:25:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.781+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f824402df80 con 0x7f8244108a70 2024-04-22T13:25:24.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.786+0000 7f8248913700 1 -- 172.21.15.5:0/2705323280 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264988 (secure 0 0 0) 0x7f823c09cc90 con 0x7f8244108a70 2024-04-22T13:25:24.787 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f822c0a3cd0 msgr2=0x7f822c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f822c0a3cd0 0x7f822c0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8234009c90 tx=0x7f8234009040 comp rx=0 tx=0).stop 2024-04-22T13:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 msgr2=0x7f824419da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 0x7f824419da40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f823c00d980 tx=0x7f823c00dc90 comp rx=0 tx=0).stop 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 shutdown_connections 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f822c0a3cd0 0x7f822c0a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8244102ac0 0x7f8244199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8244108a70 0x7f824419da40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 --2- 172.21.15.5:0/2705323280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244103480 0x7f82441996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 >> 172.21.15.5:0/2705323280 conn(0x7f8244079bb0 msgr2=0x7f82441017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 shutdown_connections 2024-04-22T13:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:24.789+0000 7f8249915700 1 -- 172.21.15.5:0/2705323280 wait complete. 2024-04-22T13:25:25.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:25 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2705323280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T13:25:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:25 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2705323280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T13:25:25.432 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":2674774949},{"type":"v1","addr":"172.21.15.5:6801","nonce":2674774949}]},"active_addr":"172.21.15.5:6801/2674774949","active_change":"2024-04-22T13:18:14.099043+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1240644936}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3837385355}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3327695745}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2682698140}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":987482747}]}]} 2024-04-22T13:25:25.438 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-22T13:25:25.438 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-22T13:25:25.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T13:25:25.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:25 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2705323280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T13:25:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:26.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:26 smithi005 ceph-mon[28006]: pgmap v294: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-04-22T13:25:26.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:26 smithi107 ceph-mon[31859]: pgmap v294: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-04-22T13:25:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:26 smithi049 ceph-mon[31379]: pgmap v294: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 -- 172.21.15.5:0/1421660356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 msgr2=0x7fb978107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 --2- 172.21.15.5:0/1421660356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb978107d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb960009990 tx=0x7fb960009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 -- 172.21.15.5:0/1421660356 shutdown_connections 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 --2- 172.21.15.5:0/1421660356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978102300 0x7fb97810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 --2- 172.21.15.5:0/1421660356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb978107d70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 --2- 172.21.15.5:0/1421660356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978101940 0x7fb978101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 -- 172.21.15.5:0/1421660356 >> 172.21.15.5:0/1421660356 conn(0x7fb9780788f0 msgr2=0x7fb978078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 -- 172.21.15.5:0/1421660356 shutdown_connections 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.321+0000 7fb97ed64700 1 -- 172.21.15.5:0/1421660356 wait complete. 2024-04-22T13:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 Processor -- start 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 -- start start 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 0x7fb978199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978102300 0x7fb978199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb97819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb978111b30 con 0x7fb978107990 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb9781119b0 con 0x7fb978102300 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97ed64700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb978111cb0 con 0x7fb978101940 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 0x7fb978199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 0x7fb978199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:40186/0 (socket says 172.21.15.5:40186) 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 -- 172.21.15.5:0/3798492980 learned_addr learned my addr 172.21.15.5:0/3798492980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97d301700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb97819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb977fff700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978102300 0x7fb978199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 -- 172.21.15.5:0/3798492980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978102300 msgr2=0x7fb978199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978102300 0x7fb978199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 -- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 msgr2=0x7fb97819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb97819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97cb00700 1 -- 172.21.15.5:0/3798492980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97819e210 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.322+0000 7fb97d301700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb97819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.323+0000 7fb97cb00700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 0x7fb978199200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb960005910 tx=0x7fb96000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.323+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960027070 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.323+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb960022470 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.323+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97819e4a0 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.323+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960017650 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.323+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9781a5d80 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.324+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb978069730 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.325+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fb96002a070 con 0x7fb978101940 2024-04-22T13:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.326+0000 7fb975ffb700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb9640a3cd0 0x7fb9640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.326+0000 7fb977fff700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb9640a3cd0 0x7fb9640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.326+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb9600d1490 con 0x7fb978101940 2024-04-22T13:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.326+0000 7fb977fff700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb9640a3cd0 0x7fb9640a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb978078f30 tx=0x7fb96800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.334+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb960013070 con 0x7fb978101940 2024-04-22T13:25:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.544+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9781a5f50 con 0x7fb978101940 2024-04-22T13:25:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.546+0000 7fb975ffb700 1 -- 172.21.15.5:0/3798492980 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22115 (secure 0 0 0) 0x7fb96009de50 con 0x7fb978101940 2024-04-22T13:25:27.545 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:27.545 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:25:11.887809+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:25:27.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb9640a3cd0 msgr2=0x7fb9640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb9640a3cd0 0x7fb9640a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb978078f30 tx=0x7fb96800b410 comp rx=0 tx=0).stop 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 msgr2=0x7fb978199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 0x7fb978199200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb960005910 tx=0x7fb96000b670 comp rx=0 tx=0).stop 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 shutdown_connections 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb978107990 0x7fb97819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb978101940 0x7fb978199200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb9640a3cd0 0x7fb9640a6190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 --2- 172.21.15.5:0/3798492980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb978102300 0x7fb978199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 >> 172.21.15.5:0/3798492980 conn(0x7fb9780788f0 msgr2=0x7fb97806a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 shutdown_connections 2024-04-22T13:25:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:27.548+0000 7fb97ed64700 1 -- 172.21.15.5:0/3798492980 wait complete. 2024-04-22T13:25:28.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:28 smithi005 ceph-mon[28006]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-22T13:25:28.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:28 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3798492980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:28.323 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:28 smithi049 ceph-mon[31379]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-22T13:25:28.323 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:28 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3798492980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:28.367 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-22T13:25:28.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T13:25:28.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:28 smithi107 ceph-mon[31859]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-22T13:25:28.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:28 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3798492980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:28.622 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 -- 172.21.15.5:0/2413279809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc081026e0 msgr2=0x7efc0810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 --2- 172.21.15.5:0/2413279809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc081026e0 0x7efc0810e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc0000b0a0 tx=0x7efc0000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 -- 172.21.15.5:0/2413279809 shutdown_connections 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 --2- 172.21.15.5:0/2413279809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc081026e0 0x7efc0810e770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 --2- 172.21.15.5:0/2413279809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc08106990 0x7efc08106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 --2- 172.21.15.5:0/2413279809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08101d20 0x7efc081021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 -- 172.21.15.5:0/2413279809 >> 172.21.15.5:0/2413279809 conn(0x7efc080788f0 msgr2=0x7efc08078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 -- 172.21.15.5:0/2413279809 shutdown_connections 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 -- 172.21.15.5:0/2413279809 wait complete. 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.281+0000 7efc0fc82700 1 Processor -- start 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 -- start start 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 0x7efc08197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc081026e0 0x7efc08198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08106990 0x7efc0819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc08111ae0 con 0x7efc08101d20 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc08111960 con 0x7efc08106990 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0fc82700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc08111c60 con 0x7efc081026e0 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 0x7efc08197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 0x7efc08197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50784/0 (socket says 172.21.15.5:50784) 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 -- 172.21.15.5:0/627351739 learned_addr learned my addr 172.21.15.5:0/627351739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0d21d700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc081026e0 0x7efc08198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0e21f700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08106990 0x7efc0819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 -- 172.21.15.5:0/627351739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc081026e0 msgr2=0x7efc08198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc081026e0 0x7efc08198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 -- 172.21.15.5:0/627351739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08106990 msgr2=0x7efc0819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08106990 0x7efc0819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 -- 172.21.15.5:0/627351739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0819cf00 con 0x7efc08101d20 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0d21d700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc081026e0 0x7efc08198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0e21f700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08106990 0x7efc0819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.282+0000 7efc0da1e700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 0x7efc08197ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc0400d950 tx=0x7efc0400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.283+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04016070 con 0x7efc08101d20 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.283+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc04010460 con 0x7efc08101d20 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.283+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0819d1f0 con 0x7efc08101d20 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.283+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0400f620 con 0x7efc08101d20 2024-04-22T13:25:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.283+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0806b110 con 0x7efc08101d20 2024-04-22T13:25:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.284+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc08103080 con 0x7efc08101d20 2024-04-22T13:25:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.285+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7efc04019070 con 0x7efc08101d20 2024-04-22T13:25:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.286+0000 7efbfeffd700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efbf40a3d50 0x7efbf40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.286+0000 7efc0d21d700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efbf40a3d50 0x7efbf40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.286+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7efc040d00b0 con 0x7efc08101d20 2024-04-22T13:25:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.286+0000 7efc0d21d700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efbf40a3d50 0x7efbf40a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005950 tx=0x7efbf80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.289+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc0409c9e0 con 0x7efc08101d20 2024-04-22T13:25:30.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:30 smithi005 ceph-mon[28006]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.2 KiB/s wr, 22 op/s 2024-04-22T13:25:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.499+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc0806bb40 con 0x7efc08101d20 2024-04-22T13:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.501+0000 7efbfeffd700 1 -- 172.21.15.5:0/627351739 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22115 (secure 0 0 0) 0x7efc0409c2e0 con 0x7efc08101d20 2024-04-22T13:25:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:25:30.499 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:25:11.887809+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efbf40a3d50 msgr2=0x7efbf40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efbf40a3d50 0x7efbf40a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005950 tx=0x7efbf80058e0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 msgr2=0x7efc08197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 0x7efc08197ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc0400d950 tx=0x7efc0400dd10 comp rx=0 tx=0).stop 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 shutdown_connections 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efbf40a3d50 0x7efbf40a6210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc08106990 0x7efc0819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc08101d20 0x7efc08197ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 --2- 172.21.15.5:0/627351739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc081026e0 0x7efc08198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.503+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 >> 172.21.15.5:0/627351739 conn(0x7efc080788f0 msgr2=0x7efc080ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.504+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 shutdown_connections 2024-04-22T13:25:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:30.504+0000 7efc0fc82700 1 -- 172.21.15.5:0/627351739 wait complete. 2024-04-22T13:25:30.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:30 smithi049 ceph-mon[31379]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.2 KiB/s wr, 22 op/s 2024-04-22T13:25:30.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:30 smithi107 ceph-mon[31859]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.2 KiB/s wr, 22 op/s 2024-04-22T13:25:31.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-22T13:25:31.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-22T13:25:31.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-22T13:25:31.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-22T13:25:31.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-22T13:25:31.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-22T13:25:31.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-22T13:25:31.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-22T13:25:31.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-22T13:25:31.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-22T13:25:31.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-22T13:25:31.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-22T13:25:31.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:31 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/627351739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:31.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:31 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/627351739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:31.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:31 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/627351739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:25:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:31.963 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:31.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:32 smithi005 ceph-mon[28006]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-04-22T13:25:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:32.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:32 smithi049 ceph-mon[31379]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-04-22T13:25:32.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:32 smithi107 ceph-mon[31859]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-04-22T13:25:34.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:34 smithi005 ceph-mon[28006]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-22T13:25:34.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:34 smithi049 ceph-mon[31379]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-22T13:25:34.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:34 smithi107 ceph-mon[31859]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-22T13:25:36.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:36 smithi005 ceph-mon[28006]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-22T13:25:36.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:36 smithi049 ceph-mon[31379]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-22T13:25:36.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:36 smithi107 ceph-mon[31859]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-22T13:25:38.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:38 smithi005 ceph-mon[28006]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-22T13:25:38.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:38 smithi049 ceph-mon[31379]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-22T13:25:38.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:38 smithi107 ceph-mon[31859]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-22T13:25:40.203 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:40 smithi005 ceph-mon[28006]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-22T13:25:40.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:40 smithi049 ceph-mon[31379]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-22T13:25:40.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:40 smithi107 ceph-mon[31859]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-22T13:25:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 -- 172.21.15.5:0/2891574089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 msgr2=0x7f966c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 --2- 172.21.15.5:0/2891574089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c108e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f965c009990 tx=0x7f965c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 -- 172.21.15.5:0/2891574089 shutdown_connections 2024-04-22T13:25:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 --2- 172.21.15.5:0/2891574089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f966c103480 0x7f966c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 --2- 172.21.15.5:0/2891574089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c108e90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 --2- 172.21.15.5:0/2891574089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c102ac0 0x7f966c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.803+0000 7f9672902700 1 -- 172.21.15.5:0/2891574089 >> 172.21.15.5:0/2891574089 conn(0x7f966c079bf0 msgr2=0x7f966c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.804+0000 7f9672902700 1 -- 172.21.15.5:0/2891574089 shutdown_connections 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.804+0000 7f9672902700 1 -- 172.21.15.5:0/2891574089 wait complete. 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.804+0000 7f9672902700 1 Processor -- start 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.804+0000 7f9672902700 1 -- start start 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9672902700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c102ac0 0x7f966c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9672902700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f966c103480 0x7f966c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9672902700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9672902700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f966c112ca0 con 0x7f966c108ab0 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9672902700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f966c112b20 con 0x7f966c102ac0 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9672902700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f966c112e20 con 0x7f966c103480 2024-04-22T13:25:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40326/0 (socket says 172.21.15.5:40326) 2024-04-22T13:25:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 -- 172.21.15.5:0/1317699109 learned_addr learned my addr 172.21.15.5:0/1317699109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9663fff700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f966c103480 0x7f966c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f966bfff700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c102ac0 0x7f966c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 -- 172.21.15.5:0/1317699109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f966c103480 msgr2=0x7f966c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f966c103480 0x7f966c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 -- 172.21.15.5:0/1317699109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c102ac0 msgr2=0x7f966c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c102ac0 0x7f966c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 -- 172.21.15.5:0/1317699109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c19e180 con 0x7f966c108ab0 2024-04-22T13:25:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.805+0000 7f9670e9f700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c19da40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9664007ee0 tx=0x7f966400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.806+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966400dbd0 con 0x7f966c108ab0 2024-04-22T13:25:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.806+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f966400dd70 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.806+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c19e470 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.806+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664005990 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.806+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c1a5cf0 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.806+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f966c063150 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.808+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f966401a020 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.808+0000 7f9669ffb700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96580a3cd0 0x7f96580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f966bfff700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96580a3cd0 0x7f96580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f96640d0200 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f9669ffb700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] conn(0x7f96580ad4b0 0x7f96580af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 --> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9664021e70 con 0x7f96580ad4b0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f96640d06e0 con 0x7f966c108ab0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f9663fff700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] conn(0x7f96580ad4b0 0x7f96580af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f966bfff700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96580a3cd0 0x7f96580a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f965c006010 tx=0x7f965c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.809+0000 7f9663fff700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] conn(0x7f96580ad4b0 0x7f96580af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.810+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== osd.9 v2:172.21.15.5:6826/3613366882 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9664021e70 con 0x7f96580ad4b0 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 --> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f966c0635d0 con 0x7f96580ad4b0 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9669ffb700 1 -- 172.21.15.5:0/1317699109 <== osd.9 v2:172.21.15.5:6826/3613366882 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f966c0635d0 con 0x7f96580ad4b0 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] conn(0x7f96580ad4b0 msgr2=0x7f96580af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] conn(0x7f96580ad4b0 0x7f96580af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96580a3cd0 msgr2=0x7f96580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96580a3cd0 0x7f96580a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f965c006010 tx=0x7f965c01a040 comp rx=0 tx=0).stop 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 msgr2=0x7f966c19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 --2- 172.21.15.5:0/1317699109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f966c108ab0 0x7f966c19da40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9664007ee0 tx=0x7f966400d3b0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9670e9f700 1 -- 172.21.15.5:0/1317699109 reap_dead start 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 shutdown_connections 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 >> 172.21.15.5:0/1317699109 conn(0x7f966c079bf0 msgr2=0x7f966c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.827+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 shutdown_connections 2024-04-22T13:25:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:40.828+0000 7f9672902700 1 -- 172.21.15.5:0/1317699109 wait complete. 2024-04-22T13:25:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 -- 172.21.15.5:0/2565229352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06a750 msgr2=0x7fa25c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2565229352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06a750 0x7fa25c06abb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009990 tx=0x7fa24c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 -- 172.21.15.5:0/2565229352 shutdown_connections 2024-04-22T13:25:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2565229352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa25c108a30 0x7fa25c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2565229352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c06b0f0 0x7fa25c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2565229352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06a750 0x7fa25c06abb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 -- 172.21.15.5:0/2565229352 >> 172.21.15.5:0/2565229352 conn(0x7fa25c0788f0 msgr2=0x7fa25c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 -- 172.21.15.5:0/2565229352 shutdown_connections 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.691+0000 7fa263ad0700 1 -- 172.21.15.5:0/2565229352 wait complete. 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 Processor -- start 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 -- start start 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa25c06a750 0x7fa25c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06b0f0 0x7fa25c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c108a30 0x7fa25c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112c10 con 0x7fa25c06b0f0 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112a90 con 0x7fa25c06a750 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa263ad0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112d90 con 0x7fa25c108a30 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa26106b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06b0f0 0x7fa25c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa26106b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06b0f0 0x7fa25c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40344/0 (socket says 172.21.15.5:40344) 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa26106b700 1 -- 172.21.15.5:0/2146828706 learned_addr learned my addr 172.21.15.5:0/2146828706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa26186c700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa25c06a750 0x7fa25c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.692+0000 7fa26206d700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c108a30 0x7fa25c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26106b700 1 -- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c108a30 msgr2=0x7fa25c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26106b700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c108a30 0x7fa25c19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26106b700 1 -- 172.21.15.5:0/2146828706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa25c06a750 msgr2=0x7fa25c199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26106b700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa25c06a750 0x7fa25c199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26106b700 1 -- 172.21.15.5:0/2146828706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c19e150 con 0x7fa25c06b0f0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26186c700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa25c06a750 0x7fa25c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26206d700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa25c108a30 0x7fa25c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa26106b700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06b0f0 0x7fa25c199680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa24c000c00 tx=0x7fa24c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c027070 con 0x7fa25c06b0f0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa24c022470 con 0x7fa25c06b0f0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c017650 con 0x7fa25c06b0f0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c19e3e0 con 0x7fa25c06b0f0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.693+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c101940 con 0x7fa25c06b0f0 2024-04-22T13:25:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.694+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa25c0631d0 con 0x7fa25c06b0f0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.695+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fa24c02a070 con 0x7fa25c06b0f0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa252ffd700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa2480a3d50 0x7fa2480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa26186c700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa2480a3d50 0x7fa2480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa24c0d1a50 con 0x7fa25c06b0f0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa252ffd700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] conn(0x7fa2480ad530 0x7fa2480af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 --> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa24c00b0b0 con 0x7fa2480ad530 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa24c0d1f30 con 0x7fa25c06b0f0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.696+0000 7fa26186c700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa2480a3d50 0x7fa2480a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa258005fd0 tx=0x7fa258005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.697+0000 7fa26206d700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] conn(0x7fa2480ad530 0x7fa2480af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.697+0000 7fa26206d700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] conn(0x7fa2480ad530 0x7fa2480af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:41.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.698+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== osd.11 v2:172.21.15.107:6824/1856713900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa24c00b0b0 con 0x7fa2480ad530 2024-04-22T13:25:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.714+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 --> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa25c0635d0 con 0x7fa2480ad530 2024-04-22T13:25:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa252ffd700 1 -- 172.21.15.5:0/2146828706 <== osd.11 v2:172.21.15.107:6824/1856713900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa25c0635d0 con 0x7fa2480ad530 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] conn(0x7fa2480ad530 msgr2=0x7fa2480af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] conn(0x7fa2480ad530 0x7fa2480af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa2480a3d50 msgr2=0x7fa2480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa2480a3d50 0x7fa2480a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa258005fd0 tx=0x7fa258005db0 comp rx=0 tx=0).stop 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06b0f0 msgr2=0x7fa25c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa263ad0700 1 --2- 172.21.15.5:0/2146828706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa25c06b0f0 0x7fa25c199680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa24c000c00 tx=0x7fa24c009580 comp rx=0 tx=0).stop 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.715+0000 7fa26206d700 1 -- 172.21.15.5:0/2146828706 reap_dead start 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.716+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 shutdown_connections 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.716+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 >> 172.21.15.5:0/2146828706 conn(0x7fa25c0788f0 msgr2=0x7fa25c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.716+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 shutdown_connections 2024-04-22T13:25:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:41.716+0000 7fa263ad0700 1 -- 172.21.15.5:0/2146828706 wait complete. 2024-04-22T13:25:42.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:42 smithi005 ceph-mon[28006]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-22T13:25:42.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:42 smithi049 ceph-mon[31379]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-22T13:25:42.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:42 smithi107 ceph-mon[31859]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-22T13:25:43.259 INFO:teuthology.orchestra.run.smithi005.stdout:360777252882 2024-04-22T13:25:43.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-22T13:25:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 -- 172.21.15.5:0/170142203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc102300 msgr2=0x7f50fc10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 --2- 172.21.15.5:0/170142203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc102300 0x7f50fc10e7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009990 tx=0x7f50ec009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 -- 172.21.15.5:0/170142203 shutdown_connections 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 --2- 172.21.15.5:0/170142203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50fc107990 0x7f50fc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 --2- 172.21.15.5:0/170142203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc102300 0x7f50fc10e7f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 --2- 172.21.15.5:0/170142203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50fc101940 0x7f50fc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 -- 172.21.15.5:0/170142203 >> 172.21.15.5:0/170142203 conn(0x7f50fc0788f0 msgr2=0x7f50fc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 -- 172.21.15.5:0/170142203 shutdown_connections 2024-04-22T13:25:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.479+0000 7f510203e700 1 -- 172.21.15.5:0/170142203 wait complete. 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 Processor -- start 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 -- start start 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50fc101940 0x7f50fc199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50fc102300 0x7f50fc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc107990 0x7f50fc1a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc111af0 con 0x7f50fc107990 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc111970 con 0x7f50fc101940 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f510203e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc111c70 con 0x7f50fc102300 2024-04-22T13:25:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f50fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc107990 0x7f50fc1a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f50fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc107990 0x7f50fc1a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40354/0 (socket says 172.21.15.5:40354) 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.480+0000 7f50fbfff700 1 -- 172.21.15.5:0/3303502534 learned_addr learned my addr 172.21.15.5:0/3303502534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50faffd700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50fc102300 0x7f50fc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fb7fe700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50fc101940 0x7f50fc199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fbfff700 1 -- 172.21.15.5:0/3303502534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50fc102300 msgr2=0x7f50fc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fbfff700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50fc102300 0x7f50fc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fbfff700 1 -- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50fc101940 msgr2=0x7f50fc199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fbfff700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50fc101940 0x7f50fc199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fbfff700 1 -- 172.21.15.5:0/3303502534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc1a5510 con 0x7f50fc107990 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50fbfff700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc107990 0x7f50fc1a4fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c010 tx=0x7f50ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec027070 con 0x7f50fc107990 2024-04-22T13:25:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ec022470 con 0x7f50fc107990 2024-04-22T13:25:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1a56e0 con 0x7f50fc107990 2024-04-22T13:25:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec017650 con 0x7f50fc107990 2024-04-22T13:25:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.481+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1a5b30 con 0x7f50fc107990 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.482+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50fc063150 con 0x7f50fc107990 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.483+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f50ec02a070 con 0x7f50fc107990 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.484+0000 7f50f8ff9700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f50e80a3d50 0x7f50e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.484+0000 7f50fb7fe700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f50e80a3d50 0x7f50e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.484+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f50ec0d1120 con 0x7f50fc107990 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.485+0000 7f50f8ff9700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] conn(0x7f50e80ad530 0x7f50e80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.485+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 --> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50ec01fec0 con 0x7f50e80ad530 2024-04-22T13:25:43.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.485+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f50ec0d8050 con 0x7f50fc107990 2024-04-22T13:25:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.485+0000 7f50fb7fe700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f50e80a3d50 0x7f50e80a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009550 tx=0x7f50f40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.485+0000 7f50faffd700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] conn(0x7f50e80ad530 0x7f50e80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.485+0000 7f50faffd700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] conn(0x7f50e80ad530 0x7f50e80af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.486+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== osd.1 v2:172.21.15.49:6800/3148163706 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50ec01fec0 con 0x7f50e80ad530 2024-04-22T13:25:43.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.504+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 --> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50fc063600 con 0x7f50e80ad530 2024-04-22T13:25:43.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f50f8ff9700 1 -- 172.21.15.5:0/3303502534 <== osd.1 v2:172.21.15.49:6800/3148163706 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f50fc063600 con 0x7f50e80ad530 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] conn(0x7f50e80ad530 msgr2=0x7f50e80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] conn(0x7f50e80ad530 0x7f50e80af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f50e80a3d50 msgr2=0x7f50e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f50e80a3d50 0x7f50e80a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009550 tx=0x7f50f40091b0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc107990 msgr2=0x7f50fc1a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 --2- 172.21.15.5:0/3303502534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc107990 0x7f50fc1a4fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c010 tx=0x7f50ec0047b0 comp rx=0 tx=0).stop 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f50fbfff700 1 -- 172.21.15.5:0/3303502534 reap_dead start 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 shutdown_connections 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 >> 172.21.15.5:0/3303502534 conn(0x7f50fc0788f0 msgr2=0x7f50fc0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 shutdown_connections 2024-04-22T13:25:43.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:43.505+0000 7f510203e700 1 -- 172.21.15.5:0/3303502534 wait complete. 2024-04-22T13:25:44.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:44 smithi005 ceph-mon[28006]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:44.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:44 smithi049 ceph-mon[31379]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:44.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:44 smithi107 ceph-mon[31859]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:44.663 INFO:teuthology.orchestra.run.smithi005.stdout:304942678045 2024-04-22T13:25:44.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-22T13:25:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 -- 172.21.15.5:0/695399354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740102a90 msgr2=0x7fe740102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 --2- 172.21.15.5:0/695399354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740102a90 0x7fe740102f10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe730009990 tx=0x7fe730009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 -- 172.21.15.5:0/695399354 shutdown_connections 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 --2- 172.21.15.5:0/695399354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe740103450 0x7fe74010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 --2- 172.21.15.5:0/695399354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe740108a50 0x7fe740108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 --2- 172.21.15.5:0/695399354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740102a90 0x7fe740102f10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 -- 172.21.15.5:0/695399354 >> 172.21.15.5:0/695399354 conn(0x7fe740079ba0 msgr2=0x7fe740079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 -- 172.21.15.5:0/695399354 shutdown_connections 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.472+0000 7fe74827f700 1 -- 172.21.15.5:0/695399354 wait complete. 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 Processor -- start 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 -- start start 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe740102a90 0x7fe7401991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe740103450 0x7fe7401996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740108a50 0x7fe74019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112c80 con 0x7fe740108a50 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112b00 con 0x7fe740103450 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74827f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112e00 con 0x7fe740102a90 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.473+0000 7fe74681c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740108a50 0x7fe74019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740108a50 0x7fe74019da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40384/0 (socket says 172.21.15.5:40384) 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 -- 172.21.15.5:0/237337281 learned_addr learned my addr 172.21.15.5:0/237337281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74581a700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe740103450 0x7fe7401996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74601b700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe740102a90 0x7fe7401991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 -- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe740102a90 msgr2=0x7fe7401991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe740102a90 0x7fe7401991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 -- 172.21.15.5:0/237337281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe740103450 msgr2=0x7fe7401996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe740103450 0x7fe7401996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 -- 172.21.15.5:0/237337281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74019e1c0 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74681c700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740108a50 0x7fe74019da80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b970 tx=0x7fe73800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800d860 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73800e430 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74019e4b0 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738017660 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.474+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7401a5d30 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.475+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe740063190 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.476+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fe73800da00 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe7377fe700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe72c0a3cd0 0x7fe72c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe74601b700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe72c0a3cd0 0x7fe72c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fe7380d0590 con 0x7fe740108a50 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe7377fe700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] conn(0x7fe72c0ad4b0 0x7fe72c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 --> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe72c0aff80 con 0x7fe72c0ad4b0 2024-04-22T13:25:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe7380d0a70 con 0x7fe740108a50 2024-04-22T13:25:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe74601b700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe72c0a3cd0 0x7fe72c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe73c004630 tx=0x7fe73c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.477+0000 7fe74581a700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] conn(0x7fe72c0ad4b0 0x7fe72c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.478+0000 7fe74581a700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] conn(0x7fe72c0ad4b0 0x7fe72c0af8d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.479+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== osd.8 v2:172.21.15.107:6816/4036561619 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe72c0aff80 con 0x7fe72c0ad4b0 2024-04-22T13:25:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.495+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 --> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe740063610 con 0x7fe72c0ad4b0 2024-04-22T13:25:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe7377fe700 1 -- 172.21.15.5:0/237337281 <== osd.8 v2:172.21.15.107:6816/4036561619 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe740063610 con 0x7fe72c0ad4b0 2024-04-22T13:25:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] conn(0x7fe72c0ad4b0 msgr2=0x7fe72c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] conn(0x7fe72c0ad4b0 0x7fe72c0af8d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe72c0a3cd0 msgr2=0x7fe72c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe72c0a3cd0 0x7fe72c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe73c004630 tx=0x7fe73c00b3c0 comp rx=0 tx=0).stop 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740108a50 msgr2=0x7fe74019da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 --2- 172.21.15.5:0/237337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe740108a50 0x7fe74019da80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b970 tx=0x7fe73800bc80 comp rx=0 tx=0).stop 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74681c700 1 -- 172.21.15.5:0/237337281 reap_dead start 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 shutdown_connections 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 >> 172.21.15.5:0/237337281 conn(0x7fe740079ba0 msgr2=0x7fe740101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 shutdown_connections 2024-04-22T13:25:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:45.496+0000 7fe74827f700 1 -- 172.21.15.5:0/237337281 wait complete. 2024-04-22T13:25:46.177 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:46 smithi005 ceph-mon[28006]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-22T13:25:46.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:46 smithi049 ceph-mon[31379]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-22T13:25:46.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:46 smithi107 ceph-mon[31859]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-22T13:25:46.810 INFO:teuthology.orchestra.run.smithi005.stdout:270582939684 2024-04-22T13:25:46.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-22T13:25:47.950 INFO:teuthology.orchestra.run.smithi005.stdout:55834574916 2024-04-22T13:25:47.951 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-22T13:25:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 -- 172.21.15.5:0/954341668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c108a20 msgr2=0x7f6e9c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/954341668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c108a20 0x7f6e9c108e00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009990 tx=0x7f6e98009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 -- 172.21.15.5:0/954341668 shutdown_connections 2024-04-22T13:25:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/954341668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e9c06b980 0x7f6e9c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/954341668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c108a20 0x7f6e9c108e00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/954341668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e9c06afe0 0x7f6e9c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 -- 172.21.15.5:0/954341668 >> 172.21.15.5:0/954341668 conn(0x7f6e9c0788b0 msgr2=0x7f6e9c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 -- 172.21.15.5:0/954341668 shutdown_connections 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.520+0000 7f6ea35c3700 1 -- 172.21.15.5:0/954341668 wait complete. 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 Processor -- start 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 -- start start 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c06afe0 0x7f6e9c101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e9c06b980 0x7f6e9c101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e9c108a20 0x7f6e9c102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c112c20 con 0x7f6e9c06afe0 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c112aa0 con 0x7f6e9c108a20 2024-04-22T13:25:48.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea35c3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c112da0 con 0x7f6e9c06b980 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.521+0000 7f6ea135f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c06afe0 0x7f6e9c101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c06afe0 0x7f6e9c101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40394/0 (socket says 172.21.15.5:40394) 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 -- 172.21.15.5:0/2575274422 learned_addr learned my addr 172.21.15.5:0/2575274422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea0b5e700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e9c06b980 0x7f6e9c101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea1b60700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e9c108a20 0x7f6e9c102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 -- 172.21.15.5:0/2575274422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e9c06b980 msgr2=0x7f6e9c101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e9c06b980 0x7f6e9c101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 -- 172.21.15.5:0/2575274422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e9c108a20 msgr2=0x7f6e9c102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e9c108a20 0x7f6e9c102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 -- 172.21.15.5:0/2575274422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c1a76c0 con 0x7f6e9c06afe0 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea1b60700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e9c108a20 0x7f6e9c102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea135f700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c06afe0 0x7f6e9c101a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e98000c00 tx=0x7f6e980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:48.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98027070 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c1a7950 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e980041d0 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800f460 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.522+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c1a7db0 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.523+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e9c0630d0 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.524+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f6e9802a070 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6e927fc700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e880a3cd0 0x7f6e880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6ea0b5e700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e880a3cd0 0x7f6e880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f6e980d1ab0 con 0x7f6e9c06afe0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6e927fc700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] conn(0x7f6e880ad4b0 0x7f6e880af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 --> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e980043d0 con 0x7f6e880ad4b0 2024-04-22T13:25:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f6e980d1f90 con 0x7f6e9c06afe0 2024-04-22T13:25:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6ea0b5e700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e880a3cd0 0x7f6e880a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005ea0 tx=0x7f6e8c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.525+0000 7f6ea1b60700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] conn(0x7f6e880ad4b0 0x7f6e880af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.526+0000 7f6ea1b60700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] conn(0x7f6e880ad4b0 0x7f6e880af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.526+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== osd.0 v2:172.21.15.5:6802/236509498 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e980043d0 con 0x7f6e880ad4b0 2024-04-22T13:25:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 --> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e9c0634d0 con 0x7f6e880ad4b0 2024-04-22T13:25:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6e927fc700 1 -- 172.21.15.5:0/2575274422 <== osd.0 v2:172.21.15.5:6802/236509498 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6e9c0634d0 con 0x7f6e880ad4b0 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] conn(0x7f6e880ad4b0 msgr2=0x7f6e880af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] conn(0x7f6e880ad4b0 0x7f6e880af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e880a3cd0 msgr2=0x7f6e880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6e880a3cd0 0x7f6e880a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005ea0 tx=0x7f6e8c005e30 comp rx=0 tx=0).stop 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c06afe0 msgr2=0x7f6e9c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 --2- 172.21.15.5:0/2575274422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e9c06afe0 0x7f6e9c101a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e98000c00 tx=0x7f6e980047b0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea1b60700 1 -- 172.21.15.5:0/2575274422 reap_dead start 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 shutdown_connections 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.543+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 >> 172.21.15.5:0/2575274422 conn(0x7f6e9c0788b0 msgr2=0x7f6e9c0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.544+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 shutdown_connections 2024-04-22T13:25:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:46.544+0000 7f6ea35c3700 1 -- 172.21.15.5:0/2575274422 wait complete. 2024-04-22T13:25:48.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:48 smithi005 ceph-mon[28006]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:48.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:48 smithi049 ceph-mon[31379]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:48.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:48 smithi107 ceph-mon[31859]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:48.796 INFO:teuthology.orchestra.run.smithi005.stdout:34359738442 2024-04-22T13:25:48.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-22T13:25:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/786196616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a541022f0 msgr2=0x7f0a5410e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/786196616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a541022f0 0x7f0a5410e7e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009990 tx=0x7f0a44009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/786196616 shutdown_connections 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/786196616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a54107980 0x7f0a54107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/786196616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a541022f0 0x7f0a5410e7e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/786196616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54101930 0x7f0a54101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/786196616 >> 172.21.15.5:0/786196616 conn(0x7f0a540788b0 msgr2=0x7f0a54078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/786196616 shutdown_connections 2024-04-22T13:25:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.956+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/786196616 wait complete. 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 Processor -- start 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 -- start start 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a54101930 0x7f0a54199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a541022f0 0x7f0a5419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a54107980 0x7f0a541a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54111ad0 con 0x7f0a54101930 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54111950 con 0x7f0a54107980 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5b2a2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54111c50 con 0x7f0a541022f0 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5903e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a54101930 0x7f0a54199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5903e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a54101930 0x7f0a54199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60358/0 (socket says 172.21.15.5:60358) 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5903e700 1 -- 172.21.15.5:0/3930140266 learned_addr learned my addr 172.21.15.5:0/3930140266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5883d700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a541022f0 0x7f0a5419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.957+0000 7f0a5983f700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a54107980 0x7f0a541a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5903e700 1 -- 172.21.15.5:0/3930140266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a541022f0 msgr2=0x7f0a5419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5903e700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a541022f0 0x7f0a5419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5903e700 1 -- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a54107980 msgr2=0x7f0a541a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5903e700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a54107980 0x7f0a541a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5903e700 1 -- 172.21.15.5:0/3930140266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a541a5470 con 0x7f0a54101930 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5903e700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a54101930 0x7f0a54199eb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0060b0 tx=0x7f0a4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c01a070 con 0x7f0a54101930 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a4c00dd20 con 0x7f0a54101930 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a541a56a0 con 0x7f0a54101930 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c014600 con 0x7f0a54101930 2024-04-22T13:25:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.958+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a541a5bb0 con 0x7f0a54101930 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.959+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a54063150 con 0x7f0a54101930 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.960+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0a4c01d070 con 0x7f0a54101930 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a527fc700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0a3c0a3cd0 0x7f0a3c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a5883d700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0a3c0a3cd0 0x7f0a3c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f0a4c0d0400 con 0x7f0a54101930 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a527fc700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] conn(0x7f0a3c0ad4b0 0x7f0a3c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 --> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a3c0aff80 con 0x7f0a3c0ad4b0 2024-04-22T13:25:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c0d07b0 con 0x7f0a54101930 2024-04-22T13:25:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.961+0000 7f0a5883d700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0a3c0a3cd0 0x7f0a3c0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a54078e90 tx=0x7f0a4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.962+0000 7f0a5983f700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] conn(0x7f0a3c0ad4b0 0x7f0a3c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.962+0000 7f0a5983f700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] conn(0x7f0a3c0ad4b0 0x7f0a3c0af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.963+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== osd.10 v2:172.21.15.49:6824/2579440718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a3c0aff80 con 0x7f0a3c0ad4b0 2024-04-22T13:25:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.985+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 --> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a54063580 con 0x7f0a3c0ad4b0 2024-04-22T13:25:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.985+0000 7f0a527fc700 1 -- 172.21.15.5:0/3930140266 <== osd.10 v2:172.21.15.49:6824/2579440718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a54063580 con 0x7f0a3c0ad4b0 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] conn(0x7f0a3c0ad4b0 msgr2=0x7f0a3c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] conn(0x7f0a3c0ad4b0 0x7f0a3c0af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0a3c0a3cd0 msgr2=0x7f0a3c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f0a3c0a3cd0 0x7f0a3c0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a54078e90 tx=0x7f0a4800b410 comp rx=0 tx=0).stop 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a54101930 msgr2=0x7f0a54199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 --2- 172.21.15.5:0/3930140266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a54101930 0x7f0a54199eb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0060b0 tx=0x7f0a4c00d3b0 comp rx=0 tx=0).stop 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5983f700 1 -- 172.21.15.5:0/3930140266 reap_dead start 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 shutdown_connections 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 >> 172.21.15.5:0/3930140266 conn(0x7f0a540788b0 msgr2=0x7f0a540ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 shutdown_connections 2024-04-22T13:25:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:48.986+0000 7f0a5b2a2700 1 -- 172.21.15.5:0/3930140266 wait complete. 2024-04-22T13:25:50.173 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:50 smithi005 ceph-mon[28006]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:50.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:50 smithi049 ceph-mon[31379]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:50.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:50 smithi107 ceph-mon[31859]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 -- 172.21.15.5:0/1933488541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8103480 msgr2=0x7f1cf810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/1933488541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8103480 0x7f1cf810f8d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000b0a0 tx=0x7f1cf000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 -- 172.21.15.5:0/1933488541 shutdown_connections 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/1933488541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8103480 0x7f1cf810f8d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/1933488541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf8108a70 0x7f1cf8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/1933488541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf8102ac0 0x7f1cf8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 -- 172.21.15.5:0/1933488541 >> 172.21.15.5:0/1933488541 conn(0x7f1cf8079bb0 msgr2=0x7f1cf8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 -- 172.21.15.5:0/1933488541 shutdown_connections 2024-04-22T13:25:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 -- 172.21.15.5:0/1933488541 wait complete. 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.549+0000 7f1cfd62e700 1 Processor -- start 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 -- start start 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf8102ac0 0x7f1cf819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf8103480 0x7f1cf819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8108a70 0x7f1cf819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8112c20 con 0x7f1cf8108a70 2024-04-22T13:25:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8112aa0 con 0x7f1cf8103480 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cfd62e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8112da0 con 0x7f1cf8102ac0 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8108a70 0x7f1cf819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8108a70 0x7f1cf819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60376/0 (socket says 172.21.15.5:60376) 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 -- 172.21.15.5:0/2328267476 learned_addr learned my addr 172.21.15.5:0/2328267476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf6ffd700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf8102ac0 0x7f1cf819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1ceffff700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf8103480 0x7f1cf819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 -- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf8102ac0 msgr2=0x7f1cf819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf8102ac0 0x7f1cf819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 -- 172.21.15.5:0/2328267476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf8103480 msgr2=0x7f1cf819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf8103480 0x7f1cf819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 -- 172.21.15.5:0/2328267476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf81a0320 con 0x7f1cf8108a70 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf77fe700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8108a70 0x7f1cf819fbe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000b830 tx=0x7f1cf0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.550+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0021070 con 0x7f1cf8108a70 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.551+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf0007e30 con 0x7f1cf8108a70 2024-04-22T13:25:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.551+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf81a05b0 con 0x7f1cf8108a70 2024-04-22T13:25:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.551+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0004700 con 0x7f1cf8108a70 2024-04-22T13:25:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.551+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf81a7e90 con 0x7f1cf8108a70 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.551+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1cf8101f20 con 0x7f1cf8108a70 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.553+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f1cf0003d70 con 0x7f1cf8108a70 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.553+0000 7f1cf4ff9700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.553+0000 7f1cf6ffd700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.553+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f1cf000f020 con 0x7f1cf8108a70 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.554+0000 7f1cf4ff9700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.554+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 --> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ce40aff80 con 0x7f1ce40ad4b0 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.554+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1cf00d1f40 con 0x7f1cf8108a70 2024-04-22T13:25:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.554+0000 7f1cf6ffd700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80045b0 tx=0x7f1ce800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.554+0000 7f1ceffff700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.554+0000 7f1ceffff700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.555+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== osd.2 v2:172.21.15.107:6800/1765086658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ce40aff80 con 0x7f1ce40ad4b0 2024-04-22T13:25:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.572+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 --> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1cf81041a0 con 0x7f1ce40ad4b0 2024-04-22T13:25:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.572+0000 7f1cf4ff9700 1 -- 172.21.15.5:0/2328267476 <== osd.2 v2:172.21.15.107:6800/1765086658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1cf81041a0 con 0x7f1ce40ad4b0 2024-04-22T13:25:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] conn(0x7f1ce40ad4b0 msgr2=0x7f1ce40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1ce40a3cd0 msgr2=0x7f1ce40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80045b0 tx=0x7f1ce800b3c0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8108a70 msgr2=0x7f1cf819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 --2- 172.21.15.5:0/2328267476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf8108a70 0x7f1cf819fbe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000b830 tx=0x7f1cf0004ce0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cf77fe700 1 -- 172.21.15.5:0/2328267476 reap_dead start 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 shutdown_connections 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 >> 172.21.15.5:0/2328267476 conn(0x7f1cf8079bb0 msgr2=0x7f1cf8101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 shutdown_connections 2024-04-22T13:25:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:50.573+0000 7f1cfd62e700 1 -- 172.21.15.5:0/2328267476 wait complete. 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 -- 172.21.15.5:0/3418039788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c06bb70 msgr2=0x7f1a7c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 --2- 172.21.15.5:0/3418039788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c06bb70 0x7f1a7c10f900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400b0d0 tx=0x7f1a7400b3e0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 -- 172.21.15.5:0/3418039788 shutdown_connections 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 --2- 172.21.15.5:0/3418039788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a7c108a50 0x7f1a7c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 --2- 172.21.15.5:0/3418039788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c06bb70 0x7f1a7c10f900 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 --2- 172.21.15.5:0/3418039788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a7c06b1b0 0x7f1a7c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 -- 172.21.15.5:0/3418039788 >> 172.21.15.5:0/3418039788 conn(0x7f1a7c077990 msgr2=0x7f1a7c077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 -- 172.21.15.5:0/3418039788 shutdown_connections 2024-04-22T13:25:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.244+0000 7f1a80878700 1 -- 172.21.15.5:0/3418039788 wait complete. 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 Processor -- start 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 -- start start 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a7c06b1b0 0x7f1a7c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a7c06bb70 0x7f1a7c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c108a50 0x7f1a7c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c112c40 con 0x7f1a7c108a50 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c112ac0 con 0x7f1a7c06bb70 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.245+0000 7f1a80878700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c112dc0 con 0x7f1a7c06b1b0 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c108a50 0x7f1a7c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c108a50 0x7f1a7c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60396/0 (socket says 172.21.15.5:60396) 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 -- 172.21.15.5:0/472272892 learned_addr learned my addr 172.21.15.5:0/472272892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a79d9b700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a7c06bb70 0x7f1a7c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7a59c700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a7c06b1b0 0x7f1a7c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 -- 172.21.15.5:0/472272892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a7c06b1b0 msgr2=0x7f1a7c199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a7c06b1b0 0x7f1a7c199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 -- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a7c06bb70 msgr2=0x7f1a7c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a7c06bb70 0x7f1a7c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 -- 172.21.15.5:0/472272892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c19e040 con 0x7f1a7c108a50 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7a59c700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a7c06b1b0 0x7f1a7c199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a7ad9d700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c108a50 0x7f1a7c19d900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1a740095c0 tx=0x7f1a74007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401f070 con 0x7f1a7c108a50 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c19e2d0 con 0x7f1a7c108a50 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.246+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a74003c00 con 0x7f1a7c108a50 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.247+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1a5bb0 con 0x7f1a7c108a50 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.247+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401b6a0 con 0x7f1a7c108a50 2024-04-22T13:25:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.247+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a7c063190 con 0x7f1a7c108a50 2024-04-22T13:25:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f1a74022070 con 0x7f1a7c108a50 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a6b7fe700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1a640a3cd0 0x7f1a640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a7a59c700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1a640a3cd0 0x7f1a640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f1a7400f020 con 0x7f1a7c108a50 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a6b7fe700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] conn(0x7f1a640ad4b0 0x7f1a640af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 --> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a640aff80 con 0x7f1a640ad4b0 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.249+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1a740d1cd0 con 0x7f1a7c108a50 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.250+0000 7f1a79d9b700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] conn(0x7f1a640ad4b0 0x7f1a640af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.250+0000 7f1a7a59c700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1a640a3cd0 0x7f1a640a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00baa0 tx=0x7f1a6c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.250+0000 7f1a79d9b700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] conn(0x7f1a640ad4b0 0x7f1a640af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.251+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== osd.4 v2:172.21.15.49:6808/939487791 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a640aff80 con 0x7f1a640ad4b0 2024-04-22T13:25:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.267+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 --> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a7c0635c0 con 0x7f1a640ad4b0 2024-04-22T13:25:51.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a6b7fe700 1 -- 172.21.15.5:0/472272892 <== osd.4 v2:172.21.15.49:6808/939487791 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a7c0635c0 con 0x7f1a640ad4b0 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] conn(0x7f1a640ad4b0 msgr2=0x7f1a640af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] conn(0x7f1a640ad4b0 0x7f1a640af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1a640a3cd0 msgr2=0x7f1a640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1a640a3cd0 0x7f1a640a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00baa0 tx=0x7f1a6c005db0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c108a50 msgr2=0x7f1a7c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 --2- 172.21.15.5:0/472272892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a7c108a50 0x7f1a7c19d900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1a740095c0 tx=0x7f1a74007a20 comp rx=0 tx=0).stop 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a7ad9d700 1 -- 172.21.15.5:0/472272892 reap_dead start 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 shutdown_connections 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 >> 172.21.15.5:0/472272892 conn(0x7f1a7c077990 msgr2=0x7f1a7c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 shutdown_connections 2024-04-22T13:25:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:51.268+0000 7f1a80878700 1 -- 172.21.15.5:0/472272892 wait complete. 2024-04-22T13:25:51.575 INFO:teuthology.orchestra.run.smithi005.stdout:77309411394 2024-04-22T13:25:51.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-22T13:25:52.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:52 smithi005 ceph-mon[28006]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 340 B/s wr, 11 op/s 2024-04-22T13:25:52.212 INFO:teuthology.orchestra.run.smithi005.stdout:326417514521 2024-04-22T13:25:52.212 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.422+0000 7f81a2d9f700 1 -- 172.21.15.5:0/1415963394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102d40 msgr2=0x7f819c10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.422+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/1415963394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102d40 0x7f819c10e830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f819400b0d0 tx=0x7f819400b3e0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 -- 172.21.15.5:0/1415963394 shutdown_connections 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/1415963394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102d40 0x7f819c10e830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/1415963394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c107a60 0x7f819c107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/1415963394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f819c102380 0x7f819c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 -- 172.21.15.5:0/1415963394 >> 172.21.15.5:0/1415963394 conn(0x7f819c076bf0 msgr2=0x7f819c077000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 -- 172.21.15.5:0/1415963394 shutdown_connections 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 -- 172.21.15.5:0/1415963394 wait complete. 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 Processor -- start 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.423+0000 7f81a2d9f700 1 -- start start 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102380 0x7f819c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c102d40 0x7f819c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f819c107a60 0x7f819c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f819c111c20 con 0x7f819c102380 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f819c111aa0 con 0x7f819c107a60 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f819c111da0 con 0x7f819c102d40 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102380 0x7f819c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102380 0x7f819c199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60406/0 (socket says 172.21.15.5:60406) 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 -- 172.21.15.5:0/537956515 learned_addr learned my addr 172.21.15.5:0/537956515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a159c700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c102d40 0x7f819c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a259e700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f819c107a60 0x7f819c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 -- 172.21.15.5:0/537956515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c102d40 msgr2=0x7f819c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c102d40 0x7f819c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 -- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f819c107a60 msgr2=0x7f819c19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f819c107a60 0x7f819c19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 -- 172.21.15.5:0/537956515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c19e050 con 0x7f819c102380 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a259e700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f819c107a60 0x7f819c19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a1d9d700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102380 0x7f819c199040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9c0 tx=0x7f819800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800cbd0 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819800cd70 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c19e340 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.424+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198005610 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.425+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a5bc0 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.425+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f819c063150 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f8198004750 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f8192ffd700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81880a3d50 0x7f81880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f81a159c700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81880a3d50 0x7f81880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f81980d0a20 con 0x7f819c102380 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f8192ffd700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] conn(0x7f81880ad530 0x7f81880af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 --> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f819800be50 con 0x7f81880ad530 2024-04-22T13:25:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.427+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f81980d0f00 con 0x7f819c102380 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.428+0000 7f81a159c700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81880a3d50 0x7f81880a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f819c077260 tx=0x7f818c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.428+0000 7f81a259e700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] conn(0x7f81880ad530 0x7f81880af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.428+0000 7f81a259e700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] conn(0x7f81880ad530 0x7f81880af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.429+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== osd.7 v2:172.21.15.49:6816/3440466171 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f819800be50 con 0x7f81880ad530 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.446+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 --> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f819c063550 con 0x7f81880ad530 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.446+0000 7f8192ffd700 1 -- 172.21.15.5:0/537956515 <== osd.7 v2:172.21.15.49:6816/3440466171 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f819c063550 con 0x7f81880ad530 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] conn(0x7f81880ad530 msgr2=0x7f81880af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] conn(0x7f81880ad530 0x7f81880af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81880a3d50 msgr2=0x7f81880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f81880a3d50 0x7f81880a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f819c077260 tx=0x7f818c00a380 comp rx=0 tx=0).stop 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102380 msgr2=0x7f819c199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 --2- 172.21.15.5:0/537956515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819c102380 0x7f819c199040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9c0 tx=0x7f819800ecd0 comp rx=0 tx=0).stop 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a259e700 1 -- 172.21.15.5:0/537956515 reap_dead start 2024-04-22T13:25:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 shutdown_connections 2024-04-22T13:25:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 >> 172.21.15.5:0/537956515 conn(0x7f819c076bf0 msgr2=0x7f819c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 shutdown_connections 2024-04-22T13:25:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:52.447+0000 7f81a2d9f700 1 -- 172.21.15.5:0/537956515 wait complete. 2024-04-22T13:25:52.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:52 smithi049 ceph-mon[31379]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 340 B/s wr, 11 op/s 2024-04-22T13:25:52.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:52 smithi107 ceph-mon[31859]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 340 B/s wr, 11 op/s 2024-04-22T13:25:53.716 INFO:teuthology.orchestra.run.smithi005.stdout:231928234025 2024-04-22T13:25:53.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-22T13:25:54.198 INFO:teuthology.orchestra.run.smithi005.stdout:146028888120 2024-04-22T13:25:54.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-22T13:25:54.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:54 smithi005 ceph-mon[28006]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:54.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:54 smithi049 ceph-mon[31379]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:54.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:54 smithi107 ceph-mon[31859]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 -- 172.21.15.5:0/1149151966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844108a50 msgr2=0x7f4844108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 --2- 172.21.15.5:0/1149151966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844108a50 0x7f4844108e30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4834009990 tx=0x7f4834009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 -- 172.21.15.5:0/1149151966 shutdown_connections 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 --2- 172.21.15.5:0/1149151966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4844103450 0x7f484410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 --2- 172.21.15.5:0/1149151966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844108a50 0x7f4844108e30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 --2- 172.21.15.5:0/1149151966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4844102a90 0x7f4844102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 -- 172.21.15.5:0/1149151966 >> 172.21.15.5:0/1149151966 conn(0x7f4844079ba0 msgr2=0x7f4844079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 -- 172.21.15.5:0/1149151966 shutdown_connections 2024-04-22T13:25:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.628+0000 7f484985f700 1 -- 172.21.15.5:0/1149151966 wait complete. 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 Processor -- start 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 -- start start 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844102a90 0x7f484419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4844103450 0x7f484419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4844108a50 0x7f484419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4844112c60 con 0x7f4844102a90 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4844112ae0 con 0x7f4844103450 2024-04-22T13:25:54.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f484985f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4844112de0 con 0x7f4844108a50 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f4842ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844102a90 0x7f484419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f4842ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844102a90 0x7f484419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60436/0 (socket says 172.21.15.5:60436) 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f4842ffd700 1 -- 172.21.15.5:0/3216238760 learned_addr learned my addr 172.21.15.5:0/3216238760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f48437fe700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4844108a50 0x7f484419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.629+0000 7f48427fc700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4844103450 0x7f484419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f4842ffd700 1 -- 172.21.15.5:0/3216238760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4844108a50 msgr2=0x7f484419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f4842ffd700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4844108a50 0x7f484419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f4842ffd700 1 -- 172.21.15.5:0/3216238760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4844103450 msgr2=0x7f484419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f4842ffd700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4844103450 0x7f484419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f4842ffd700 1 -- 172.21.15.5:0/3216238760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48441a0460 con 0x7f4844102a90 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f4842ffd700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844102a90 0x7f484419b450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4834000c00 tx=0x7f48340057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834027070 con 0x7f4844102a90 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48441a06f0 con 0x7f4844102a90 2024-04-22T13:25:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4834003680 con 0x7f4844102a90 2024-04-22T13:25:54.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834022b80 con 0x7f4844102a90 2024-04-22T13:25:54.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.630+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48441a7fd0 con 0x7f4844102a90 2024-04-22T13:25:54.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.631+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48440630d0 con 0x7f4844102a90 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.632+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f483402a070 con 0x7f4844102a90 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f484885d700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f482c0a3cd0 0x7f482c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f48427fc700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f482c0a3cd0 0x7f482c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f48340d1f60 con 0x7f4844102a90 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f484885d700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] conn(0x7f482c0ad4b0 0x7f482c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 --> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4834009030 con 0x7f482c0ad4b0 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f48340d2310 con 0x7f4844102a90 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f48437fe700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] conn(0x7f482c0ad4b0 0x7f482c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.633+0000 7f48427fc700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f482c0a3cd0 0x7f482c0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f484407a530 tx=0x7f483800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.634+0000 7f48437fe700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] conn(0x7f482c0ad4b0 0x7f482c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.634+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== osd.6 v2:172.21.15.5:6818/2491838519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4834009030 con 0x7f482c0ad4b0 2024-04-22T13:25:54.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.651+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 --> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4844063540 con 0x7f482c0ad4b0 2024-04-22T13:25:54.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484885d700 1 -- 172.21.15.5:0/3216238760 <== osd.6 v2:172.21.15.5:6818/2491838519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4844063540 con 0x7f482c0ad4b0 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] conn(0x7f482c0ad4b0 msgr2=0x7f482c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] conn(0x7f482c0ad4b0 0x7f482c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f482c0a3cd0 msgr2=0x7f482c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f482c0a3cd0 0x7f482c0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f484407a530 tx=0x7f483800b3c0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844102a90 msgr2=0x7f484419b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 --2- 172.21.15.5:0/3216238760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4844102a90 0x7f484419b450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4834000c00 tx=0x7f48340057a0 comp rx=0 tx=0).stop 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f48437fe700 1 -- 172.21.15.5:0/3216238760 reap_dead start 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 shutdown_connections 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.652+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 >> 172.21.15.5:0/3216238760 conn(0x7f4844079ba0 msgr2=0x7f484410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.653+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 shutdown_connections 2024-04-22T13:25:54.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:54.653+0000 7f484985f700 1 -- 172.21.15.5:0/3216238760 wait complete. 2024-04-22T13:25:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 -- 172.21.15.5:0/2480562894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06b900 msgr2=0x7f07ec10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 --2- 172.21.15.5:0/2480562894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06b900 0x7f07ec10efa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009990 tx=0x7f07dc009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 -- 172.21.15.5:0/2480562894 shutdown_connections 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 --2- 172.21.15.5:0/2480562894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07ec10f4e0 0x7f07ec1118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 --2- 172.21.15.5:0/2480562894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07ec06afe0 0x7f07ec06b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 --2- 172.21.15.5:0/2480562894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06b900 0x7f07ec10efa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 -- 172.21.15.5:0/2480562894 >> 172.21.15.5:0/2480562894 conn(0x7f07ec0788b0 msgr2=0x7f07ec078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 -- 172.21.15.5:0/2480562894 shutdown_connections 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 -- 172.21.15.5:0/2480562894 wait complete. 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 Processor -- start 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.210+0000 7f07f32e1700 1 -- start start 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f32e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06afe0 0x7f07ec19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f32e1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07ec06b900 0x7f07ec19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f32e1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07ec10f4e0 0x7f07ec19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f32e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec114e10 con 0x7f07ec06afe0 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f32e1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec114c90 con 0x7f07ec10f4e0 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f32e1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec114f90 con 0x7f07ec06b900 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06afe0 0x7f07ec19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06afe0 0x7f07ec19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60452/0 (socket says 172.21.15.5:60452) 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 -- 172.21.15.5:0/3031890919 learned_addr learned my addr 172.21.15.5:0/3031890919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f087c700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07ec06b900 0x7f07ec19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f187e700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07ec10f4e0 0x7f07ec19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 -- 172.21.15.5:0/3031890919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07ec06b900 msgr2=0x7f07ec19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07ec06b900 0x7f07ec19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 -- 172.21.15.5:0/3031890919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07ec10f4e0 msgr2=0x7f07ec19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07ec10f4e0 0x7f07ec19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 -- 172.21.15.5:0/3031890919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec1a0340 con 0x7f07ec06afe0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f187e700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07ec10f4e0 0x7f07ec19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07f107d700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06afe0 0x7f07ec19b380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f07e800d980 tx=0x7f07e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.211+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8017070 con 0x7f07ec06afe0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.212+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec1a0630 con 0x7f07ec06afe0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.212+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07e8004830 con 0x7f07ec06afe0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.212+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8005440 con 0x7f07ec06afe0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.212+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec1a7eb0 con 0x7f07ec06afe0 2024-04-22T13:25:55.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.212+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07ec063190 con 0x7f07ec06afe0 2024-04-22T13:25:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.214+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f07e80055e0 con 0x7f07ec06afe0 2024-04-22T13:25:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07e27fc700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f07d80a3cd0 0x7f07d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07f087c700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f07d80a3cd0 0x7f07d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f07e80d09f0 con 0x7f07ec06afe0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07e27fc700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] conn(0x7f07d80ad4b0 0x7f07d80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 --> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07d80aff80 con 0x7f07d80ad4b0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f07e80d0da0 con 0x7f07ec06afe0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07f187e700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] conn(0x7f07d80ad4b0 0x7f07d80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.215+0000 7f07f087c700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f07d80a3cd0 0x7f07d80a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006010 tx=0x7f07dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.216+0000 7f07f187e700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] conn(0x7f07d80ad4b0 0x7f07d80af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.216+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== osd.3 v2:172.21.15.5:6810/466638912 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07d80aff80 con 0x7f07d80ad4b0 2024-04-22T13:25:55.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.233+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 --> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07ec063610 con 0x7f07d80ad4b0 2024-04-22T13:25:55.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07e27fc700 1 -- 172.21.15.5:0/3031890919 <== osd.3 v2:172.21.15.5:6810/466638912 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07ec063610 con 0x7f07d80ad4b0 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] conn(0x7f07d80ad4b0 msgr2=0x7f07d80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] conn(0x7f07d80ad4b0 0x7f07d80af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f07d80a3cd0 msgr2=0x7f07d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f07d80a3cd0 0x7f07d80a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006010 tx=0x7f07dc01a040 comp rx=0 tx=0).stop 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06afe0 msgr2=0x7f07ec19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 --2- 172.21.15.5:0/3031890919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec06afe0 0x7f07ec19b380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f07e800d980 tx=0x7f07e800dd40 comp rx=0 tx=0).stop 2024-04-22T13:25:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f187e700 1 -- 172.21.15.5:0/3031890919 reap_dead start 2024-04-22T13:25:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 shutdown_connections 2024-04-22T13:25:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 >> 172.21.15.5:0/3031890919 conn(0x7f07ec0788b0 msgr2=0x7f07ec110b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 shutdown_connections 2024-04-22T13:25:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.234+0000 7f07f32e1700 1 -- 172.21.15.5:0/3031890919 wait complete. 2024-04-22T13:25:55.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:55 smithi005 ceph-mon[28006]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-22T13:25:55.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:55 smithi049 ceph-mon[31379]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-22T13:25:55.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:55 smithi107 ceph-mon[31859]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-22T13:25:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.793+0000 7fa348ed3700 1 -- 172.21.15.5:0/1985396054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344102ac0 msgr2=0x7fa344102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.793+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1985396054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344102ac0 0x7fa344102f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa338009990 tx=0x7fa338009ca0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 -- 172.21.15.5:0/1985396054 shutdown_connections 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1985396054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344103480 0x7fa34410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1985396054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344108a70 0x7fa344108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1985396054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344102ac0 0x7fa344102f40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 -- 172.21.15.5:0/1985396054 >> 172.21.15.5:0/1985396054 conn(0x7fa344079bb0 msgr2=0x7fa344079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 -- 172.21.15.5:0/1985396054 shutdown_connections 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 -- 172.21.15.5:0/1985396054 wait complete. 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 Processor -- start 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.794+0000 7fa348ed3700 1 -- start start 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344102ac0 0x7fa3441991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344103480 0x7fa3441996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344108a70 0x7fa34419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112c70 con 0x7fa344108a70 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112af0 con 0x7fa344102ac0 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112df0 con 0x7fa344103480 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344108a70 0x7fa34419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344108a70 0x7fa34419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60468/0 (socket says 172.21.15.5:60468) 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 -- 172.21.15.5:0/1671129544 learned_addr learned my addr 172.21.15.5:0/1671129544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa341d9b700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344103480 0x7fa3441996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa34259c700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344102ac0 0x7fa3441991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 -- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344103480 msgr2=0x7fa3441996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344103480 0x7fa3441996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 -- 172.21.15.5:0/1671129544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344102ac0 msgr2=0x7fa3441991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa344102ac0 0x7fa3441991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 -- 172.21.15.5:0/1671129544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34419e160 con 0x7fa344108a70 2024-04-22T13:25:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa341d9b700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa344103480 0x7fa3441996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa342d9d700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344108a70 0x7fa34419da20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa33c007ee0 tx=0x7fa33c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00dbd0 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c00dd70 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34419e450 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c005990 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.795+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a5cd0 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.796+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3440631d0 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fa33c01a020 con 0x7fa344108a70 2024-04-22T13:25:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa3337fe700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3cd0 0x7fa32c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa34259c700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3cd0 0x7fa32c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa33c0d0430 con 0x7fa344108a70 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa3337fe700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] conn(0x7fa32c0ad4b0 0x7fa32c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 --> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa33c021e70 con 0x7fa32c0ad4b0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.798+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa33c0d07e0 con 0x7fa344108a70 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.799+0000 7fa34259c700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3cd0 0x7fa32c0a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa33400dad0 tx=0x7fa33400d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.799+0000 7fa341d9b700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] conn(0x7fa32c0ad4b0 0x7fa32c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.799+0000 7fa341d9b700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] conn(0x7fa32c0ad4b0 0x7fa32c0af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.800+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== osd.5 v2:172.21.15.107:6808/2834490578 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa33c021e70 con 0x7fa32c0ad4b0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.816+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 --> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa344063650 con 0x7fa32c0ad4b0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa3337fe700 1 -- 172.21.15.5:0/1671129544 <== osd.5 v2:172.21.15.107:6808/2834490578 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa344063650 con 0x7fa32c0ad4b0 2024-04-22T13:25:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] conn(0x7fa32c0ad4b0 msgr2=0x7fa32c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] conn(0x7fa32c0ad4b0 0x7fa32c0af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3cd0 msgr2=0x7fa32c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa32c0a3cd0 0x7fa32c0a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa33400dad0 tx=0x7fa33400d4b0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344108a70 msgr2=0x7fa34419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 --2- 172.21.15.5:0/1671129544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa344108a70 0x7fa34419da20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa33c007ee0 tx=0x7fa33c00d3b0 comp rx=0 tx=0).stop 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa342d9d700 1 -- 172.21.15.5:0/1671129544 reap_dead start 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 shutdown_connections 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 >> 172.21.15.5:0/1671129544 conn(0x7fa344079bb0 msgr2=0x7fa344101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 shutdown_connections 2024-04-22T13:25:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:25:55.817+0000 7fa348ed3700 1 -- 172.21.15.5:0/1671129544 wait complete. 2024-04-22T13:25:56.572 INFO:teuthology.orchestra.run.smithi005.stdout:171798691892 2024-04-22T13:25:56.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-22T13:25:56.996 INFO:teuthology.orchestra.run.smithi005.stdout:206158430253 2024-04-22T13:25:56.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-22T13:25:57.474 INFO:teuthology.orchestra.run.smithi005.stdout:111669149757 2024-04-22T13:25:57.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-22T13:25:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:57.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:57 smithi049 ceph-mon[31379]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-22T13:25:57.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:57 smithi107 ceph-mon[31859]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-22T13:25:57.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:57 smithi005 ceph-mon[28006]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-22T13:25:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:58.300 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:25:59.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:25:59 smithi049 ceph-mon[31379]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:25:59.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:25:59 smithi107 ceph-mon[31859]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:25:59.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:25:59 smithi005 ceph-mon[28006]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:26:01.355 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:01 smithi005 ceph-mon[28006]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-22T13:26:01.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:01 smithi049 ceph-mon[31379]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-22T13:26:01.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:01 smithi107 ceph-mon[31859]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-22T13:26:03.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:03 smithi049 ceph-mon[31379]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:26:03.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:03 smithi107 ceph-mon[31859]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:26:03.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:03 smithi005 ceph-mon[28006]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 -- 172.21.15.5:0/2337172953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248101940 msgr2=0x7f5248101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 --2- 172.21.15.5:0/2337172953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248101940 0x7f5248101dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f523800b0a0 tx=0x7f523800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 -- 172.21.15.5:0/2337172953 shutdown_connections 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 --2- 172.21.15.5:0/2337172953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5248107990 0x7f5248107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 --2- 172.21.15.5:0/2337172953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248102300 0x7f524810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 --2- 172.21.15.5:0/2337172953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248101940 0x7f5248101dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 -- 172.21.15.5:0/2337172953 >> 172.21.15.5:0/2337172953 conn(0x7f52480788f0 msgr2=0x7f5248078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 -- 172.21.15.5:0/2337172953 shutdown_connections 2024-04-22T13:26:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.991+0000 7f524ed9c700 1 -- 172.21.15.5:0/2337172953 wait complete. 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.992+0000 7f524ed9c700 1 Processor -- start 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.992+0000 7f524ed9c700 1 -- start start 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.992+0000 7f524ed9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5248101940 0x7f52481034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.992+0000 7f524ed9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 0x7f5248103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.992+0000 7f524ed9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248107990 0x7f5248103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.992+0000 7f5247fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 0x7f5248103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f524ed9c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5248111ae0 con 0x7f5248102300 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f524ed9c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5248111960 con 0x7f5248101940 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f524ed9c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5248111c60 con 0x7f5248107990 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 0x7f5248103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36926/0 (socket says 172.21.15.5:36926) 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 -- 172.21.15.5:0/4054116265 learned_addr learned my addr 172.21.15.5:0/4054116265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f524cb38700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5248101940 0x7f52481034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f524d339700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248107990 0x7f5248103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 -- 172.21.15.5:0/4054116265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248107990 msgr2=0x7f5248103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248107990 0x7f5248103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 -- 172.21.15.5:0/4054116265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5248101940 msgr2=0x7f52481034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5248101940 0x7f52481034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 -- 172.21.15.5:0/4054116265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52481a7640 con 0x7f5248102300 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f524d339700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248107990 0x7f5248103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5247fff700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 0x7f5248103a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f523800b070 tx=0x7f5238004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.993+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238021070 con 0x7f5248102300 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.994+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5238007e30 con 0x7f5248102300 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.994+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238004840 con 0x7f5248102300 2024-04-22T13:26:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.994+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52481a78d0 con 0x7f5248102300 2024-04-22T13:26:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.994+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52481a7d30 con 0x7f5248102300 2024-04-22T13:26:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.995+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5248069730 con 0x7f5248102300 2024-04-22T13:26:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.996+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f5238007980 con 0x7f5248102300 2024-04-22T13:26:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.997+0000 7f5245ffb700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f52340a3cd0 0x7f52340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.997+0000 7f524cb38700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f52340a3cd0 0x7f52340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.997+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f523800f020 con 0x7f5248102300 2024-04-22T13:26:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:04.997+0000 7f524cb38700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f52340a3cd0 0x7f52340a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5230005950 tx=0x7f523000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:04.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.001+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f523802a020 con 0x7f5248102300 2024-04-22T13:26:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 -- 172.21.15.5:0/4182834105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010102300 msgr2=0x7f601010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 --2- 172.21.15.5:0/4182834105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010102300 0x7f601010e7f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f600800b0a0 tx=0x7f600800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 -- 172.21.15.5:0/4182834105 shutdown_connections 2024-04-22T13:26:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 --2- 172.21.15.5:0/4182834105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010102300 0x7f601010e7f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 --2- 172.21.15.5:0/4182834105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6010107990 0x7f6010107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 --2- 172.21.15.5:0/4182834105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6010101940 0x7f6010101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 -- 172.21.15.5:0/4182834105 >> 172.21.15.5:0/4182834105 conn(0x7f60100788f0 msgr2=0x7f6010078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 -- 172.21.15.5:0/4182834105 shutdown_connections 2024-04-22T13:26:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.208+0000 7f60163dd700 1 -- 172.21.15.5:0/4182834105 wait complete. 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 Processor -- start 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 -- start start 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 0x7f60101034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6010102300 0x7f6010103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6010107990 0x7f6010103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111b00 con 0x7f6010101940 2024-04-22T13:26:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111980 con 0x7f6010102300 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f60163dd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111c80 con 0x7f6010107990 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f600ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 0x7f60101034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f600ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 0x7f60101034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36932/0 (socket says 172.21.15.5:36932) 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f600ffff700 1 -- 172.21.15.5:0/3019092946 learned_addr learned my addr 172.21.15.5:0/3019092946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f600f7fe700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6010102300 0x7f6010103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.209+0000 7f601497a700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6010107990 0x7f6010103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600ffff700 1 -- 172.21.15.5:0/3019092946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6010107990 msgr2=0x7f6010103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600ffff700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6010107990 0x7f6010103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600ffff700 1 -- 172.21.15.5:0/3019092946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6010102300 msgr2=0x7f6010103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600ffff700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6010102300 0x7f6010103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600ffff700 1 -- 172.21.15.5:0/3019092946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60101a7610 con 0x7f6010101940 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600ffff700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 0x7f60101034f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f600000d980 tx=0x7f600000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000017070 con 0x7f6010101940 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60101a7900 con 0x7f6010101940 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a7e40 con 0x7f6010101940 2024-04-22T13:26:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6000005020 con 0x7f6010101940 2024-04-22T13:26:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.210+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000fa10 con 0x7f6010101940 2024-04-22T13:26:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.211+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc005330 con 0x7f6010101940 2024-04-22T13:26:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.213+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f600000fbb0 con 0x7f6010101940 2024-04-22T13:26:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.213+0000 7f600d7fa700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5ff80a3d50 0x7f5ff80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.213+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f52481a7f00 con 0x7f5248102300 2024-04-22T13:26:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.214+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f60000d0050 con 0x7f6010101940 2024-04-22T13:26:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.214+0000 7f600f7fe700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5ff80a3d50 0x7f5ff80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.214+0000 7f600f7fe700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5ff80a3d50 0x7f5ff80a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6010079040 tx=0x7f6004009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:05.214 INFO:teuthology.orchestra.run.smithi005.stdout:77309411396 2024-04-22T13:26:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.218+0000 7f5245ffb700 1 -- 172.21.15.5:0/4054116265 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f523809e380 con 0x7f5248102300 2024-04-22T13:26:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.218+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6000020020 con 0x7f6010101940 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.220+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f52340a3cd0 msgr2=0x7f52340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f52340a3cd0 0x7f52340a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5230005950 tx=0x7f523000a300 comp rx=0 tx=0).stop 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 msgr2=0x7f5248103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 0x7f5248103a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f523800b070 tx=0x7f5238004d00 comp rx=0 tx=0).stop 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 shutdown_connections 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f52340a3cd0 0x7f52340a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5248101940 0x7f52481034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5248107990 0x7f5248103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 --2- 172.21.15.5:0/4054116265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5248102300 0x7f5248103a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 >> 172.21.15.5:0/4054116265 conn(0x7f52480788f0 msgr2=0x7f5248105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 shutdown_connections 2024-04-22T13:26:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.221+0000 7f524ed9c700 1 -- 172.21.15.5:0/4054116265 wait complete. 2024-04-22T13:26:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.465+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5ffc0058f0 con 0x7f6010101940 2024-04-22T13:26:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.465+0000 7f600d7fa700 1 -- 172.21.15.5:0/3019092946 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f600009ca80 con 0x7f6010101940 2024-04-22T13:26:05.462 INFO:teuthology.orchestra.run.smithi005.stdout:360777252887 2024-04-22T13:26:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5ff80a3d50 msgr2=0x7f5ff80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5ff80a3d50 0x7f5ff80a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6010079040 tx=0x7f6004009260 comp rx=0 tx=0).stop 2024-04-22T13:26:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 msgr2=0x7f60101034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 0x7f60101034f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f600000d980 tx=0x7f600000dc90 comp rx=0 tx=0).stop 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 shutdown_connections 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6010107990 0x7f6010103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101940 0x7f60101034f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5ff80a3d50 0x7f5ff80a6210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 --2- 172.21.15.5:0/3019092946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6010102300 0x7f6010103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 >> 172.21.15.5:0/3019092946 conn(0x7f60100788f0 msgr2=0x7f601006aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 shutdown_connections 2024-04-22T13:26:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:05.468+0000 7f60163dd700 1 -- 172.21.15.5:0/3019092946 wait complete. 2024-04-22T13:26:05.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:05 smithi005 ceph-mon[28006]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-22T13:26:05.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:05 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4054116265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T13:26:05.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:05 smithi049 ceph-mon[31379]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-22T13:26:05.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:05 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4054116265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T13:26:05.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:05 smithi107 ceph-mon[31859]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-22T13:26:05.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:05 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4054116265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T13:26:06.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:06 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3019092946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T13:26:06.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:06 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3019092946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T13:26:06.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:06 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3019092946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T13:26:07.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411396 for osd.2 2024-04-22T13:26:07.059 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 -- 172.21.15.5:0/165339452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84141078a0 msgr2=0x7f8414107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 --2- 172.21.15.5:0/165339452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84141078a0 0x7f8414107c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8404009990 tx=0x7f8404009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 -- 172.21.15.5:0/165339452 shutdown_connections 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 --2- 172.21.15.5:0/165339452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8414106c80 0x7f841410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 --2- 172.21.15.5:0/165339452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84141078a0 0x7f8414107c80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 --2- 172.21.15.5:0/165339452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8414108250 0x7f84141065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 -- 172.21.15.5:0/165339452 >> 172.21.15.5:0/165339452 conn(0x7f8414077b40 msgr2=0x7f84140feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 -- 172.21.15.5:0/165339452 shutdown_connections 2024-04-22T13:26:07.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.250+0000 7f8418c6c700 1 -- 172.21.15.5:0/165339452 wait complete. 2024-04-22T13:26:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 Processor -- start 2024-04-22T13:26:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 -- start start 2024-04-22T13:26:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8414106c80 0x7f84141993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84141078a0 0x7f84141998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 0x7f841419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111b10 con 0x7f8414108250 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111990 con 0x7f84141078a0 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8418c6c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111c90 con 0x7f8414106c80 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 0x7f841419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 0x7f841419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36952/0 (socket says 172.21.15.5:36952) 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 -- 172.21.15.5:0/261957838 learned_addr learned my addr 172.21.15.5:0/261957838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8411d9b700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84141078a0 0x7f84141998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f841259c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8414106c80 0x7f84141993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 -- 172.21.15.5:0/261957838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8414106c80 msgr2=0x7f84141993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8414106c80 0x7f84141993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 -- 172.21.15.5:0/261957838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84141078a0 msgr2=0x7f84141998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84141078a0 0x7f84141998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.252+0000 7f8412d9d700 1 -- 172.21.15.5:0/261957838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841419e0f0 con 0x7f8414108250 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.253+0000 7f8412d9d700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 0x7f841419da20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f840c007ee0 tx=0x7f840c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.253+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c004bb0 con 0x7f8414108250 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.253+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f840c004d50 con 0x7f8414108250 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.253+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c005920 con 0x7f8414108250 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.253+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841419e3e0 con 0x7f8414108250 2024-04-22T13:26:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.253+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141a5cd0 con 0x7f8414108250 2024-04-22T13:26:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.254+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8414069730 con 0x7f8414108250 2024-04-22T13:26:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.255+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f840c01a020 con 0x7f8414108250 2024-04-22T13:26:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.256+0000 7f84037fe700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f83fc0a3d50 0x7f83fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.256+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f840c0cf9e0 con 0x7f8414108250 2024-04-22T13:26:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.256+0000 7f841259c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f83fc0a3d50 0x7f83fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.256+0000 7f841259c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f83fc0a3d50 0x7f83fc0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f840400b2e0 tx=0x7f8404005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:07.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.261+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f840c010070 con 0x7f8414108250 2024-04-22T13:26:07.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:07 smithi005 ceph-mon[28006]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T13:26:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.470+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f84141a5ea0 con 0x7f8414108250 2024-04-22T13:26:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.470+0000 7f84037fe700 1 -- 172.21.15.5:0/261957838 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f840c09c410 con 0x7f8414108250 2024-04-22T13:26:07.466 INFO:teuthology.orchestra.run.smithi005.stdout:304942678050 2024-04-22T13:26:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f83fc0a3d50 msgr2=0x7f83fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f83fc0a3d50 0x7f83fc0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f840400b2e0 tx=0x7f8404005960 comp rx=0 tx=0).stop 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 msgr2=0x7f841419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 0x7f841419da20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f840c007ee0 tx=0x7f840c00d3b0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 shutdown_connections 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8414108250 0x7f841419da20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8414106c80 0x7f84141993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f83fc0a3d50 0x7f83fc0a6210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 --2- 172.21.15.5:0/261957838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84141078a0 0x7f84141998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 >> 172.21.15.5:0/261957838 conn(0x7f8414077b40 msgr2=0x7f84141036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 shutdown_connections 2024-04-22T13:26:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:07.472+0000 7f8418c6c700 1 -- 172.21.15.5:0/261957838 wait complete. 2024-04-22T13:26:07.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:07 smithi049 ceph-mon[31379]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T13:26:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:07 smithi107 ceph-mon[31859]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T13:26:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:07.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 360777252882 got 360777252887 for osd.11 2024-04-22T13:26:07.899 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:08.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:08 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/261957838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T13:26:08.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:08 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/261957838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T13:26:08.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:08 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/261957838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T13:26:09.296 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:09 smithi005 ceph-mon[28006]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-22T13:26:09.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:09 smithi049 ceph-mon[31379]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-22T13:26:09.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:09 smithi107 ceph-mon[31859]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-22T13:26:10.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 -- 172.21.15.5:0/2660041326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 msgr2=0x7fcb60107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 --2- 172.21.15.5:0/2660041326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 0x7fcb60107d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0098f0 tx=0x7fcb5c009c00 comp rx=0 tx=0).stop 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 -- 172.21.15.5:0/2660041326 shutdown_connections 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 --2- 172.21.15.5:0/2660041326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60102300 0x7fcb6010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 --2- 172.21.15.5:0/2660041326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 0x7fcb60107d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 --2- 172.21.15.5:0/2660041326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 -- 172.21.15.5:0/2660041326 >> 172.21.15.5:0/2660041326 conn(0x7fcb600788f0 msgr2=0x7fcb60078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 -- 172.21.15.5:0/2660041326 shutdown_connections 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 -- 172.21.15.5:0/2660041326 wait complete. 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.179+0000 7fcb6783b700 1 Processor -- start 2024-04-22T13:26:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 -- start start 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60102300 0x7fcb60199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 0x7fcb6019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60111b30 con 0x7fcb60102300 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601119b0 con 0x7fcb60101940 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb6783b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60111cb0 con 0x7fcb60107990 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb64dd6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60102300 0x7fcb60199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:54870/0 (socket says 172.21.15.5:54870) 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 -- 172.21.15.5:0/4117837369 learned_addr learned my addr 172.21.15.5:0/4117837369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb65dd8700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 0x7fcb6019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 -- 172.21.15.5:0/4117837369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 msgr2=0x7fcb6019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 0x7fcb6019dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 -- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60102300 msgr2=0x7fcb60199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60102300 0x7fcb60199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.180+0000 7fcb655d7700 1 -- 172.21.15.5:0/4117837369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6019e210 con 0x7fcb60101940 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.181+0000 7fcb655d7700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60199200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009230 tx=0x7fcb5c00fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.181+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0050c0 con 0x7fcb60101940 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.181+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5c00b560 con 0x7fcb60101940 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.181+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6019e4a0 con 0x7fcb60101940 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.181+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c003a00 con 0x7fcb60101940 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.181+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb601a5d80 con 0x7fcb60101940 2024-04-22T13:26:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.182+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb60069730 con 0x7fcb60101940 2024-04-22T13:26:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.183+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fcb5c00b710 con 0x7fcb60101940 2024-04-22T13:26:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.184+0000 7fcb567fc700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcb4c0a3d50 0x7fcb4c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.184+0000 7fcb64dd6700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcb4c0a3d50 0x7fcb4c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.184+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fcb5c0d1510 con 0x7fcb60101940 2024-04-22T13:26:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.184+0000 7fcb64dd6700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcb4c0a3d50 0x7fcb4c0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb60078f70 tx=0x7fcb50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:10.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.187+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb5c013030 con 0x7fcb60101940 2024-04-22T13:26:10.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.405+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcb601a5f50 con 0x7fcb60101940 2024-04-22T13:26:10.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.405+0000 7fcb567fc700 1 -- 172.21.15.5:0/4117837369 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb5c021380 con 0x7fcb60101940 2024-04-22T13:26:10.403 INFO:teuthology.orchestra.run.smithi005.stdout:146028888124 2024-04-22T13:26:10.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/3487648127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08103480 msgr2=0x7f9a0810f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/3487648127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08103480 0x7f9a0810f980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000b0a0 tx=0x7f9a0000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/3487648127 shutdown_connections 2024-04-22T13:26:10.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/3487648127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08103480 0x7f9a0810f980 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/3487648127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08108ab0 0x7f9a08108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/3487648127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08102ac0 0x7f9a08102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/3487648127 >> 172.21.15.5:0/3487648127 conn(0x7f9a08079bf0 msgr2=0x7f9a0807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/3487648127 shutdown_connections 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.399+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/3487648127 wait complete. 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 Processor -- start 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 -- start start 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 0x7f9a081991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103480 0x7f9a081996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08108ab0 0x7f9a0819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08112cb0 con 0x7f9a08102ac0 2024-04-22T13:26:10.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08112b30 con 0x7f9a08108ab0 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.400+0000 7f9a0e2f5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08112e30 con 0x7f9a08103480 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a07fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 0x7f9a081991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a0c892700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08108ab0 0x7f9a0819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103480 0x7f9a081996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a07fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 0x7f9a081991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58156/0 (socket says 172.21.15.5:58156) 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a07fff700 1 -- 172.21.15.5:0/2991617856 learned_addr learned my addr 172.21.15.5:0/2991617856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 -- 172.21.15.5:0/2991617856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08108ab0 msgr2=0x7f9a0819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08108ab0 0x7f9a0819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 -- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 msgr2=0x7f9a081991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 0x7f9a081991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 -- 172.21.15.5:0/2991617856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0819e1b0 con 0x7f9a08103480 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a0c892700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08108ab0 0x7f9a0819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a07fff700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 0x7f9a081991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a077fe700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103480 0x7f9a081996e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99f000e9f0 tx=0x7f99f000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f000c9c0 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f000cb60 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0819e4a0 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0018680 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.401+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081a5d20 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.402+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a08069730 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.404+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f99f0010430 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.405+0000 7f9a057fa700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99f40a3cd0 0x7f99f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.405+0000 7f9a07fff700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99f40a3cd0 0x7f99f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.405+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f99f00d0620 con 0x7f9a08103480 2024-04-22T13:26:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.405+0000 7f9a07fff700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99f40a3cd0 0x7f99f40a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99f8011550 tx=0x7f99f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcb4c0a3d50 msgr2=0x7fcb4c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcb4c0a3d50 0x7fcb4c0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb60078f70 tx=0x7fcb50009260 comp rx=0 tx=0).stop 2024-04-22T13:26:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 msgr2=0x7fcb60199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60199200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009230 tx=0x7fcb5c00fa70 comp rx=0 tx=0).stop 2024-04-22T13:26:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99f009d050 con 0x7f9a08103480 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 shutdown_connections 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcb4c0a3d50 0x7fcb4c0a6210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb60101940 0x7fcb60199200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb60107990 0x7fcb6019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 --2- 172.21.15.5:0/4117837369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb60102300 0x7fcb60199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 >> 172.21.15.5:0/4117837369 conn(0x7fcb600788f0 msgr2=0x7fcb6006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 shutdown_connections 2024-04-22T13:26:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.408+0000 7fcb6783b700 1 -- 172.21.15.5:0/4117837369 wait complete. 2024-04-22T13:26:10.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:10 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:26:10.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:10 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:26:10.584 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:10 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:26:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.626+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9a0802df80 con 0x7f9a08103480 2024-04-22T13:26:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.627+0000 7f9a057fa700 1 -- 172.21.15.5:0/2991617856 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99f009c950 con 0x7f9a08103480 2024-04-22T13:26:10.623 INFO:teuthology.orchestra.run.smithi005.stdout:270582939689 2024-04-22T13:26:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99f40a3cd0 msgr2=0x7f99f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99f40a3cd0 0x7f99f40a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99f8011550 tx=0x7f99f8009260 comp rx=0 tx=0).stop 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103480 msgr2=0x7f9a081996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103480 0x7f9a081996e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99f000e9f0 tx=0x7f99f000ed00 comp rx=0 tx=0).stop 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 shutdown_connections 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99f40a3cd0 0x7f99f40a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a08108ab0 0x7f9a0819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a08102ac0 0x7f9a081991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 --2- 172.21.15.5:0/2991617856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103480 0x7f9a081996e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 >> 172.21.15.5:0/2991617856 conn(0x7f9a08079bf0 msgr2=0x7f9a08100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 shutdown_connections 2024-04-22T13:26:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:10.629+0000 7f9a0e2f5700 1 -- 172.21.15.5:0/2991617856 wait complete. 2024-04-22T13:26:11.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:11 smithi107 ceph-mon[31859]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-22T13:26:11.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4117837369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T13:26:11.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2991617856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T13:26:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:11 smithi005 ceph-mon[28006]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-22T13:26:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4117837369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T13:26:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2991617856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T13:26:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:11 smithi049 ceph-mon[31379]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-22T13:26:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4117837369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T13:26:11.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2991617856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T13:26:12.233 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888120 got 146028888124 for osd.4 2024-04-22T13:26:12.233 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:12.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939684 got 270582939689 for osd.8 2024-04-22T13:26:12.364 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:12 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:12 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:12.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:12 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:13.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678045 got 304942678050 for osd.9 2024-04-22T13:26:13.183 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:13.710 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.719+0000 7fdd20eeb700 1 -- 172.21.15.5:0/3083871606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c102d50 msgr2=0x7fdd1c1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.719+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/3083871606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c102d50 0x7fdd1c1065e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd100099c0 tx=0x7fdd10009cd0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 -- 172.21.15.5:0/3083871606 shutdown_connections 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/3083871606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/3083871606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd1c1023a0 0x7fdd1c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/3083871606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c102d50 0x7fdd1c1065e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 -- 172.21.15.5:0/3083871606 >> 172.21.15.5:0/3083871606 conn(0x7fdd1c077fb0 msgr2=0x7fdd1c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 -- 172.21.15.5:0/3083871606 shutdown_connections 2024-04-22T13:26:13.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.720+0000 7fdd20eeb700 1 -- 172.21.15.5:0/3083871606 wait complete. 2024-04-22T13:26:13.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.721+0000 7fdd20eeb700 1 Processor -- start 2024-04-22T13:26:13.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.721+0000 7fdd20eeb700 1 -- start start 2024-04-22T13:26:13.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd20eeb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd1c1023a0 0x7fdd1c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd20eeb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd1c102d50 0x7fdd1c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd20eeb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd20eeb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c111b70 con 0x7fdd1c106cc0 2024-04-22T13:26:13.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd20eeb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1119f0 con 0x7fdd1c1023a0 2024-04-22T13:26:13.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd20eeb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c111cf0 con 0x7fdd1c102d50 2024-04-22T13:26:13.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.722+0000 7fdd1ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:13 smithi005 ceph-mon[28006]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58170/0 (socket says 172.21.15.5:58170) 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1ad9d700 1 -- 172.21.15.5:0/2417346612 learned_addr learned my addr 172.21.15.5:0/2417346612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd19d9b700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd1c102d50 0x7fdd1c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1a59c700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd1c1023a0 0x7fdd1c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1ad9d700 1 -- 172.21.15.5:0/2417346612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd1c102d50 msgr2=0x7fdd1c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1ad9d700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd1c102d50 0x7fdd1c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1ad9d700 1 -- 172.21.15.5:0/2417346612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd1c1023a0 msgr2=0x7fdd1c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.723+0000 7fdd1ad9d700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd1c1023a0 0x7fdd1c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.724+0000 7fdd1ad9d700 1 -- 172.21.15.5:0/2417346612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c19e1e0 con 0x7fdd1c106cc0 2024-04-22T13:26:13.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.724+0000 7fdd1ad9d700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c19daa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400ef30 tx=0x7fdd1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.725+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd140092e0 con 0x7fdd1c106cc0 2024-04-22T13:26:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.725+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd14004bb0 con 0x7fdd1c106cc0 2024-04-22T13:26:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.725+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c19e470 con 0x7fdd1c106cc0 2024-04-22T13:26:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.726+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c0fd350 con 0x7fdd1c106cc0 2024-04-22T13:26:13.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.727+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004280 con 0x7fdd1c106cc0 2024-04-22T13:26:13.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.727+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c069730 con 0x7fdd1c106cc0 2024-04-22T13:26:13.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.729+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fdd1401b030 con 0x7fdd1c106cc0 2024-04-22T13:26:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.730+0000 7fdd0b7fe700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdd040a3bb0 0x7fdd040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.730+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fdd140cfe20 con 0x7fdd1c106cc0 2024-04-22T13:26:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.733+0000 7fdd1a59c700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdd040a3bb0 0x7fdd040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.733+0000 7fdd1a59c700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdd040a3bb0 0x7fdd040a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009b30 tx=0x7fdd0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.738+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd14022020 con 0x7fdd1c106cc0 2024-04-22T13:26:13.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:13 smithi049 ceph-mon[31379]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-22T13:26:13.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:13 smithi107 ceph-mon[31859]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-22T13:26:13.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.953+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdd1c19a530 con 0x7fdd1c106cc0 2024-04-22T13:26:13.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.953+0000 7fdd0b7fe700 1 -- 172.21.15.5:0/2417346612 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdd14010040 con 0x7fdd1c106cc0 2024-04-22T13:26:13.951 INFO:teuthology.orchestra.run.smithi005.stdout:55834574922 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdd040a3bb0 msgr2=0x7fdd040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdd040a3bb0 0x7fdd040a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009b30 tx=0x7fdd0c009260 comp rx=0 tx=0).stop 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 msgr2=0x7fdd1c19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c19daa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400ef30 tx=0x7fdd1400c3a0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 shutdown_connections 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdd040a3bb0 0x7fdd040a6070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd1c1023a0 0x7fdd1c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c106cc0 0x7fdd1c19daa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 --2- 172.21.15.5:0/2417346612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd1c102d50 0x7fdd1c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.955+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 >> 172.21.15.5:0/2417346612 conn(0x7fdd1c077fb0 msgr2=0x7fdd1c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.956+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 shutdown_connections 2024-04-22T13:26:13.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:13.956+0000 7fdd20eeb700 1 -- 172.21.15.5:0/2417346612 wait complete. 2024-04-22T13:26:14.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 -- 172.21.15.5:0/2419350105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50108a70 msgr2=0x7f3d50108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 --2- 172.21.15.5:0/2419350105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50108a70 0x7f3d50108e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 -- 172.21.15.5:0/2419350105 shutdown_connections 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 --2- 172.21.15.5:0/2419350105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50103480 0x7f3d5010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 --2- 172.21.15.5:0/2419350105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50108a70 0x7f3d50108e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 --2- 172.21.15.5:0/2419350105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50102ac0 0x7f3d50102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 -- 172.21.15.5:0/2419350105 >> 172.21.15.5:0/2419350105 conn(0x7f3d50079bb0 msgr2=0x7f3d50079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 -- 172.21.15.5:0/2419350105 shutdown_connections 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.397+0000 7f3d575db700 1 -- 172.21.15.5:0/2419350105 wait complete. 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 Processor -- start 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 -- start start 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 0x7f3d50199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50103480 0x7f3d50199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50108a70 0x7f3d5019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50112c00 con 0x7f3d50102ac0 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50112a80 con 0x7f3d50108a70 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d575db700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50112d80 con 0x7f3d50103480 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 0x7f3d50199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 0x7f3d50199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58196/0 (socket says 172.21.15.5:58196) 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 -- 172.21.15.5:0/171587863 learned_addr learned my addr 172.21.15.5:0/171587863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d54b76700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50103480 0x7f3d50199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 -- 172.21.15.5:0/171587863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50103480 msgr2=0x7f3d50199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50103480 0x7f3d50199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 -- 172.21.15.5:0/171587863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50108a70 msgr2=0x7f3d5019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50108a70 0x7f3d5019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.398+0000 7f3d55377700 1 -- 172.21.15.5:0/171587863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5019e040 con 0x7f3d50102ac0 2024-04-22T13:26:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.399+0000 7f3d55377700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 0x7f3d50199030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c000c00 tx=0x7f3d4c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.399+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c027070 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.399+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c0041d0 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.399+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5019e2d0 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.399+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00f460 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.399+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d501a5bb0 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.400+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d50069730 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.401+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f3d4c02a070 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.401+0000 7f3d467fc700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d3c0a3d50 0x7f3d3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.402+0000 7f3d54b76700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d3c0a3d50 0x7f3d3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.402+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f3d4c0d1770 con 0x7f3d50102ac0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.402+0000 7f3d54b76700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d3c0a3d50 0x7f3d3c0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005ea0 tx=0x7f3d40005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.405+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d4c031070 con 0x7f3d50102ac0 2024-04-22T13:26:14.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2417346612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T13:26:14.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-22T13:26:14.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-22T13:26:14.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-22T13:26:14.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-04-22T13:26:14.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-22T13:26:14.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-04-22T13:26:14.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-22T13:26:14.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:14 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-22T13:26:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.637+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3d5002df80 con 0x7f3d50102ac0 2024-04-22T13:26:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.637+0000 7f3d467fc700 1 -- 172.21.15.5:0/171587863 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d4c013070 con 0x7f3d50102ac0 2024-04-22T13:26:14.635 INFO:teuthology.orchestra.run.smithi005.stdout:34359738448 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d3c0a3d50 msgr2=0x7f3d3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d3c0a3d50 0x7f3d3c0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005ea0 tx=0x7f3d40005e30 comp rx=0 tx=0).stop 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 msgr2=0x7f3d50199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 0x7f3d50199030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c000c00 tx=0x7f3d4c0047b0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 shutdown_connections 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d3c0a3d50 0x7f3d3c0a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50108a70 0x7f3d5019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d50102ac0 0x7f3d50199030 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 --2- 172.21.15.5:0/171587863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50103480 0x7f3d50199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.639+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 >> 172.21.15.5:0/171587863 conn(0x7f3d50079bb0 msgr2=0x7f3d50101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.640+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 shutdown_connections 2024-04-22T13:26:14.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:14.640+0000 7f3d575db700 1 -- 172.21.15.5:0/171587863 wait complete. 2024-04-22T13:26:14.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2417346612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T13:26:14.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-22T13:26:14.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:14 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-22T13:26:14.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2417346612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T13:26:14.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-22T13:26:14.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-22T13:26:14.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-22T13:26:14.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-04-22T13:26:14.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-22T13:26:14.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-04-22T13:26:14.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-22T13:26:14.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:14 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 -- 172.21.15.5:0/2287692202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c108aa0 msgr2=0x7f066c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 --2- 172.21.15.5:0/2287692202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c108aa0 0x7f066c108e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0668009990 tx=0x7f0668009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 -- 172.21.15.5:0/2287692202 shutdown_connections 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 --2- 172.21.15.5:0/2287692202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c103490 0x7f066c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 --2- 172.21.15.5:0/2287692202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c108aa0 0x7f066c108e80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 --2- 172.21.15.5:0/2287692202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066c102ad0 0x7f066c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.072+0000 7f06738d5700 1 -- 172.21.15.5:0/2287692202 >> 172.21.15.5:0/2287692202 conn(0x7f066c079c30 msgr2=0x7f066c07a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.073+0000 7f06738d5700 1 -- 172.21.15.5:0/2287692202 shutdown_connections 2024-04-22T13:26:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.073+0000 7f06738d5700 1 -- 172.21.15.5:0/2287692202 wait complete. 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.073+0000 7f06738d5700 1 Processor -- start 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.073+0000 7f06738d5700 1 -- start start 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f06738d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066c102ad0 0x7f066c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f06738d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 0x7f066c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f06738d5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c108aa0 0x7f066c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f0670e70700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 0x7f066c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f06738d5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f066c112ca0 con 0x7f066c103490 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f0670e70700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 0x7f066c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58208/0 (socket says 172.21.15.5:58208) 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f0670e70700 1 -- 172.21.15.5:0/136615388 learned_addr learned my addr 172.21.15.5:0/136615388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f0671671700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066c102ad0 0x7f066c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f066c112b20 con 0x7f066c102ad0 2024-04-22T13:26:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f0671e72700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c108aa0 0x7f066c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.074+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f066c112e20 con 0x7f066c108aa0 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f0670e70700 1 -- 172.21.15.5:0/136615388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c108aa0 msgr2=0x7f066c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f0670e70700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c108aa0 0x7f066c19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f0670e70700 1 -- 172.21.15.5:0/136615388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066c102ad0 msgr2=0x7f066c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f0670e70700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066c102ad0 0x7f066c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f0670e70700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c19e200 con 0x7f066c103490 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f0670e70700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 0x7f066c199730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e9a0 tx=0x7f065c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00ca10 con 0x7f066c103490 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f065c00cbb0 con 0x7f066c103490 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c018680 con 0x7f066c103490 2024-04-22T13:26:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c19e4f0 con 0x7f066c103490 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.075+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a5d70 con 0x7f066c103490 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.076+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c10b0a0 con 0x7f066c103490 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.078+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f065c010430 con 0x7f066c103490 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.079+0000 7f06627fc700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06580a3d50 0x7f06580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.079+0000 7f0671671700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06580a3d50 0x7f06580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.079+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f065c0d0950 con 0x7f066c103490 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.079+0000 7f0671671700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06580a3d50 0x7f06580a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f066800b2e0 tx=0x7f0668005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.082+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f065c09d1e0 con 0x7f066c103490 2024-04-22T13:26:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.292+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f066c19a550 con 0x7f066c103490 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stdout:231928234030 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.293+0000 7f06627fc700 1 -- 172.21.15.5:0/136615388 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f065c09cae0 con 0x7f066c103490 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06580a3d50 msgr2=0x7f06580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06580a3d50 0x7f06580a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f066800b2e0 tx=0x7f0668005960 comp rx=0 tx=0).stop 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 msgr2=0x7f066c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 0x7f066c199730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e9a0 tx=0x7f065c00ecb0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 shutdown_connections 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f06580a3d50 0x7f06580a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066c102ad0 0x7f066c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c108aa0 0x7f066c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 --2- 172.21.15.5:0/136615388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066c103490 0x7f066c199730 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 >> 172.21.15.5:0/136615388 conn(0x7f066c079c30 msgr2=0x7f066c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 shutdown_connections 2024-04-22T13:26:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:15.295+0000 7f06738d5700 1 -- 172.21.15.5:0/136615388 wait complete. 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-04-22T13:26:15.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-22T13:26:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-22T13:26:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T13:26:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/171587863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T13:26:15.772 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/136615388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T13:26:15.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-04-22T13:26:15.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-22T13:26:15.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-22T13:26:15.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/171587863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T13:26:15.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/136615388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-22T13:26:15.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-04-22T13:26:15.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-22T13:26:15.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-22T13:26:15.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T13:26:15.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/171587863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T13:26:15.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/136615388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T13:26:16.251 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234025 got 231928234030 for osd.7 2024-04-22T13:26:16.252 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:16.554 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:16 smithi005 ceph-mon[28006]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T13:26:16.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:16 smithi049 ceph-mon[31379]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T13:26:16.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:16 smithi107 ceph-mon[31859]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 -- 172.21.15.5:0/235134772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 msgr2=0x7f55dc102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 --2- 172.21.15.5:0/235134772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc102f40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009990 tx=0x7f55d0009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 -- 172.21.15.5:0/235134772 shutdown_connections 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 --2- 172.21.15.5:0/235134772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc103480 0x7f55dc10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 --2- 172.21.15.5:0/235134772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55dc108a70 0x7f55dc108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 --2- 172.21.15.5:0/235134772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc102f40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 -- 172.21.15.5:0/235134772 >> 172.21.15.5:0/235134772 conn(0x7f55dc079bb0 msgr2=0x7f55dc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:17.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 -- 172.21.15.5:0/235134772 shutdown_connections 2024-04-22T13:26:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 -- 172.21.15.5:0/235134772 wait complete. 2024-04-22T13:26:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.121+0000 7f55e1549700 1 Processor -- start 2024-04-22T13:26:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 -- start start 2024-04-22T13:26:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:17.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55dc103480 0x7f55dc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc108a70 0x7f55dc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc112ca0 con 0x7f55dc102ac0 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc112b20 con 0x7f55dc108a70 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55e1549700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc112e20 con 0x7f55dc103480 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58220/0 (socket says 172.21.15.5:58220) 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55daffd700 1 -- 172.21.15.5:0/3935147403 learned_addr learned my addr 172.21.15.5:0/3935147403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55da7fc700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55dc103480 0x7f55dc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.122+0000 7f55db7fe700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc108a70 0x7f55dc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55daffd700 1 -- 172.21.15.5:0/3935147403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55dc103480 msgr2=0x7f55dc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55daffd700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55dc103480 0x7f55dc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55daffd700 1 -- 172.21.15.5:0/3935147403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc108a70 msgr2=0x7f55dc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55daffd700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc108a70 0x7f55dc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55daffd700 1 -- 172.21.15.5:0/3935147403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1a7670 con 0x7f55dc102ac0 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55db7fe700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc108a70 0x7f55dc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55daffd700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc074d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00c920 tx=0x7f55cc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc007b00 con 0x7f55dc102ac0 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1a7960 con 0x7f55dc102ac0 2024-04-22T13:26:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55cc004ce0 con 0x7f55dc102ac0 2024-04-22T13:26:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc005500 con 0x7f55dc102ac0 2024-04-22T13:26:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.123+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a7ec0 con 0x7f55dc102ac0 2024-04-22T13:26:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.124+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc069730 con 0x7f55dc102ac0 2024-04-22T13:26:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.125+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f55cc004830 con 0x7f55dc102ac0 2024-04-22T13:26:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.126+0000 7f55c3fff700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55c40a3cd0 0x7f55c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.126+0000 7f55da7fc700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55c40a3cd0 0x7f55c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.126+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f55cc0d0180 con 0x7f55dc102ac0 2024-04-22T13:26:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.127+0000 7f55da7fc700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55c40a3cd0 0x7f55c40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b5c0 tx=0x7f55d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.131+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55cc014030 con 0x7f55dc102ac0 2024-04-22T13:26:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.344+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f55dc02df80 con 0x7f55dc102ac0 2024-04-22T13:26:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.345+0000 7f55c3fff700 1 -- 172.21.15.5:0/3935147403 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55cc09ca10 con 0x7f55dc102ac0 2024-04-22T13:26:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:171798691896 2024-04-22T13:26:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55c40a3cd0 msgr2=0x7f55c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55c40a3cd0 0x7f55c40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b5c0 tx=0x7f55d0005960 comp rx=0 tx=0).stop 2024-04-22T13:26:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 msgr2=0x7f55dc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc074d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00c920 tx=0x7f55cc00cc30 comp rx=0 tx=0).stop 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 shutdown_connections 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55c40a3cd0 0x7f55c40a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55dc108a70 0x7f55dc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55dc102ac0 0x7f55dc074d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 --2- 172.21.15.5:0/3935147403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55dc103480 0x7f55dc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 >> 172.21.15.5:0/3935147403 conn(0x7f55dc079bb0 msgr2=0x7f55dc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 shutdown_connections 2024-04-22T13:26:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:17.347+0000 7f55e1549700 1 -- 172.21.15.5:0/3935147403 wait complete. 2024-04-22T13:26:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:17.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738448 for osd.0 2024-04-22T13:26:17.565 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:17 smithi049 ceph-mon[31379]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-04-22T13:26:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:17 smithi049 ceph-mon[31379]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-22T13:26:17.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:17 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3935147403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T13:26:17.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:17 smithi107 ceph-mon[31859]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-04-22T13:26:17.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:17 smithi107 ceph-mon[31859]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-22T13:26:17.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:17 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3935147403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T13:26:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:17 smithi005 ceph-mon[28006]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-04-22T13:26:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:17 smithi005 ceph-mon[28006]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-22T13:26:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:17 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3935147403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T13:26:18.145 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691892 got 171798691896 for osd.5 2024-04-22T13:26:18.145 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:18.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574916 got 55834574922 for osd.1 2024-04-22T13:26:18.509 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:19.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:19 smithi005 ceph-mon[28006]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-22T13:26:19.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:19 smithi049 ceph-mon[31379]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-22T13:26:19.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:19 smithi107 ceph-mon[31859]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-22T13:26:20.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 -- 172.21.15.5:0/77808840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0102ac0 msgr2=0x7fd2e0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:20.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/77808840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0102ac0 0x7fd2e0102f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009990 tx=0x7fd2d4009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:20.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 -- 172.21.15.5:0/77808840 shutdown_connections 2024-04-22T13:26:20.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/77808840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2e0108a70 0x7fd2e0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/77808840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e0103480 0x7fd2e010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/77808840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0102ac0 0x7fd2e0102f40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 -- 172.21.15.5:0/77808840 >> 172.21.15.5:0/77808840 conn(0x7fd2e0079bb0 msgr2=0x7fd2e0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 -- 172.21.15.5:0/77808840 shutdown_connections 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.807+0000 7fd2e61c3700 1 -- 172.21.15.5:0/77808840 wait complete. 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 Processor -- start 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 -- start start 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e0102ac0 0x7fd2e0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2e0103480 0x7fd2e0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:20.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 0x7fd2e019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0112c40 con 0x7fd2e0108a70 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0112ac0 con 0x7fd2e0103480 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2e61c3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0112dc0 con 0x7fd2e0102ac0 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.808+0000 7fd2dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 0x7fd2e019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 0x7fd2e019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49782/0 (socket says 172.21.15.5:49782) 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 -- 172.21.15.5:0/2577577932 learned_addr learned my addr 172.21.15.5:0/2577577932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2deffd700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2e0103480 0x7fd2e0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2df7fe700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e0102ac0 0x7fd2e0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 -- 172.21.15.5:0/2577577932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e0102ac0 msgr2=0x7fd2e0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e0102ac0 0x7fd2e0199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 -- 172.21.15.5:0/2577577932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2e0103480 msgr2=0x7fd2e0199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2e0103480 0x7fd2e0199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 -- 172.21.15.5:0/2577577932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e019e230 con 0x7fd2e0108a70 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dffff700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 0x7fd2e019daf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800ada0 tx=0x7fd2d800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8017020 con 0x7fd2e0108a70 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e019e520 con 0x7fd2e0108a70 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d800ce30 con 0x7fd2e0108a70 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8005350 con 0x7fd2e0108a70 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.809+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e01a5da0 con 0x7fd2e0108a70 2024-04-22T13:26:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.810+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e0069730 con 0x7fd2e0108a70 2024-04-22T13:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.811+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd2d800c980 con 0x7fd2e0108a70 2024-04-22T13:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.812+0000 7fd2dcff9700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd2c80a3cd0 0x7fd2c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.812+0000 7fd2df7fe700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd2c80a3cd0 0x7fd2c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.812+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd2d80d02a0 con 0x7fd2e0108a70 2024-04-22T13:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.813+0000 7fd2df7fe700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd2c80a3cd0 0x7fd2c80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0005fd0 tx=0x7fd2d0005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:20.816+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2d8021020 con 0x7fd2e0108a70 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.016+0000 7fcc5b373700 1 -- 172.21.15.5:0/226632486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54102ac0 msgr2=0x7fcc54102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.016+0000 7fcc5b373700 1 --2- 172.21.15.5:0/226632486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54102ac0 0x7fcc54102f40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009990 tx=0x7fcc48009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 -- 172.21.15.5:0/226632486 shutdown_connections 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 --2- 172.21.15.5:0/226632486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc54108ab0 0x7fcc54108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 --2- 172.21.15.5:0/226632486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc54103480 0x7fcc5410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 --2- 172.21.15.5:0/226632486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54102ac0 0x7fcc54102f40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 -- 172.21.15.5:0/226632486 >> 172.21.15.5:0/226632486 conn(0x7fcc54079bf0 msgr2=0x7fcc5407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 -- 172.21.15.5:0/226632486 shutdown_connections 2024-04-22T13:26:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 -- 172.21.15.5:0/226632486 wait complete. 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.017+0000 7fcc5b373700 1 Processor -- start 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 -- start start 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc54102ac0 0x7fcc541990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 0x7fcc54199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc54108ab0 0x7fcc5419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 0x7fcc54199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54112cb0 con 0x7fcc54103480 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54112b30 con 0x7fcc54108ab0 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5b373700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54112e30 con 0x7fcc54102ac0 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 0x7fcc54199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49802/0 (socket says 172.21.15.5:49802) 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 -- 172.21.15.5:0/497176116 learned_addr learned my addr 172.21.15.5:0/497176116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5910f700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc54102ac0 0x7fcc541990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc59910700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc54108ab0 0x7fcc5419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 -- 172.21.15.5:0/497176116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc54102ac0 msgr2=0x7fcc541990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc54102ac0 0x7fcc541990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 -- 172.21.15.5:0/497176116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc54108ab0 msgr2=0x7fcc5419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc54108ab0 0x7fcc5419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.018+0000 7fcc5890e700 1 -- 172.21.15.5:0/497176116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5419e0a0 con 0x7fcc54103480 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.019+0000 7fcc5890e700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 0x7fcc54199620 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.019+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48027070 con 0x7fcc54103480 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.019+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5419e330 con 0x7fcc54103480 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.019+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc48003f20 con 0x7fcc54103480 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.019+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48022cf0 con 0x7fcc54103480 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.019+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a5c10 con 0x7fcc54103480 2024-04-22T13:26:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.020+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc54069730 con 0x7fcc54103480 2024-04-22T13:26:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.021+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fcc4802a070 con 0x7fcc54103480 2024-04-22T13:26:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.022+0000 7fcc467fc700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcc400a3cd0 0x7fcc400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.022+0000 7fcc5910f700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcc400a3cd0 0x7fcc400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.022+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fcc480d1750 con 0x7fcc54103480 2024-04-22T13:26:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.023+0000 7fcc5910f700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcc400a3cd0 0x7fcc400a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000abd0 tx=0x7fcc5000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.026+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc48013070 con 0x7fcc54103480 2024-04-22T13:26:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.036+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd2e019a580 con 0x7fd2e0108a70 2024-04-22T13:26:21.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.036+0000 7fd2dcff9700 1 -- 172.21.15.5:0/2577577932 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd2d801a310 con 0x7fd2e0108a70 2024-04-22T13:26:21.033 INFO:teuthology.orchestra.run.smithi005.stdout:326417514527 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd2c80a3cd0 msgr2=0x7fd2c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd2c80a3cd0 0x7fd2c80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0005fd0 tx=0x7fd2d0005db0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 msgr2=0x7fd2e019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 0x7fd2e019daf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800ada0 tx=0x7fd2d800c330 comp rx=0 tx=0).stop 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 shutdown_connections 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e0108a70 0x7fd2e019daf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e0102ac0 0x7fd2e0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd2c80a3cd0 0x7fd2c80a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 --2- 172.21.15.5:0/2577577932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2e0103480 0x7fd2e0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 >> 172.21.15.5:0/2577577932 conn(0x7fd2e0079bb0 msgr2=0x7fd2e0101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 shutdown_connections 2024-04-22T13:26:21.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.039+0000 7fd2e61c3700 1 -- 172.21.15.5:0/2577577932 wait complete. 2024-04-22T13:26:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.252+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcc5402df80 con 0x7fcc54103480 2024-04-22T13:26:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.252+0000 7fcc467fc700 1 -- 172.21.15.5:0/497176116 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc4809e010 con 0x7fcc54103480 2024-04-22T13:26:21.249 INFO:teuthology.orchestra.run.smithi005.stdout:206158430258 2024-04-22T13:26:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcc400a3cd0 msgr2=0x7fcc400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcc400a3cd0 0x7fcc400a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000abd0 tx=0x7fcc5000a380 comp rx=0 tx=0).stop 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 msgr2=0x7fcc54199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 0x7fcc54199620 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc480047b0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 shutdown_connections 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcc400a3cd0 0x7fcc400a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc54108ab0 0x7fcc5419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc54102ac0 0x7fcc541990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 --2- 172.21.15.5:0/497176116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc54103480 0x7fcc54199620 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 >> 172.21.15.5:0/497176116 conn(0x7fcc54079bf0 msgr2=0x7fcc54100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 shutdown_connections 2024-04-22T13:26:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.254+0000 7fcc5b373700 1 -- 172.21.15.5:0/497176116 wait complete. 2024-04-22T13:26:21.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:21 smithi005 ceph-mon[28006]: pgmap v324: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T13:26:21.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:21 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2577577932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T13:26:21.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:21 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/497176116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 -- 172.21.15.5:0/2214333887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 msgr2=0x7f5d00107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 --2- 172.21.15.5:0/2214333887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d00107d70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009990 tx=0x7f5cfc009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 -- 172.21.15.5:0/2214333887 shutdown_connections 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 --2- 172.21.15.5:0/2214333887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d00102300 0x7f5d0010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 --2- 172.21.15.5:0/2214333887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d00107d70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 --2- 172.21.15.5:0/2214333887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d00101940 0x7f5d00101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 -- 172.21.15.5:0/2214333887 >> 172.21.15.5:0/2214333887 conn(0x7f5d000788f0 msgr2=0x7f5d00078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 -- 172.21.15.5:0/2214333887 shutdown_connections 2024-04-22T13:26:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 -- 172.21.15.5:0/2214333887 wait complete. 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 Processor -- start 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.637+0000 7f5d07e43700 1 -- start start 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d00101940 0x7f5d00199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d00102300 0x7f5d001997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d0019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111bd0 con 0x7f5d00107990 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111a50 con 0x7f5d00101940 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111d50 con 0x7f5d00102300 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d0019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d0019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49824/0 (socket says 172.21.15.5:49824) 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 -- 172.21.15.5:0/1717848340 learned_addr learned my addr 172.21.15.5:0/1717848340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d053de700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d00102300 0x7f5d001997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d05bdf700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d00101940 0x7f5d00199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 -- 172.21.15.5:0/1717848340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d00102300 msgr2=0x7f5d001997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d00102300 0x7f5d001997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 -- 172.21.15.5:0/1717848340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d00101940 msgr2=0x7f5d00199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d00101940 0x7f5d00199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 -- 172.21.15.5:0/1717848340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0019e2a0 con 0x7f5d00107990 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d063e0700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d0019db60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800b970 tx=0x7f5cf800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8019070 con 0x7f5d00107990 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0019e590 con 0x7f5d00107990 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.638+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf800cde0 con 0x7f5d00107990 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.639+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8007370 con 0x7f5d00107990 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.639+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d001a5e10 con 0x7f5d00107990 2024-04-22T13:26:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.639+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d00069730 con 0x7f5d00107990 2024-04-22T13:26:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.641+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f5cf801c070 con 0x7f5d00107990 2024-04-22T13:26:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.641+0000 7f5cf6ffd700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5cec0a3cd0 0x7f5cec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.641+0000 7f5d05bdf700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5cec0a3cd0 0x7f5cec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.641+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f5cf80d00a0 con 0x7f5d00107990 2024-04-22T13:26:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.642+0000 7f5d05bdf700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5cec0a3cd0 0x7f5cec0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc006010 tx=0x7f5cfc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.645+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cf8023070 con 0x7f5d00107990 2024-04-22T13:26:21.770 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:21 smithi049 ceph-mon[31379]: pgmap v324: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T13:26:21.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:21 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2577577932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T13:26:21.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:21 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/497176116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T13:26:21.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:21 smithi107 ceph-mon[31859]: pgmap v324: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T13:26:21.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:21 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2577577932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T13:26:21.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:21 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/497176116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T13:26:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.856+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5d001a5fe0 con 0x7f5d00107990 2024-04-22T13:26:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.857+0000 7f5cf6ffd700 1 -- 172.21.15.5:0/1717848340 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5cf809ca40 con 0x7f5d00107990 2024-04-22T13:26:21.854 INFO:teuthology.orchestra.run.smithi005.stdout:111669149762 2024-04-22T13:26:21.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5cec0a3cd0 msgr2=0x7f5cec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5cec0a3cd0 0x7f5cec0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc006010 tx=0x7f5cfc01a040 comp rx=0 tx=0).stop 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 msgr2=0x7f5d0019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d0019db60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800b970 tx=0x7f5cf800bc80 comp rx=0 tx=0).stop 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 shutdown_connections 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f5cec0a3cd0 0x7f5cec0a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d00101940 0x7f5d00199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d00107990 0x7f5d0019db60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 --2- 172.21.15.5:0/1717848340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d00102300 0x7f5d001997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 >> 172.21.15.5:0/1717848340 conn(0x7f5d000788f0 msgr2=0x7f5d0006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 shutdown_connections 2024-04-22T13:26:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:21.859+0000 7f5d07e43700 1 -- 172.21.15.5:0/1717848340 wait complete. 2024-04-22T13:26:22.387 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514521 got 326417514527 for osd.10 2024-04-22T13:26:22.387 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:22.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:22 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1717848340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T13:26:22.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:22 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1717848340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T13:26:22.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:22 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1717848340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T13:26:23.302 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149757 got 111669149762 for osd.3 2024-04-22T13:26:23.303 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:23 smithi005 ceph-mon[28006]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T13:26:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:23 smithi005 ceph-mon[28006]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-22T13:26:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:23 smithi005 ceph-mon[28006]: Cluster is now healthy 2024-04-22T13:26:23.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:23 smithi049 ceph-mon[31379]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T13:26:23.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:23 smithi049 ceph-mon[31379]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-22T13:26:23.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:23 smithi049 ceph-mon[31379]: Cluster is now healthy 2024-04-22T13:26:23.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:23 smithi107 ceph-mon[31859]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T13:26:23.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:23 smithi107 ceph-mon[31859]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-22T13:26:23.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:23 smithi107 ceph-mon[31859]: Cluster is now healthy 2024-04-22T13:26:23.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430253 got 206158430258 for osd.6 2024-04-22T13:26:23.889 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:23.889 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-22T13:26:23.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T13:26:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 -- 172.21.15.5:0/972249636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc108a20 msgr2=0x7f7efc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 --2- 172.21.15.5:0/972249636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc108a20 0x7f7efc108e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009990 tx=0x7f7ee4009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 -- 172.21.15.5:0/972249636 shutdown_connections 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 --2- 172.21.15.5:0/972249636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7efc06b980 0x7f7efc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 --2- 172.21.15.5:0/972249636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc108a20 0x7f7efc108e00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 --2- 172.21.15.5:0/972249636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7efc06afe0 0x7f7efc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 -- 172.21.15.5:0/972249636 >> 172.21.15.5:0/972249636 conn(0x7f7efc0788b0 msgr2=0x7f7efc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 -- 172.21.15.5:0/972249636 shutdown_connections 2024-04-22T13:26:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.746+0000 7f7f03161700 1 -- 172.21.15.5:0/972249636 wait complete. 2024-04-22T13:26:25.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:25 smithi005 ceph-mon[28006]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-22T13:26:25.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:25.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:25 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 Processor -- start 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 -- start start 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7efc06afe0 0x7f7efc101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 0x7f7efc101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7efc108a20 0x7f7efc102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc112ca0 con 0x7f7efc06b980 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc112b20 con 0x7f7efc06afe0 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f03161700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc112e20 con 0x7f7efc108a20 2024-04-22T13:26:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7efbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 0x7f7efc101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7efbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 0x7f7efc101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49854/0 (socket says 172.21.15.5:49854) 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7efbfff700 1 -- 172.21.15.5:0/1520801659 learned_addr learned my addr 172.21.15.5:0/1520801659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.747+0000 7f7f00efd700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7efc06afe0 0x7f7efc101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7f016fe700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7efc108a20 0x7f7efc102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7efbfff700 1 -- 172.21.15.5:0/1520801659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7efc108a20 msgr2=0x7f7efc102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7efbfff700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7efc108a20 0x7f7efc102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7efbfff700 1 -- 172.21.15.5:0/1520801659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7efc06afe0 msgr2=0x7f7efc101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7efbfff700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7efc06afe0 0x7f7efc101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7efbfff700 1 -- 172.21.15.5:0/1520801659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1a77c0 con 0x7f7efc06b980 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7efbfff700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 0x7f7efc101f40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00ba50 tx=0x7f7eec00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec019070 con 0x7f7efc06b980 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1a7ab0 con 0x7f7efc06b980 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec00cde0 con 0x7f7efc06b980 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.748+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007370 con 0x7f7efc06b980 2024-04-22T13:26:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.749+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1a8040 con 0x7f7efc06b980 2024-04-22T13:26:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.750+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc069730 con 0x7f7efc06b980 2024-04-22T13:26:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.751+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f7eec01c070 con 0x7f7efc06b980 2024-04-22T13:26:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.751+0000 7f7ef9ffb700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7ee80a3d50 0x7f7ee80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.751+0000 7f7f00efd700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7ee80a3d50 0x7f7ee80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.751+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f7eec0cf870 con 0x7f7efc06b980 2024-04-22T13:26:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.752+0000 7f7f00efd700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7ee80a3d50 0x7f7ee80a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40098e0 tx=0x7f7ee4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.756+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7eec09c210 con 0x7f7efc06b980 2024-04-22T13:26:25.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:25 smithi049 ceph-mon[31379]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-22T13:26:25.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:25.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:25 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:25.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:25 smithi107 ceph-mon[31859]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-22T13:26:25.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:25.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:25 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:25.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.969+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7efc062e80 con 0x7f7ee80a3d50 2024-04-22T13:26:25.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.977+0000 7f7ef9ffb700 1 -- 172.21.15.5:0/1520801659 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258772 (secure 0 0 0) 0x7f7efc062e80 con 0x7f7ee80a3d50 2024-04-22T13:26:25.976 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7ee80a3d50 msgr2=0x7f7ee80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7ee80a3d50 0x7f7ee80a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40098e0 tx=0x7f7ee4005960 comp rx=0 tx=0).stop 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 msgr2=0x7f7efc101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 0x7f7efc101f40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00ba50 tx=0x7f7eec00bd60 comp rx=0 tx=0).stop 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 shutdown_connections 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7ee80a3d50 0x7f7ee80a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7efc06afe0 0x7f7efc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7efc108a20 0x7f7efc102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 --2- 172.21.15.5:0/1520801659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7efc06b980 0x7f7efc101f40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 >> 172.21.15.5:0/1520801659 conn(0x7f7efc0788b0 msgr2=0x7f7efc0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 shutdown_connections 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:25.981+0000 7f7f03161700 1 -- 172.21.15.5:0/1520801659 wait complete. 2024-04-22T13:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:26:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:26:26.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:26:26.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:26 smithi005 ceph-mon[28006]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:26.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:26 smithi005 ceph-mon[28006]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:26:26.635 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-22T13:26:24.206926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656256,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":828,"num_read_kb":540,"num_write":417,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404676,"kb_used_data":13836,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537148,"statfs":{"total":1151940427776,"available":1151526039552,"internally_reserved":0,"allocated":14168064,"data_stored":7660464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.755216"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457418+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:26:15.457418+0000","last_peered":"2024-04-22T13:26:15.457418+0000","last_clean":"2024-04-22T13:26:15.457418+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:26:15.457418+0000","last_undegraded":"2024-04-22T13:26:15.457418+0000","last_fullsized":"2024-04-22T13:26:15.457418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457956+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:26:15.457956+0000","last_peered":"2024-04-22T13:26:15.457956+0000","last_clean":"2024-04-22T13:26:15.457956+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:26:15.457956+0000","last_undegraded":"2024-04-22T13:26:15.457956+0000","last_fullsized":"2024-04-22T13:26:15.457956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457942+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:26:15.457942+0000","last_peered":"2024-04-22T13:26:15.457942+0000","last_clean":"2024-04-22T13:26:15.457942+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:26:15.457942+0000","last_undegraded":"2024-04-22T13:26:15.457942+0000","last_fullsized":"2024-04-22T13:26:15.457942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455570+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:26:15.455570+0000","last_peered":"2024-04-22T13:26:15.455570+0000","last_clean":"2024-04-22T13:26:15.455570+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:26:15.455570+0000","last_undegraded":"2024-04-22T13:26:15.455570+0000","last_fullsized":"2024-04-22T13:26:15.455570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460330+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:26:15.460330+0000","last_peered":"2024-04-22T13:26:15.460330+0000","last_clean":"2024-04-22T13:26:15.460330+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:26:15.460330+0000","last_undegraded":"2024-04-22T13:26:15.460330+0000","last_fullsized":"2024-04-22T13:26:15.460330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515185+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:26:15.515185+0000","last_peered":"2024-04-22T13:26:15.515185+0000","last_clean":"2024-04-22T13:26:15.515185+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:26:15.515185+0000","last_undegraded":"2024-04-22T13:26:15.515185+0000","last_fullsized":"2024-04-22T13:26:15.515185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460180+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:26:15.460180+0000","last_peered":"2024-04-22T13:26:15.460180+0000","last_clean":"2024-04-22T13:26:15.460180+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:26:15.460180+0000","last_undegraded":"2024-04-22T13:26:15.460180+0000","last_fullsized":"2024-04-22T13:26:15.460180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459528+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:26:15.459528+0000","last_peered":"2024-04-22T13:26:15.459528+0000","last_clean":"2024-04-22T13:26:15.459528+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:26:15.459528+0000","last_undegraded":"2024-04-22T13:26:15.459528+0000","last_fullsized":"2024-04-22T13:26:15.459528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515484+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:26:15.515484+0000","last_peered":"2024-04-22T13:26:15.515484+0000","last_clean":"2024-04-22T13:26:15.515484+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:26:15.515484+0000","last_undegraded":"2024-04-22T13:26:15.515484+0000","last_fullsized":"2024-04-22T13:26:15.515484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515163+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:26:15.515163+0000","last_peered":"2024-04-22T13:26:15.515163+0000","last_clean":"2024-04-22T13:26:15.515163+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:26:15.515163+0000","last_undegraded":"2024-04-22T13:26:15.515163+0000","last_fullsized":"2024-04-22T13:26:15.515163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460618+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:26:15.460618+0000","last_peered":"2024-04-22T13:26:15.460618+0000","last_clean":"2024-04-22T13:26:15.460618+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:26:15.460618+0000","last_undegraded":"2024-04-22T13:26:15.460618+0000","last_fullsized":"2024-04-22T13:26:15.460618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455531+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:26:15.455531+0000","last_peered":"2024-04-22T13:26:15.455531+0000","last_clean":"2024-04-22T13:26:15.455531+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:26:15.455531+0000","last_undegraded":"2024-04-22T13:26:15.455531+0000","last_fullsized":"2024-04-22T13:26:15.455531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456372+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:26:15.456372+0000","last_peered":"2024-04-22T13:26:15.456372+0000","last_clean":"2024-04-22T13:26:15.456372+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:26:15.456372+0000","last_undegraded":"2024-04-22T13:26:15.456372+0000","last_fullsized":"2024-04-22T13:26:15.456372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457324+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:26:15.457324+0000","last_peered":"2024-04-22T13:26:15.457324+0000","last_clean":"2024-04-22T13:26:15.457324+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:26:15.457324+0000","last_undegraded":"2024-04-22T13:26:15.457324+0000","last_fullsized":"2024-04-22T13:26:15.457324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457305+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:26:15.457305+0000","last_peered":"2024-04-22T13:26:15.457305+0000","last_clean":"2024-04-22T13:26:15.457305+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:26:15.457305+0000","last_undegraded":"2024-04-22T13:26:15.457305+0000","last_fullsized":"2024-04-22T13:26:15.457305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514704+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:26:15.514704+0000","last_peered":"2024-04-22T13:26:15.514704+0000","last_clean":"2024-04-22T13:26:15.514704+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:26:15.514704+0000","last_undegraded":"2024-04-22T13:26:15.514704+0000","last_fullsized":"2024-04-22T13:26:15.514704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457627+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:26:15.457627+0000","last_peered":"2024-04-22T13:26:15.457627+0000","last_clean":"2024-04-22T13:26:15.457627+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:26:15.457627+0000","last_undegraded":"2024-04-22T13:26:15.457627+0000","last_fullsized":"2024-04-22T13:26:15.457627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454145+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:26:15.454145+0000","last_peered":"2024-04-22T13:26:15.454145+0000","last_clean":"2024-04-22T13:26:15.454145+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:26:15.454145+0000","last_undegraded":"2024-04-22T13:26:15.454145+0000","last_fullsized":"2024-04-22T13:26:15.454145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455134+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:26:15.455134+0000","last_peered":"2024-04-22T13:26:15.455134+0000","last_clean":"2024-04-22T13:26:15.455134+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:26:15.455134+0000","last_undegraded":"2024-04-22T13:26:15.455134+0000","last_fullsized":"2024-04-22T13:26:15.455134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:21.330633+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:26:21.330633+0000","last_peered":"2024-04-22T13:26:21.330633+0000","last_clean":"2024-04-22T13:26:21.330633+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:26:21.330633+0000","last_undegraded":"2024-04-22T13:26:21.330633+0000","last_fullsized":"2024-04-22T13:26:21.330633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459882+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:26:15.459882+0000","last_peered":"2024-04-22T13:26:15.459882+0000","last_clean":"2024-04-22T13:26:15.459882+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:26:15.459882+0000","last_undegraded":"2024-04-22T13:26:15.459882+0000","last_fullsized":"2024-04-22T13:26:15.459882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460034+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:26:15.460034+0000","last_peered":"2024-04-22T13:26:15.460034+0000","last_clean":"2024-04-22T13:26:15.460034+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:26:15.460034+0000","last_undegraded":"2024-04-22T13:26:15.460034+0000","last_fullsized":"2024-04-22T13:26:15.460034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457537+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:26:15.457537+0000","last_peered":"2024-04-22T13:26:15.457537+0000","last_clean":"2024-04-22T13:26:15.457537+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:26:15.457537+0000","last_undegraded":"2024-04-22T13:26:15.457537+0000","last_fullsized":"2024-04-22T13:26:15.457537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.473813+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:26:15.473813+0000","last_peered":"2024-04-22T13:26:15.473813+0000","last_clean":"2024-04-22T13:26:15.473813+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:26:15.473813+0000","last_undegraded":"2024-04-22T13:26:15.473813+0000","last_fullsized":"2024-04-22T13:26:15.473813+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514831+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:26:15.514831+0000","last_peered":"2024-04-22T13:26:15.514831+0000","last_clean":"2024-04-22T13:26:15.514831+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:26:15.514831+0000","last_undegraded":"2024-04-22T13:26:15.514831+0000","last_fullsized":"2024-04-22T13:26:15.514831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460781+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:26:15.460781+0000","last_peered":"2024-04-22T13:26:15.460781+0000","last_clean":"2024-04-22T13:26:15.460781+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:26:15.460781+0000","last_undegraded":"2024-04-22T13:26:15.460781+0000","last_fullsized":"2024-04-22T13:26:15.460781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514984+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:26:15.514984+0000","last_peered":"2024-04-22T13:26:15.514984+0000","last_clean":"2024-04-22T13:26:15.514984+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:26:15.514984+0000","last_undegraded":"2024-04-22T13:26:15.514984+0000","last_fullsized":"2024-04-22T13:26:15.514984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.333757+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:26:22.333757+0000","last_peered":"2024-04-22T13:26:22.333757+0000","last_clean":"2024-04-22T13:26:22.333757+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:26:22.333757+0000","last_undegraded":"2024-04-22T13:26:22.333757+0000","last_fullsized":"2024-04-22T13:26:22.333757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461017+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:26:15.461017+0000","last_peered":"2024-04-22T13:26:15.461017+0000","last_clean":"2024-04-22T13:26:15.461017+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:26:15.461017+0000","last_undegraded":"2024-04-22T13:26:15.461017+0000","last_fullsized":"2024-04-22T13:26:15.461017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515311+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:26:15.515311+0000","last_peered":"2024-04-22T13:26:15.515311+0000","last_clean":"2024-04-22T13:26:15.515311+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:26:15.515311+0000","last_undegraded":"2024-04-22T13:26:15.515311+0000","last_fullsized":"2024-04-22T13:26:15.515311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514710+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:26:15.514710+0000","last_peered":"2024-04-22T13:26:15.514710+0000","last_clean":"2024-04-22T13:26:15.514710+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:26:15.514710+0000","last_undegraded":"2024-04-22T13:26:15.514710+0000","last_fullsized":"2024-04-22T13:26:15.514710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455542+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:26:15.455542+0000","last_peered":"2024-04-22T13:26:15.455542+0000","last_clean":"2024-04-22T13:26:15.455542+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:26:15.455542+0000","last_undegraded":"2024-04-22T13:26:15.455542+0000","last_fullsized":"2024-04-22T13:26:15.455542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459629+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:26:15.459629+0000","last_peered":"2024-04-22T13:26:15.459629+0000","last_clean":"2024-04-22T13:26:15.459629+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:26:15.459629+0000","last_undegraded":"2024-04-22T13:26:15.459629+0000","last_fullsized":"2024-04-22T13:26:15.459629+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514508+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:26:15.514508+0000","last_peered":"2024-04-22T13:26:15.514508+0000","last_clean":"2024-04-22T13:26:15.514508+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:26:15.514508+0000","last_undegraded":"2024-04-22T13:26:15.514508+0000","last_fullsized":"2024-04-22T13:26:15.514508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.453875+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:26:15.453875+0000","last_peered":"2024-04-22T13:26:15.453875+0000","last_clean":"2024-04-22T13:26:15.453875+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:26:15.453875+0000","last_undegraded":"2024-04-22T13:26:15.453875+0000","last_fullsized":"2024-04-22T13:26:15.453875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.333152+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:26:22.333152+0000","last_peered":"2024-04-22T13:26:22.333152+0000","last_clean":"2024-04-22T13:26:22.333152+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:26:22.333152+0000","last_undegraded":"2024-04-22T13:26:22.333152+0000","last_fullsized":"2024-04-22T13:26:22.333152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515596+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:26:15.515596+0000","last_peered":"2024-04-22T13:26:15.515596+0000","last_clean":"2024-04-22T13:26:15.515596+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:26:15.515596+0000","last_undegraded":"2024-04-22T13:26:15.515596+0000","last_fullsized":"2024-04-22T13:26:15.515596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460664+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:26:15.460664+0000","last_peered":"2024-04-22T13:26:15.460664+0000","last_clean":"2024-04-22T13:26:15.460664+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:26:15.460664+0000","last_undegraded":"2024-04-22T13:26:15.460664+0000","last_fullsized":"2024-04-22T13:26:15.460664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.516279+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:26:15.516279+0000","last_peered":"2024-04-22T13:26:15.516279+0000","last_clean":"2024-04-22T13:26:15.516279+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:26:15.516279+0000","last_undegraded":"2024-04-22T13:26:15.516279+0000","last_fullsized":"2024-04-22T13:26:15.516279+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515193+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:26:15.515193+0000","last_peered":"2024-04-22T13:26:15.515193+0000","last_clean":"2024-04-22T13:26:15.515193+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:26:15.515193+0000","last_undegraded":"2024-04-22T13:26:15.515193+0000","last_fullsized":"2024-04-22T13:26:15.515193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515050+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:26:15.515050+0000","last_peered":"2024-04-22T13:26:15.515050+0000","last_clean":"2024-04-22T13:26:15.515050+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:26:15.515050+0000","last_undegraded":"2024-04-22T13:26:15.515050+0000","last_fullsized":"2024-04-22T13:26:15.515050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'28","reported_seq":185,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.340140+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:26:22.340140+0000","last_peered":"2024-04-22T13:26:22.340140+0000","last_clean":"2024-04-22T13:26:22.340140+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:26:22.340140+0000","last_undegraded":"2024-04-22T13:26:22.340140+0000","last_fullsized":"2024-04-22T13:26:22.340140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58841,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":127,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461477+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:26:15.461477+0000","last_peered":"2024-04-22T13:26:15.461477+0000","last_clean":"2024-04-22T13:26:15.461477+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:26:15.461477+0000","last_undegraded":"2024-04-22T13:26:15.461477+0000","last_fullsized":"2024-04-22T13:26:15.461477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454737+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:26:15.454737+0000","last_peered":"2024-04-22T13:26:15.454737+0000","last_clean":"2024-04-22T13:26:15.454737+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:26:15.454737+0000","last_undegraded":"2024-04-22T13:26:15.454737+0000","last_fullsized":"2024-04-22T13:26:15.454737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455592+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:26:15.455592+0000","last_peered":"2024-04-22T13:26:15.455592+0000","last_clean":"2024-04-22T13:26:15.455592+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:26:15.455592+0000","last_undegraded":"2024-04-22T13:26:15.455592+0000","last_fullsized":"2024-04-22T13:26:15.455592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514494+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:26:15.514494+0000","last_peered":"2024-04-22T13:26:15.514494+0000","last_clean":"2024-04-22T13:26:15.514494+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:26:15.514494+0000","last_undegraded":"2024-04-22T13:26:15.514494+0000","last_fullsized":"2024-04-22T13:26:15.514494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460279+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:26:15.460279+0000","last_peered":"2024-04-22T13:26:15.460279+0000","last_clean":"2024-04-22T13:26:15.460279+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:26:15.460279+0000","last_undegraded":"2024-04-22T13:26:15.460279+0000","last_fullsized":"2024-04-22T13:26:15.460279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:17.332881+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:26:17.332881+0000","last_peered":"2024-04-22T13:26:17.332881+0000","last_clean":"2024-04-22T13:26:17.332881+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:26:17.332881+0000","last_undegraded":"2024-04-22T13:26:17.332881+0000","last_fullsized":"2024-04-22T13:26:17.332881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459808+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:26:15.459808+0000","last_peered":"2024-04-22T13:26:15.459808+0000","last_clean":"2024-04-22T13:26:15.459808+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:26:15.459808+0000","last_undegraded":"2024-04-22T13:26:15.459808+0000","last_fullsized":"2024-04-22T13:26:15.459808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460239+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:26:15.460239+0000","last_peered":"2024-04-22T13:26:15.460239+0000","last_clean":"2024-04-22T13:26:15.460239+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:26:15.460239+0000","last_undegraded":"2024-04-22T13:26:15.460239+0000","last_fullsized":"2024-04-22T13:26:15.460239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514680+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:26:15.514680+0000","last_peered":"2024-04-22T13:26:15.514680+0000","last_clean":"2024-04-22T13:26:15.514680+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:26:15.514680+0000","last_undegraded":"2024-04-22T13:26:15.514680+0000","last_fullsized":"2024-04-22T13:26:15.514680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461355+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:26:15.461355+0000","last_peered":"2024-04-22T13:26:15.461355+0000","last_clean":"2024-04-22T13:26:15.461355+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:26:15.461355+0000","last_undegraded":"2024-04-22T13:26:15.461355+0000","last_fullsized":"2024-04-22T13:26:15.461355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457316+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:26:15.457316+0000","last_peered":"2024-04-22T13:26:15.457316+0000","last_clean":"2024-04-22T13:26:15.457316+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:26:15.457316+0000","last_undegraded":"2024-04-22T13:26:15.457316+0000","last_fullsized":"2024-04-22T13:26:15.457316+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454072+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:26:15.454072+0000","last_peered":"2024-04-22T13:26:15.454072+0000","last_clean":"2024-04-22T13:26:15.454072+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:26:15.454072+0000","last_undegraded":"2024-04-22T13:26:15.454072+0000","last_fullsized":"2024-04-22T13:26:15.454072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515481+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:26:15.515481+0000","last_peered":"2024-04-22T13:26:15.515481+0000","last_clean":"2024-04-22T13:26:15.515481+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:26:15.515481+0000","last_undegraded":"2024-04-22T13:26:15.515481+0000","last_fullsized":"2024-04-22T13:26:15.515481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.333082+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:26:22.333082+0000","last_peered":"2024-04-22T13:26:22.333082+0000","last_clean":"2024-04-22T13:26:22.333082+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:26:22.333082+0000","last_undegraded":"2024-04-22T13:26:22.333082+0000","last_fullsized":"2024-04-22T13:26:22.333082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515533+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:26:15.515533+0000","last_peered":"2024-04-22T13:26:15.515533+0000","last_clean":"2024-04-22T13:26:15.515533+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:26:15.515533+0000","last_undegraded":"2024-04-22T13:26:15.515533+0000","last_fullsized":"2024-04-22T13:26:15.515533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457644+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:26:15.457644+0000","last_peered":"2024-04-22T13:26:15.457644+0000","last_clean":"2024-04-22T13:26:15.457644+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:26:15.457644+0000","last_undegraded":"2024-04-22T13:26:15.457644+0000","last_fullsized":"2024-04-22T13:26:15.457644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":91,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:21.337443+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:26:21.337443+0000","last_peered":"2024-04-22T13:26:21.337443+0000","last_clean":"2024-04-22T13:26:21.337443+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:26:21.337443+0000","last_undegraded":"2024-04-22T13:26:21.337443+0000","last_fullsized":"2024-04-22T13:26:21.337443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460605+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:26:15.460605+0000","last_peered":"2024-04-22T13:26:15.460605+0000","last_clean":"2024-04-22T13:26:15.460605+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:26:15.460605+0000","last_undegraded":"2024-04-22T13:26:15.460605+0000","last_fullsized":"2024-04-22T13:26:15.460605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455295+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:26:15.455295+0000","last_peered":"2024-04-22T13:26:15.455295+0000","last_clean":"2024-04-22T13:26:15.455295+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:26:15.455295+0000","last_undegraded":"2024-04-22T13:26:15.455295+0000","last_fullsized":"2024-04-22T13:26:15.455295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459901+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:26:15.459901+0000","last_peered":"2024-04-22T13:26:15.459901+0000","last_clean":"2024-04-22T13:26:15.459901+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:26:15.459901+0000","last_undegraded":"2024-04-22T13:26:15.459901+0000","last_fullsized":"2024-04-22T13:26:15.459901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455425+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:26:15.455425+0000","last_peered":"2024-04-22T13:26:15.455425+0000","last_clean":"2024-04-22T13:26:15.455425+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:26:15.455425+0000","last_undegraded":"2024-04-22T13:26:15.455425+0000","last_fullsized":"2024-04-22T13:26:15.455425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460045+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:26:15.460045+0000","last_peered":"2024-04-22T13:26:15.460045+0000","last_clean":"2024-04-22T13:26:15.460045+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:26:15.460045+0000","last_undegraded":"2024-04-22T13:26:15.460045+0000","last_fullsized":"2024-04-22T13:26:15.460045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461596+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:26:15.461596+0000","last_peered":"2024-04-22T13:26:15.461596+0000","last_clean":"2024-04-22T13:26:15.461596+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:26:15.461596+0000","last_undegraded":"2024-04-22T13:26:15.461596+0000","last_fullsized":"2024-04-22T13:26:15.461596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456351+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:26:15.456351+0000","last_peered":"2024-04-22T13:26:15.456351+0000","last_clean":"2024-04-22T13:26:15.456351+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:26:15.456351+0000","last_undegraded":"2024-04-22T13:26:15.456351+0000","last_fullsized":"2024-04-22T13:26:15.456351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454186+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:26:15.454186+0000","last_peered":"2024-04-22T13:26:15.454186+0000","last_clean":"2024-04-22T13:26:15.454186+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:26:15.454186+0000","last_undegraded":"2024-04-22T13:26:15.454186+0000","last_fullsized":"2024-04-22T13:26:15.454186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459867+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:26:15.459867+0000","last_peered":"2024-04-22T13:26:15.459867+0000","last_clean":"2024-04-22T13:26:15.459867+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:26:15.459867+0000","last_undegraded":"2024-04-22T13:26:15.459867+0000","last_fullsized":"2024-04-22T13:26:15.459867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455515+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:26:15.455515+0000","last_peered":"2024-04-22T13:26:15.455515+0000","last_clean":"2024-04-22T13:26:15.455515+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:26:15.455515+0000","last_undegraded":"2024-04-22T13:26:15.455515+0000","last_fullsized":"2024-04-22T13:26:15.455515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.458058+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:26:15.458058+0000","last_peered":"2024-04-22T13:26:15.458058+0000","last_clean":"2024-04-22T13:26:15.458058+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:26:15.458058+0000","last_undegraded":"2024-04-22T13:26:15.458058+0000","last_fullsized":"2024-04-22T13:26:15.458058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514866+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:26:15.514866+0000","last_peered":"2024-04-22T13:26:15.514866+0000","last_clean":"2024-04-22T13:26:15.514866+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:26:15.514866+0000","last_undegraded":"2024-04-22T13:26:15.514866+0000","last_fullsized":"2024-04-22T13:26:15.514866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460401+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:26:15.460401+0000","last_peered":"2024-04-22T13:26:15.460401+0000","last_clean":"2024-04-22T13:26:15.460401+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:26:15.460401+0000","last_undegraded":"2024-04-22T13:26:15.460401+0000","last_fullsized":"2024-04-22T13:26:15.460401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459496+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:26:15.459496+0000","last_peered":"2024-04-22T13:26:15.459496+0000","last_clean":"2024-04-22T13:26:15.459496+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:26:15.459496+0000","last_undegraded":"2024-04-22T13:26:15.459496+0000","last_fullsized":"2024-04-22T13:26:15.459496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456857+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:26:15.456857+0000","last_peered":"2024-04-22T13:26:15.456857+0000","last_clean":"2024-04-22T13:26:15.456857+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:26:15.456857+0000","last_undegraded":"2024-04-22T13:26:15.456857+0000","last_fullsized":"2024-04-22T13:26:15.456857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.477566+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:26:15.477566+0000","last_peered":"2024-04-22T13:26:15.477566+0000","last_clean":"2024-04-22T13:26:15.477566+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:26:15.477566+0000","last_undegraded":"2024-04-22T13:26:15.477566+0000","last_fullsized":"2024-04-22T13:26:15.477566+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459520+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:26:15.459520+0000","last_peered":"2024-04-22T13:26:15.459520+0000","last_clean":"2024-04-22T13:26:15.459520+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:26:15.459520+0000","last_undegraded":"2024-04-22T13:26:15.459520+0000","last_fullsized":"2024-04-22T13:26:15.459520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460456+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:26:15.460456+0000","last_peered":"2024-04-22T13:26:15.460456+0000","last_clean":"2024-04-22T13:26:15.460456+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:26:15.460456+0000","last_undegraded":"2024-04-22T13:26:15.460456+0000","last_fullsized":"2024-04-22T13:26:15.460456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460386+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:26:15.460386+0000","last_peered":"2024-04-22T13:26:15.460386+0000","last_clean":"2024-04-22T13:26:15.460386+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:26:15.460386+0000","last_undegraded":"2024-04-22T13:26:15.460386+0000","last_fullsized":"2024-04-22T13:26:15.460386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":90,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:19.329253+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:26:19.329253+0000","last_peered":"2024-04-22T13:26:19.329253+0000","last_clean":"2024-04-22T13:26:19.329253+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:26:19.329253+0000","last_undegraded":"2024-04-22T13:26:19.329253+0000","last_fullsized":"2024-04-22T13:26:19.329253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460610+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:26:15.460610+0000","last_peered":"2024-04-22T13:26:15.460610+0000","last_clean":"2024-04-22T13:26:15.460610+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:26:15.460610+0000","last_undegraded":"2024-04-22T13:26:15.460610+0000","last_fullsized":"2024-04-22T13:26:15.460610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514974+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:26:15.514974+0000","last_peered":"2024-04-22T13:26:15.514974+0000","last_clean":"2024-04-22T13:26:15.514974+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:26:15.514974+0000","last_undegraded":"2024-04-22T13:26:15.514974+0000","last_fullsized":"2024-04-22T13:26:15.514974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460403+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:26:15.460403+0000","last_peered":"2024-04-22T13:26:15.460403+0000","last_clean":"2024-04-22T13:26:15.460403+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:26:15.460403+0000","last_undegraded":"2024-04-22T13:26:15.460403+0000","last_fullsized":"2024-04-22T13:26:15.460403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456251+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:26:15.456251+0000","last_peered":"2024-04-22T13:26:15.456251+0000","last_clean":"2024-04-22T13:26:15.456251+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:26:15.456251+0000","last_undegraded":"2024-04-22T13:26:15.456251+0000","last_fullsized":"2024-04-22T13:26:15.456251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460801+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:26:15.460801+0000","last_peered":"2024-04-22T13:26:15.460801+0000","last_clean":"2024-04-22T13:26:15.460801+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:26:15.460801+0000","last_undegraded":"2024-04-22T13:26:15.460801+0000","last_fullsized":"2024-04-22T13:26:15.460801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456972+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:26:15.456972+0000","last_peered":"2024-04-22T13:26:15.456972+0000","last_clean":"2024-04-22T13:26:15.456972+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:26:15.456972+0000","last_undegraded":"2024-04-22T13:26:15.456972+0000","last_fullsized":"2024-04-22T13:26:15.456972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515905+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:26:15.515905+0000","last_peered":"2024-04-22T13:26:15.515905+0000","last_clean":"2024-04-22T13:26:15.515905+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:26:15.515905+0000","last_undegraded":"2024-04-22T13:26:15.515905+0000","last_fullsized":"2024-04-22T13:26:15.515905+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460203+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:26:15.460203+0000","last_peered":"2024-04-22T13:26:15.460203+0000","last_clean":"2024-04-22T13:26:15.460203+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:26:15.460203+0000","last_undegraded":"2024-04-22T13:26:15.460203+0000","last_fullsized":"2024-04-22T13:26:15.460203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457763+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:26:15.457763+0000","last_peered":"2024-04-22T13:26:15.457763+0000","last_clean":"2024-04-22T13:26:15.457763+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:26:15.457763+0000","last_undegraded":"2024-04-22T13:26:15.457763+0000","last_fullsized":"2024-04-22T13:26:15.457763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459853+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:26:15.459853+0000","last_peered":"2024-04-22T13:26:15.459853+0000","last_clean":"2024-04-22T13:26:15.459853+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:26:15.459853+0000","last_undegraded":"2024-04-22T13:26:15.459853+0000","last_fullsized":"2024-04-22T13:26:15.459853+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457635+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:26:15.457635+0000","last_peered":"2024-04-22T13:26:15.457635+0000","last_clean":"2024-04-22T13:26:15.457635+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:26:15.457635+0000","last_undegraded":"2024-04-22T13:26:15.457635+0000","last_fullsized":"2024-04-22T13:26:15.457635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455654+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:26:15.455654+0000","last_peered":"2024-04-22T13:26:15.455654+0000","last_clean":"2024-04-22T13:26:15.455654+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:26:15.455654+0000","last_undegraded":"2024-04-22T13:26:15.455654+0000","last_fullsized":"2024-04-22T13:26:15.455654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459765+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:26:15.459765+0000","last_peered":"2024-04-22T13:26:15.459765+0000","last_clean":"2024-04-22T13:26:15.459765+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:26:15.459765+0000","last_undegraded":"2024-04-22T13:26:15.459765+0000","last_fullsized":"2024-04-22T13:26:15.459765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.462516+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:26:15.462516+0000","last_peered":"2024-04-22T13:26:15.462516+0000","last_clean":"2024-04-22T13:26:15.462516+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:26:15.462516+0000","last_undegraded":"2024-04-22T13:26:15.462516+0000","last_fullsized":"2024-04-22T13:26:15.462516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459378+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:26:15.459378+0000","last_peered":"2024-04-22T13:26:15.459378+0000","last_clean":"2024-04-22T13:26:15.459378+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:26:15.459378+0000","last_undegraded":"2024-04-22T13:26:15.459378+0000","last_fullsized":"2024-04-22T13:26:15.459378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515020+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:26:15.515020+0000","last_peered":"2024-04-22T13:26:15.515020+0000","last_clean":"2024-04-22T13:26:15.515020+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:26:15.515020+0000","last_undegraded":"2024-04-22T13:26:15.515020+0000","last_fullsized":"2024-04-22T13:26:15.515020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456478+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:26:15.456478+0000","last_peered":"2024-04-22T13:26:15.456478+0000","last_clean":"2024-04-22T13:26:15.456478+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:26:15.456478+0000","last_undegraded":"2024-04-22T13:26:15.456478+0000","last_fullsized":"2024-04-22T13:26:15.456478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457088+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:26:15.457088+0000","last_peered":"2024-04-22T13:26:15.457088+0000","last_clean":"2024-04-22T13:26:15.457088+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:26:15.457088+0000","last_undegraded":"2024-04-22T13:26:15.457088+0000","last_fullsized":"2024-04-22T13:26:15.457088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65654,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":692,"num_read_kb":421,"num_write":234,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252891,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":475500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1695744,"data_stored":1125370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":475430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939692,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":475808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41080,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704072,"statfs":{"total":95995035648,"available":95952969728,"internally_reserved":0,"allocated":1613824,"data_stored":1068326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":475147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":534901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":476486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":535512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40500,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704652,"statfs":{"total":95995035648,"available":95953563648,"internally_reserved":0,"allocated":1019904,"data_stored":474972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":475008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1626112,"data_stored":1068004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:26:26.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T13:26:26.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:26 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:26:26.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:26 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:26:26.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:26 smithi049 ceph-mon[31379]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:26.771 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:26 smithi049 ceph-mon[31379]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:26:26.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:26 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:26:26.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:26 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:26:26.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:26 smithi107 ceph-mon[31859]: from='mgr.14152 172.21.15.5:0/517030268' entity='mgr.x' 2024-04-22T13:26:26.785 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:26 smithi107 ceph-mon[31859]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:26:26.887 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:28.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:27 smithi005 ceph-mon[28006]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:28.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:27 smithi049 ceph-mon[31379]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:28.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:27 smithi107 ceph-mon[31859]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 -- 172.21.15.5:0/42786861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108103480 msgr2=0x7f010810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 --2- 172.21.15.5:0/42786861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108103480 0x7f010810f8d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009990 tx=0x7f00f8009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 -- 172.21.15.5:0/42786861 shutdown_connections 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 --2- 172.21.15.5:0/42786861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108108a70 0x7f0108108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 --2- 172.21.15.5:0/42786861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108103480 0x7f010810f8d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 --2- 172.21.15.5:0/42786861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0108102ac0 0x7f0108102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 -- 172.21.15.5:0/42786861 >> 172.21.15.5:0/42786861 conn(0x7f0108079bb0 msgr2=0x7f0108079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 -- 172.21.15.5:0/42786861 shutdown_connections 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 -- 172.21.15.5:0/42786861 wait complete. 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 Processor -- start 2024-04-22T13:26:28.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.492+0000 7f01101e8700 1 -- start start 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f01101e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 0x7f0108199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f01101e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0108103480 0x7f0108199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f01101e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108108a70 0x7f010819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f01101e8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0108112c00 con 0x7f0108102ac0 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f01101e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0108112a80 con 0x7f0108108a70 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f01101e8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0108112d80 con 0x7f0108103480 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 0x7f0108199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 0x7f0108199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49884/0 (socket says 172.21.15.5:49884) 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 -- 172.21.15.5:0/672472890 learned_addr learned my addr 172.21.15.5:0/672472890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010d783700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0108103480 0x7f0108199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010e785700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108108a70 0x7f010819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 -- 172.21.15.5:0/672472890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0108103480 msgr2=0x7f0108199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0108103480 0x7f0108199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 -- 172.21.15.5:0/672472890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108108a70 msgr2=0x7f010819d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108108a70 0x7f010819d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 -- 172.21.15.5:0/672472890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010819e010 con 0x7f0108102ac0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f010df84700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 0x7f0108199050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0100007e20 tx=0x7f010000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.493+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100016020 con 0x7f0108102ac0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.494+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010000ac90 con 0x7f0108102ac0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.494+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010819e300 con 0x7f0108102ac0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.494+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01000115e0 con 0x7f0108102ac0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.494+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a5b80 con 0x7f0108102ac0 2024-04-22T13:26:28.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.495+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0108069730 con 0x7f0108102ac0 2024-04-22T13:26:28.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.497+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0100019020 con 0x7f0108102ac0 2024-04-22T13:26:28.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.497+0000 7f0106ffd700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00f00a3d50 0x7f00f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:28.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.497+0000 7f010d783700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00f00a3d50 0x7f00f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:28.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.497+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f01000d0420 con 0x7f0108102ac0 2024-04-22T13:26:28.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.498+0000 7f010d783700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00f00a3d50 0x7f00f00a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f010807a1b0 tx=0x7f00fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:28.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.501+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0100020070 con 0x7f0108102ac0 2024-04-22T13:26:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.708+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0108062e80 con 0x7f00f00a3d50 2024-04-22T13:26:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.715+0000 7f0106ffd700 1 -- 172.21.15.5:0/672472890 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7f0108062e80 con 0x7f00f00a3d50 2024-04-22T13:26:28.715 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00f00a3d50 msgr2=0x7f00f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00f00a3d50 0x7f00f00a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f010807a1b0 tx=0x7f00fc00b410 comp rx=0 tx=0).stop 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 msgr2=0x7f0108199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 0x7f0108199050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0100007e20 tx=0x7f010000a340 comp rx=0 tx=0).stop 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 shutdown_connections 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f00f00a3d50 0x7f00f00a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108108a70 0x7f010819d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0108102ac0 0x7f0108199050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 --2- 172.21.15.5:0/672472890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0108103480 0x7f0108199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 >> 172.21.15.5:0/672472890 conn(0x7f0108079bb0 msgr2=0x7f01081016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 shutdown_connections 2024-04-22T13:26:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:28.719+0000 7f01101e8700 1 -- 172.21.15.5:0/672472890 wait complete. 2024-04-22T13:26:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:26:29.385 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-22T13:26:28.207905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656256,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":848,"num_read_kb":550,"num_write":417,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404676,"kb_used_data":13836,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537148,"statfs":{"total":1151940427776,"available":1151526039552,"internally_reserved":0,"allocated":14168064,"data_stored":7660464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003080"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457418+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:26:15.457418+0000","last_peered":"2024-04-22T13:26:15.457418+0000","last_clean":"2024-04-22T13:26:15.457418+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:26:15.457418+0000","last_undegraded":"2024-04-22T13:26:15.457418+0000","last_fullsized":"2024-04-22T13:26:15.457418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457956+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:26:15.457956+0000","last_peered":"2024-04-22T13:26:15.457956+0000","last_clean":"2024-04-22T13:26:15.457956+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:26:15.457956+0000","last_undegraded":"2024-04-22T13:26:15.457956+0000","last_fullsized":"2024-04-22T13:26:15.457956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457942+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:26:15.457942+0000","last_peered":"2024-04-22T13:26:15.457942+0000","last_clean":"2024-04-22T13:26:15.457942+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:26:15.457942+0000","last_undegraded":"2024-04-22T13:26:15.457942+0000","last_fullsized":"2024-04-22T13:26:15.457942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455570+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:26:15.455570+0000","last_peered":"2024-04-22T13:26:15.455570+0000","last_clean":"2024-04-22T13:26:15.455570+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:26:15.455570+0000","last_undegraded":"2024-04-22T13:26:15.455570+0000","last_fullsized":"2024-04-22T13:26:15.455570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460330+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:26:15.460330+0000","last_peered":"2024-04-22T13:26:15.460330+0000","last_clean":"2024-04-22T13:26:15.460330+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:26:15.460330+0000","last_undegraded":"2024-04-22T13:26:15.460330+0000","last_fullsized":"2024-04-22T13:26:15.460330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515185+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:26:15.515185+0000","last_peered":"2024-04-22T13:26:15.515185+0000","last_clean":"2024-04-22T13:26:15.515185+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:26:15.515185+0000","last_undegraded":"2024-04-22T13:26:15.515185+0000","last_fullsized":"2024-04-22T13:26:15.515185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460180+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:26:15.460180+0000","last_peered":"2024-04-22T13:26:15.460180+0000","last_clean":"2024-04-22T13:26:15.460180+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:26:15.460180+0000","last_undegraded":"2024-04-22T13:26:15.460180+0000","last_fullsized":"2024-04-22T13:26:15.460180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459528+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:26:15.459528+0000","last_peered":"2024-04-22T13:26:15.459528+0000","last_clean":"2024-04-22T13:26:15.459528+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:26:15.459528+0000","last_undegraded":"2024-04-22T13:26:15.459528+0000","last_fullsized":"2024-04-22T13:26:15.459528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515484+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:26:15.515484+0000","last_peered":"2024-04-22T13:26:15.515484+0000","last_clean":"2024-04-22T13:26:15.515484+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:26:15.515484+0000","last_undegraded":"2024-04-22T13:26:15.515484+0000","last_fullsized":"2024-04-22T13:26:15.515484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515163+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:26:15.515163+0000","last_peered":"2024-04-22T13:26:15.515163+0000","last_clean":"2024-04-22T13:26:15.515163+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:26:15.515163+0000","last_undegraded":"2024-04-22T13:26:15.515163+0000","last_fullsized":"2024-04-22T13:26:15.515163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460618+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:26:15.460618+0000","last_peered":"2024-04-22T13:26:15.460618+0000","last_clean":"2024-04-22T13:26:15.460618+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:26:15.460618+0000","last_undegraded":"2024-04-22T13:26:15.460618+0000","last_fullsized":"2024-04-22T13:26:15.460618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455531+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:26:15.455531+0000","last_peered":"2024-04-22T13:26:15.455531+0000","last_clean":"2024-04-22T13:26:15.455531+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:26:15.455531+0000","last_undegraded":"2024-04-22T13:26:15.455531+0000","last_fullsized":"2024-04-22T13:26:15.455531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456372+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:26:15.456372+0000","last_peered":"2024-04-22T13:26:15.456372+0000","last_clean":"2024-04-22T13:26:15.456372+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:26:15.456372+0000","last_undegraded":"2024-04-22T13:26:15.456372+0000","last_fullsized":"2024-04-22T13:26:15.456372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457324+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:26:15.457324+0000","last_peered":"2024-04-22T13:26:15.457324+0000","last_clean":"2024-04-22T13:26:15.457324+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:26:15.457324+0000","last_undegraded":"2024-04-22T13:26:15.457324+0000","last_fullsized":"2024-04-22T13:26:15.457324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457305+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:26:15.457305+0000","last_peered":"2024-04-22T13:26:15.457305+0000","last_clean":"2024-04-22T13:26:15.457305+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:26:15.457305+0000","last_undegraded":"2024-04-22T13:26:15.457305+0000","last_fullsized":"2024-04-22T13:26:15.457305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514704+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:26:15.514704+0000","last_peered":"2024-04-22T13:26:15.514704+0000","last_clean":"2024-04-22T13:26:15.514704+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:26:15.514704+0000","last_undegraded":"2024-04-22T13:26:15.514704+0000","last_fullsized":"2024-04-22T13:26:15.514704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457627+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:26:15.457627+0000","last_peered":"2024-04-22T13:26:15.457627+0000","last_clean":"2024-04-22T13:26:15.457627+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:26:15.457627+0000","last_undegraded":"2024-04-22T13:26:15.457627+0000","last_fullsized":"2024-04-22T13:26:15.457627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454145+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:26:15.454145+0000","last_peered":"2024-04-22T13:26:15.454145+0000","last_clean":"2024-04-22T13:26:15.454145+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:26:15.454145+0000","last_undegraded":"2024-04-22T13:26:15.454145+0000","last_fullsized":"2024-04-22T13:26:15.454145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455134+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:26:15.455134+0000","last_peered":"2024-04-22T13:26:15.455134+0000","last_clean":"2024-04-22T13:26:15.455134+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:26:15.455134+0000","last_undegraded":"2024-04-22T13:26:15.455134+0000","last_fullsized":"2024-04-22T13:26:15.455134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:26.337866+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:26:26.337866+0000","last_peered":"2024-04-22T13:26:26.337866+0000","last_clean":"2024-04-22T13:26:26.337866+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:26:26.337866+0000","last_undegraded":"2024-04-22T13:26:26.337866+0000","last_fullsized":"2024-04-22T13:26:26.337866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459882+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:26:15.459882+0000","last_peered":"2024-04-22T13:26:15.459882+0000","last_clean":"2024-04-22T13:26:15.459882+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:26:15.459882+0000","last_undegraded":"2024-04-22T13:26:15.459882+0000","last_fullsized":"2024-04-22T13:26:15.459882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460034+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:26:15.460034+0000","last_peered":"2024-04-22T13:26:15.460034+0000","last_clean":"2024-04-22T13:26:15.460034+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:26:15.460034+0000","last_undegraded":"2024-04-22T13:26:15.460034+0000","last_fullsized":"2024-04-22T13:26:15.460034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457537+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:26:15.457537+0000","last_peered":"2024-04-22T13:26:15.457537+0000","last_clean":"2024-04-22T13:26:15.457537+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:26:15.457537+0000","last_undegraded":"2024-04-22T13:26:15.457537+0000","last_fullsized":"2024-04-22T13:26:15.457537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.473813+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:26:15.473813+0000","last_peered":"2024-04-22T13:26:15.473813+0000","last_clean":"2024-04-22T13:26:15.473813+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:26:15.473813+0000","last_undegraded":"2024-04-22T13:26:15.473813+0000","last_fullsized":"2024-04-22T13:26:15.473813+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514831+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:26:15.514831+0000","last_peered":"2024-04-22T13:26:15.514831+0000","last_clean":"2024-04-22T13:26:15.514831+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:26:15.514831+0000","last_undegraded":"2024-04-22T13:26:15.514831+0000","last_fullsized":"2024-04-22T13:26:15.514831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460781+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:26:15.460781+0000","last_peered":"2024-04-22T13:26:15.460781+0000","last_clean":"2024-04-22T13:26:15.460781+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:26:15.460781+0000","last_undegraded":"2024-04-22T13:26:15.460781+0000","last_fullsized":"2024-04-22T13:26:15.460781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514984+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:26:15.514984+0000","last_peered":"2024-04-22T13:26:15.514984+0000","last_clean":"2024-04-22T13:26:15.514984+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:26:15.514984+0000","last_undegraded":"2024-04-22T13:26:15.514984+0000","last_fullsized":"2024-04-22T13:26:15.514984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.333757+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:26:22.333757+0000","last_peered":"2024-04-22T13:26:22.333757+0000","last_clean":"2024-04-22T13:26:22.333757+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:26:22.333757+0000","last_undegraded":"2024-04-22T13:26:22.333757+0000","last_fullsized":"2024-04-22T13:26:22.333757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461017+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:26:15.461017+0000","last_peered":"2024-04-22T13:26:15.461017+0000","last_clean":"2024-04-22T13:26:15.461017+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:26:15.461017+0000","last_undegraded":"2024-04-22T13:26:15.461017+0000","last_fullsized":"2024-04-22T13:26:15.461017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515311+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:26:15.515311+0000","last_peered":"2024-04-22T13:26:15.515311+0000","last_clean":"2024-04-22T13:26:15.515311+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:26:15.515311+0000","last_undegraded":"2024-04-22T13:26:15.515311+0000","last_fullsized":"2024-04-22T13:26:15.515311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514710+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:26:15.514710+0000","last_peered":"2024-04-22T13:26:15.514710+0000","last_clean":"2024-04-22T13:26:15.514710+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:26:15.514710+0000","last_undegraded":"2024-04-22T13:26:15.514710+0000","last_fullsized":"2024-04-22T13:26:15.514710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455542+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:26:15.455542+0000","last_peered":"2024-04-22T13:26:15.455542+0000","last_clean":"2024-04-22T13:26:15.455542+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:26:15.455542+0000","last_undegraded":"2024-04-22T13:26:15.455542+0000","last_fullsized":"2024-04-22T13:26:15.455542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459629+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:26:15.459629+0000","last_peered":"2024-04-22T13:26:15.459629+0000","last_clean":"2024-04-22T13:26:15.459629+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:26:15.459629+0000","last_undegraded":"2024-04-22T13:26:15.459629+0000","last_fullsized":"2024-04-22T13:26:15.459629+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514508+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:26:15.514508+0000","last_peered":"2024-04-22T13:26:15.514508+0000","last_clean":"2024-04-22T13:26:15.514508+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:26:15.514508+0000","last_undegraded":"2024-04-22T13:26:15.514508+0000","last_fullsized":"2024-04-22T13:26:15.514508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.453875+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:26:15.453875+0000","last_peered":"2024-04-22T13:26:15.453875+0000","last_clean":"2024-04-22T13:26:15.453875+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:26:15.453875+0000","last_undegraded":"2024-04-22T13:26:15.453875+0000","last_fullsized":"2024-04-22T13:26:15.453875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.333152+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:26:22.333152+0000","last_peered":"2024-04-22T13:26:22.333152+0000","last_clean":"2024-04-22T13:26:22.333152+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:26:22.333152+0000","last_undegraded":"2024-04-22T13:26:22.333152+0000","last_fullsized":"2024-04-22T13:26:22.333152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515596+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:26:15.515596+0000","last_peered":"2024-04-22T13:26:15.515596+0000","last_clean":"2024-04-22T13:26:15.515596+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:26:15.515596+0000","last_undegraded":"2024-04-22T13:26:15.515596+0000","last_fullsized":"2024-04-22T13:26:15.515596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460664+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:26:15.460664+0000","last_peered":"2024-04-22T13:26:15.460664+0000","last_clean":"2024-04-22T13:26:15.460664+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:26:15.460664+0000","last_undegraded":"2024-04-22T13:26:15.460664+0000","last_fullsized":"2024-04-22T13:26:15.460664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.516279+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:26:15.516279+0000","last_peered":"2024-04-22T13:26:15.516279+0000","last_clean":"2024-04-22T13:26:15.516279+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:26:15.516279+0000","last_undegraded":"2024-04-22T13:26:15.516279+0000","last_fullsized":"2024-04-22T13:26:15.516279+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515193+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:26:15.515193+0000","last_peered":"2024-04-22T13:26:15.515193+0000","last_clean":"2024-04-22T13:26:15.515193+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:26:15.515193+0000","last_undegraded":"2024-04-22T13:26:15.515193+0000","last_fullsized":"2024-04-22T13:26:15.515193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515050+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:26:15.515050+0000","last_peered":"2024-04-22T13:26:15.515050+0000","last_clean":"2024-04-22T13:26:15.515050+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:26:15.515050+0000","last_undegraded":"2024-04-22T13:26:15.515050+0000","last_fullsized":"2024-04-22T13:26:15.515050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'28","reported_seq":185,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.340140+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:26:22.340140+0000","last_peered":"2024-04-22T13:26:22.340140+0000","last_clean":"2024-04-22T13:26:22.340140+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:26:22.340140+0000","last_undegraded":"2024-04-22T13:26:22.340140+0000","last_fullsized":"2024-04-22T13:26:22.340140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58841,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":127,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461477+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:26:15.461477+0000","last_peered":"2024-04-22T13:26:15.461477+0000","last_clean":"2024-04-22T13:26:15.461477+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:26:15.461477+0000","last_undegraded":"2024-04-22T13:26:15.461477+0000","last_fullsized":"2024-04-22T13:26:15.461477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454737+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:26:15.454737+0000","last_peered":"2024-04-22T13:26:15.454737+0000","last_clean":"2024-04-22T13:26:15.454737+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:26:15.454737+0000","last_undegraded":"2024-04-22T13:26:15.454737+0000","last_fullsized":"2024-04-22T13:26:15.454737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455592+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:26:15.455592+0000","last_peered":"2024-04-22T13:26:15.455592+0000","last_clean":"2024-04-22T13:26:15.455592+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:26:15.455592+0000","last_undegraded":"2024-04-22T13:26:15.455592+0000","last_fullsized":"2024-04-22T13:26:15.455592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514494+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:26:15.514494+0000","last_peered":"2024-04-22T13:26:15.514494+0000","last_clean":"2024-04-22T13:26:15.514494+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:26:15.514494+0000","last_undegraded":"2024-04-22T13:26:15.514494+0000","last_fullsized":"2024-04-22T13:26:15.514494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460279+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:26:15.460279+0000","last_peered":"2024-04-22T13:26:15.460279+0000","last_clean":"2024-04-22T13:26:15.460279+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:26:15.460279+0000","last_undegraded":"2024-04-22T13:26:15.460279+0000","last_fullsized":"2024-04-22T13:26:15.460279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":93,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.339850+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:26:22.339850+0000","last_peered":"2024-04-22T13:26:22.339850+0000","last_clean":"2024-04-22T13:26:22.339850+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:26:22.339850+0000","last_undegraded":"2024-04-22T13:26:22.339850+0000","last_fullsized":"2024-04-22T13:26:22.339850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459808+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:26:15.459808+0000","last_peered":"2024-04-22T13:26:15.459808+0000","last_clean":"2024-04-22T13:26:15.459808+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:26:15.459808+0000","last_undegraded":"2024-04-22T13:26:15.459808+0000","last_fullsized":"2024-04-22T13:26:15.459808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460239+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:26:15.460239+0000","last_peered":"2024-04-22T13:26:15.460239+0000","last_clean":"2024-04-22T13:26:15.460239+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:26:15.460239+0000","last_undegraded":"2024-04-22T13:26:15.460239+0000","last_fullsized":"2024-04-22T13:26:15.460239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514680+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:26:15.514680+0000","last_peered":"2024-04-22T13:26:15.514680+0000","last_clean":"2024-04-22T13:26:15.514680+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:26:15.514680+0000","last_undegraded":"2024-04-22T13:26:15.514680+0000","last_fullsized":"2024-04-22T13:26:15.514680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461355+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:26:15.461355+0000","last_peered":"2024-04-22T13:26:15.461355+0000","last_clean":"2024-04-22T13:26:15.461355+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:26:15.461355+0000","last_undegraded":"2024-04-22T13:26:15.461355+0000","last_fullsized":"2024-04-22T13:26:15.461355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457316+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:26:15.457316+0000","last_peered":"2024-04-22T13:26:15.457316+0000","last_clean":"2024-04-22T13:26:15.457316+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:26:15.457316+0000","last_undegraded":"2024-04-22T13:26:15.457316+0000","last_fullsized":"2024-04-22T13:26:15.457316+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454072+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:26:15.454072+0000","last_peered":"2024-04-22T13:26:15.454072+0000","last_clean":"2024-04-22T13:26:15.454072+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:26:15.454072+0000","last_undegraded":"2024-04-22T13:26:15.454072+0000","last_fullsized":"2024-04-22T13:26:15.454072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515481+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:26:15.515481+0000","last_peered":"2024-04-22T13:26:15.515481+0000","last_clean":"2024-04-22T13:26:15.515481+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:26:15.515481+0000","last_undegraded":"2024-04-22T13:26:15.515481+0000","last_fullsized":"2024-04-22T13:26:15.515481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:22.333082+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:26:22.333082+0000","last_peered":"2024-04-22T13:26:22.333082+0000","last_clean":"2024-04-22T13:26:22.333082+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:26:22.333082+0000","last_undegraded":"2024-04-22T13:26:22.333082+0000","last_fullsized":"2024-04-22T13:26:22.333082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515533+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:26:15.515533+0000","last_peered":"2024-04-22T13:26:15.515533+0000","last_clean":"2024-04-22T13:26:15.515533+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:26:15.515533+0000","last_undegraded":"2024-04-22T13:26:15.515533+0000","last_fullsized":"2024-04-22T13:26:15.515533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457644+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:26:15.457644+0000","last_peered":"2024-04-22T13:26:15.457644+0000","last_clean":"2024-04-22T13:26:15.457644+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:26:15.457644+0000","last_undegraded":"2024-04-22T13:26:15.457644+0000","last_fullsized":"2024-04-22T13:26:15.457644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":96,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:26.344235+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:26:26.344235+0000","last_peered":"2024-04-22T13:26:26.344235+0000","last_clean":"2024-04-22T13:26:26.344235+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:26:26.344235+0000","last_undegraded":"2024-04-22T13:26:26.344235+0000","last_fullsized":"2024-04-22T13:26:26.344235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460605+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:26:15.460605+0000","last_peered":"2024-04-22T13:26:15.460605+0000","last_clean":"2024-04-22T13:26:15.460605+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:26:15.460605+0000","last_undegraded":"2024-04-22T13:26:15.460605+0000","last_fullsized":"2024-04-22T13:26:15.460605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455295+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:26:15.455295+0000","last_peered":"2024-04-22T13:26:15.455295+0000","last_clean":"2024-04-22T13:26:15.455295+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:26:15.455295+0000","last_undegraded":"2024-04-22T13:26:15.455295+0000","last_fullsized":"2024-04-22T13:26:15.455295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459901+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:26:15.459901+0000","last_peered":"2024-04-22T13:26:15.459901+0000","last_clean":"2024-04-22T13:26:15.459901+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:26:15.459901+0000","last_undegraded":"2024-04-22T13:26:15.459901+0000","last_fullsized":"2024-04-22T13:26:15.459901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455425+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:26:15.455425+0000","last_peered":"2024-04-22T13:26:15.455425+0000","last_clean":"2024-04-22T13:26:15.455425+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:26:15.455425+0000","last_undegraded":"2024-04-22T13:26:15.455425+0000","last_fullsized":"2024-04-22T13:26:15.455425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460045+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:26:15.460045+0000","last_peered":"2024-04-22T13:26:15.460045+0000","last_clean":"2024-04-22T13:26:15.460045+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:26:15.460045+0000","last_undegraded":"2024-04-22T13:26:15.460045+0000","last_fullsized":"2024-04-22T13:26:15.460045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.461596+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:26:15.461596+0000","last_peered":"2024-04-22T13:26:15.461596+0000","last_clean":"2024-04-22T13:26:15.461596+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:26:15.461596+0000","last_undegraded":"2024-04-22T13:26:15.461596+0000","last_fullsized":"2024-04-22T13:26:15.461596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456351+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:26:15.456351+0000","last_peered":"2024-04-22T13:26:15.456351+0000","last_clean":"2024-04-22T13:26:15.456351+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:26:15.456351+0000","last_undegraded":"2024-04-22T13:26:15.456351+0000","last_fullsized":"2024-04-22T13:26:15.456351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.454186+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:26:15.454186+0000","last_peered":"2024-04-22T13:26:15.454186+0000","last_clean":"2024-04-22T13:26:15.454186+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:26:15.454186+0000","last_undegraded":"2024-04-22T13:26:15.454186+0000","last_fullsized":"2024-04-22T13:26:15.454186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459867+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:26:15.459867+0000","last_peered":"2024-04-22T13:26:15.459867+0000","last_clean":"2024-04-22T13:26:15.459867+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:26:15.459867+0000","last_undegraded":"2024-04-22T13:26:15.459867+0000","last_fullsized":"2024-04-22T13:26:15.459867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455515+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:26:15.455515+0000","last_peered":"2024-04-22T13:26:15.455515+0000","last_clean":"2024-04-22T13:26:15.455515+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:26:15.455515+0000","last_undegraded":"2024-04-22T13:26:15.455515+0000","last_fullsized":"2024-04-22T13:26:15.455515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.458058+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:26:15.458058+0000","last_peered":"2024-04-22T13:26:15.458058+0000","last_clean":"2024-04-22T13:26:15.458058+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:26:15.458058+0000","last_undegraded":"2024-04-22T13:26:15.458058+0000","last_fullsized":"2024-04-22T13:26:15.458058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514866+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:26:15.514866+0000","last_peered":"2024-04-22T13:26:15.514866+0000","last_clean":"2024-04-22T13:26:15.514866+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:26:15.514866+0000","last_undegraded":"2024-04-22T13:26:15.514866+0000","last_fullsized":"2024-04-22T13:26:15.514866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460401+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:26:15.460401+0000","last_peered":"2024-04-22T13:26:15.460401+0000","last_clean":"2024-04-22T13:26:15.460401+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:26:15.460401+0000","last_undegraded":"2024-04-22T13:26:15.460401+0000","last_fullsized":"2024-04-22T13:26:15.460401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459496+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:26:15.459496+0000","last_peered":"2024-04-22T13:26:15.459496+0000","last_clean":"2024-04-22T13:26:15.459496+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:26:15.459496+0000","last_undegraded":"2024-04-22T13:26:15.459496+0000","last_fullsized":"2024-04-22T13:26:15.459496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456857+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:26:15.456857+0000","last_peered":"2024-04-22T13:26:15.456857+0000","last_clean":"2024-04-22T13:26:15.456857+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:26:15.456857+0000","last_undegraded":"2024-04-22T13:26:15.456857+0000","last_fullsized":"2024-04-22T13:26:15.456857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.477566+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:26:15.477566+0000","last_peered":"2024-04-22T13:26:15.477566+0000","last_clean":"2024-04-22T13:26:15.477566+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:26:15.477566+0000","last_undegraded":"2024-04-22T13:26:15.477566+0000","last_fullsized":"2024-04-22T13:26:15.477566+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459520+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:26:15.459520+0000","last_peered":"2024-04-22T13:26:15.459520+0000","last_clean":"2024-04-22T13:26:15.459520+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:26:15.459520+0000","last_undegraded":"2024-04-22T13:26:15.459520+0000","last_fullsized":"2024-04-22T13:26:15.459520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460456+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:26:15.460456+0000","last_peered":"2024-04-22T13:26:15.460456+0000","last_clean":"2024-04-22T13:26:15.460456+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:26:15.460456+0000","last_undegraded":"2024-04-22T13:26:15.460456+0000","last_fullsized":"2024-04-22T13:26:15.460456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460386+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:26:15.460386+0000","last_peered":"2024-04-22T13:26:15.460386+0000","last_clean":"2024-04-22T13:26:15.460386+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:26:15.460386+0000","last_undegraded":"2024-04-22T13:26:15.460386+0000","last_fullsized":"2024-04-22T13:26:15.460386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":95,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:24.336399+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:26:24.336399+0000","last_peered":"2024-04-22T13:26:24.336399+0000","last_clean":"2024-04-22T13:26:24.336399+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:26:24.336399+0000","last_undegraded":"2024-04-22T13:26:24.336399+0000","last_fullsized":"2024-04-22T13:26:24.336399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460610+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:26:15.460610+0000","last_peered":"2024-04-22T13:26:15.460610+0000","last_clean":"2024-04-22T13:26:15.460610+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:26:15.460610+0000","last_undegraded":"2024-04-22T13:26:15.460610+0000","last_fullsized":"2024-04-22T13:26:15.460610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.514974+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:26:15.514974+0000","last_peered":"2024-04-22T13:26:15.514974+0000","last_clean":"2024-04-22T13:26:15.514974+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:26:15.514974+0000","last_undegraded":"2024-04-22T13:26:15.514974+0000","last_fullsized":"2024-04-22T13:26:15.514974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460403+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:26:15.460403+0000","last_peered":"2024-04-22T13:26:15.460403+0000","last_clean":"2024-04-22T13:26:15.460403+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:26:15.460403+0000","last_undegraded":"2024-04-22T13:26:15.460403+0000","last_fullsized":"2024-04-22T13:26:15.460403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456251+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:26:15.456251+0000","last_peered":"2024-04-22T13:26:15.456251+0000","last_clean":"2024-04-22T13:26:15.456251+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:26:15.456251+0000","last_undegraded":"2024-04-22T13:26:15.456251+0000","last_fullsized":"2024-04-22T13:26:15.456251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460801+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:26:15.460801+0000","last_peered":"2024-04-22T13:26:15.460801+0000","last_clean":"2024-04-22T13:26:15.460801+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:26:15.460801+0000","last_undegraded":"2024-04-22T13:26:15.460801+0000","last_fullsized":"2024-04-22T13:26:15.460801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456972+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:26:15.456972+0000","last_peered":"2024-04-22T13:26:15.456972+0000","last_clean":"2024-04-22T13:26:15.456972+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:26:15.456972+0000","last_undegraded":"2024-04-22T13:26:15.456972+0000","last_fullsized":"2024-04-22T13:26:15.456972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515905+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:26:15.515905+0000","last_peered":"2024-04-22T13:26:15.515905+0000","last_clean":"2024-04-22T13:26:15.515905+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:26:15.515905+0000","last_undegraded":"2024-04-22T13:26:15.515905+0000","last_fullsized":"2024-04-22T13:26:15.515905+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.460203+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:26:15.460203+0000","last_peered":"2024-04-22T13:26:15.460203+0000","last_clean":"2024-04-22T13:26:15.460203+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:26:15.460203+0000","last_undegraded":"2024-04-22T13:26:15.460203+0000","last_fullsized":"2024-04-22T13:26:15.460203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457763+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:26:15.457763+0000","last_peered":"2024-04-22T13:26:15.457763+0000","last_clean":"2024-04-22T13:26:15.457763+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:26:15.457763+0000","last_undegraded":"2024-04-22T13:26:15.457763+0000","last_fullsized":"2024-04-22T13:26:15.457763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459853+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:26:15.459853+0000","last_peered":"2024-04-22T13:26:15.459853+0000","last_clean":"2024-04-22T13:26:15.459853+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:26:15.459853+0000","last_undegraded":"2024-04-22T13:26:15.459853+0000","last_fullsized":"2024-04-22T13:26:15.459853+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457635+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:26:15.457635+0000","last_peered":"2024-04-22T13:26:15.457635+0000","last_clean":"2024-04-22T13:26:15.457635+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:26:15.457635+0000","last_undegraded":"2024-04-22T13:26:15.457635+0000","last_fullsized":"2024-04-22T13:26:15.457635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.455654+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:26:15.455654+0000","last_peered":"2024-04-22T13:26:15.455654+0000","last_clean":"2024-04-22T13:26:15.455654+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:26:15.455654+0000","last_undegraded":"2024-04-22T13:26:15.455654+0000","last_fullsized":"2024-04-22T13:26:15.455654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459765+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:26:15.459765+0000","last_peered":"2024-04-22T13:26:15.459765+0000","last_clean":"2024-04-22T13:26:15.459765+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:26:15.459765+0000","last_undegraded":"2024-04-22T13:26:15.459765+0000","last_fullsized":"2024-04-22T13:26:15.459765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.462516+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:26:15.462516+0000","last_peered":"2024-04-22T13:26:15.462516+0000","last_clean":"2024-04-22T13:26:15.462516+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:26:15.462516+0000","last_undegraded":"2024-04-22T13:26:15.462516+0000","last_fullsized":"2024-04-22T13:26:15.462516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.459378+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:26:15.459378+0000","last_peered":"2024-04-22T13:26:15.459378+0000","last_clean":"2024-04-22T13:26:15.459378+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:26:15.459378+0000","last_undegraded":"2024-04-22T13:26:15.459378+0000","last_fullsized":"2024-04-22T13:26:15.459378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.515020+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:26:15.515020+0000","last_peered":"2024-04-22T13:26:15.515020+0000","last_clean":"2024-04-22T13:26:15.515020+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:26:15.515020+0000","last_undegraded":"2024-04-22T13:26:15.515020+0000","last_fullsized":"2024-04-22T13:26:15.515020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.456478+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:26:15.456478+0000","last_peered":"2024-04-22T13:26:15.456478+0000","last_clean":"2024-04-22T13:26:15.456478+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:26:15.456478+0000","last_undegraded":"2024-04-22T13:26:15.456478+0000","last_fullsized":"2024-04-22T13:26:15.456478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-22T13:26:15.457088+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:26:15.457088+0000","last_peered":"2024-04-22T13:26:15.457088+0000","last_clean":"2024-04-22T13:26:15.457088+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:26:15.457088+0000","last_undegraded":"2024-04-22T13:26:15.457088+0000","last_fullsized":"2024-04-22T13:26:15.457088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65654,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":712,"num_read_kb":431,"num_write":234,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252891,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":475500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1695744,"data_stored":1125370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":475430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939692,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":475808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41080,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704072,"statfs":{"total":95995035648,"available":95952969728,"internally_reserved":0,"allocated":1613824,"data_stored":1068326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":475147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":534901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":476486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":535512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40500,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704652,"statfs":{"total":95995035648,"available":95953563648,"internally_reserved":0,"allocated":1019904,"data_stored":474972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":475008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1626112,"data_stored":1068004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:26:29.389 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-22T13:26:29.389 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T13:26:29.389 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-22T13:26:29.389 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-22T13:26:29.637 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:30.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:29 smithi005 ceph-mon[28006]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:30.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:29 smithi005 ceph-mon[28006]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:26:30.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:29 smithi049 ceph-mon[31379]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:30.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:29 smithi049 ceph-mon[31379]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:26:30.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:29 smithi107 ceph-mon[31859]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:30.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:29 smithi107 ceph-mon[31859]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 -- 172.21.15.5:0/934071062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006b1f0 msgr2=0x7efe4006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 --2- 172.21.15.5:0/934071062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006b1f0 0x7efe4006b670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efe30009990 tx=0x7efe30009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 -- 172.21.15.5:0/934071062 shutdown_connections 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 --2- 172.21.15.5:0/934071062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe40108a60 0x7efe40108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 --2- 172.21.15.5:0/934071062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe4006bbb0 0x7efe4010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 --2- 172.21.15.5:0/934071062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006b1f0 0x7efe4006b670 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.372+0000 7efe47e86700 1 -- 172.21.15.5:0/934071062 >> 172.21.15.5:0/934071062 conn(0x7efe400789e0 msgr2=0x7efe40078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 -- 172.21.15.5:0/934071062 shutdown_connections 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 -- 172.21.15.5:0/934071062 wait complete. 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 Processor -- start 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 -- start start 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4006b1f0 0x7efe40199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 0x7efe401996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.373+0000 7efe47e86700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe40108a60 0x7efe4019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe47e86700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe40112ca0 con 0x7efe4006bbb0 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe47e86700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efe40112b20 con 0x7efe4006b1f0 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe47e86700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe40112e20 con 0x7efe40108a60 2024-04-22T13:26:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 0x7efe401996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 0x7efe401996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55732/0 (socket says 172.21.15.5:55732) 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 -- 172.21.15.5:0/953285845 learned_addr learned my addr 172.21.15.5:0/953285845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45c22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4006b1f0 0x7efe40199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe46423700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe40108a60 0x7efe4019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 -- 172.21.15.5:0/953285845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe40108a60 msgr2=0x7efe4019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe40108a60 0x7efe4019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 -- 172.21.15.5:0/953285845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4006b1f0 msgr2=0x7efe40199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4006b1f0 0x7efe40199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 -- 172.21.15.5:0/953285845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4019e1a0 con 0x7efe4006bbb0 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45c22700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4006b1f0 0x7efe40199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe45421700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 0x7efe401996d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe30009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30027070 con 0x7efe4006bbb0 2024-04-22T13:26:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe30022470 con 0x7efe4006bbb0 2024-04-22T13:26:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4019e430 con 0x7efe4006bbb0 2024-04-22T13:26:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30017650 con 0x7efe4006bbb0 2024-04-22T13:26:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.374+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe401018e0 con 0x7efe4006bbb0 2024-04-22T13:26:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.375+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe401041a0 con 0x7efe4006bbb0 2024-04-22T13:26:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.376+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7efe3002a070 con 0x7efe4006bbb0 2024-04-22T13:26:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.377+0000 7efe36ffd700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe2c0a3cd0 0x7efe2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.377+0000 7efe45c22700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe2c0a3cd0 0x7efe2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.377+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7efe300d12e0 con 0x7efe4006bbb0 2024-04-22T13:26:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.377+0000 7efe45c22700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe2c0a3cd0 0x7efe2c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009fd0 tx=0x7efe3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.382+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe30013070 con 0x7efe4006bbb0 2024-04-22T13:26:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.640+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efe4019a4f0 con 0x7efe4006bbb0 2024-04-22T13:26:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.640+0000 7efe36ffd700 1 -- 172.21.15.5:0/953285845 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efe3009dc00 con 0x7efe4006bbb0 2024-04-22T13:26:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:31.637 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe2c0a3cd0 msgr2=0x7efe2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe2c0a3cd0 0x7efe2c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009fd0 tx=0x7efe3c009260 comp rx=0 tx=0).stop 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 msgr2=0x7efe401996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 0x7efe401996d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe30009580 comp rx=0 tx=0).stop 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 shutdown_connections 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7efe2c0a3cd0 0x7efe2c0a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4006b1f0 0x7efe40199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe40108a60 0x7efe4019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 --2- 172.21.15.5:0/953285845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4006bbb0 0x7efe401996d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.642+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 >> 172.21.15.5:0/953285845 conn(0x7efe400789e0 msgr2=0x7efe400ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.643+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 shutdown_connections 2024-04-22T13:26:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:31.643+0000 7efe47e86700 1 -- 172.21.15.5:0/953285845 wait complete. 2024-04-22T13:26:32.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:31 smithi005 ceph-mon[28006]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:32.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:31 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/953285845' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T13:26:32.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:31 smithi049 ceph-mon[31379]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:32.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:31 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/953285845' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T13:26:32.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:31 smithi107 ceph-mon[31859]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T13:26:32.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:31 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/953285845' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T13:26:32.290 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-22T13:26:32.290 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:26:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 -- 172.21.15.5:0/2218944225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 msgr2=0x7fba40102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 --2- 172.21.15.5:0/2218944225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40102470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba30009900 tx=0x7fba3002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 -- 172.21.15.5:0/2218944225 shutdown_connections 2024-04-22T13:26:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 --2- 172.21.15.5:0/2218944225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40102470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 --2- 172.21.15.5:0/2218944225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba401029b0 0x7fba401092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 --2- 172.21.15.5:0/2218944225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba40100ea0 0x7fba401012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 -- 172.21.15.5:0/2218944225 >> 172.21.15.5:0/2218944225 conn(0x7fba400fc410 msgr2=0x7fba400fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 -- 172.21.15.5:0/2218944225 shutdown_connections 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.399+0000 7fba4774b640 1 -- 172.21.15.5:0/2218944225 wait complete. 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 Processor -- start 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 -- start start 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba40100ea0 0x7fba40104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba401029b0 0x7fba401039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010c9f0 con 0x7fba40101ff0 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010c870 con 0x7fba401029b0 2024-04-22T13:26:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba4774b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010cb70 con 0x7fba40100ea0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40103460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55756/0 (socket says 172.21.15.5:55756) 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 -- 172.21.15.5:0/1663647096 learned_addr learned my addr 172.21.15.5:0/1663647096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba454e7640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba40100ea0 0x7fba40104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba45ce8640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba401029b0 0x7fba401039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 -- 172.21.15.5:0/1663647096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba40100ea0 msgr2=0x7fba40104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba40100ea0 0x7fba40104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 -- 172.21.15.5:0/1663647096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba401029b0 msgr2=0x7fba401039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba401029b0 0x7fba401039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.400+0000 7fba44ce6640 1 -- 172.21.15.5:0/1663647096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba40104230 con 0x7fba40101ff0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.401+0000 7fba44ce6640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40103460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fba30000c00 tx=0x7fba30004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.401+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30043020 con 0x7fba40101ff0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.401+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba30004440 con 0x7fba40101ff0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.401+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba401a22a0 con 0x7fba40101ff0 2024-04-22T13:26:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.401+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30003c90 con 0x7fba40101ff0 2024-04-22T13:26:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.401+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba401a27f0 con 0x7fba40101ff0 2024-04-22T13:26:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.402+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba401a29f0 con 0x7fba40101ff0 2024-04-22T13:26:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.403+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fba30046020 con 0x7fba40101ff0 2024-04-22T13:26:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.403+0000 7fba367fc640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fba2c0a3c40 0x7fba2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.404+0000 7fba454e7640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fba2c0a3c40 0x7fba2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.404+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fba300f2a60 con 0x7fba40101ff0 2024-04-22T13:26:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.404+0000 7fba454e7640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fba2c0a3c40 0x7fba2c0a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009bd0 tx=0x7fba3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.407+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba3004d020 con 0x7fba40101ff0 2024-04-22T13:26:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.595+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba401a3010 con 0x7fba40101ff0 2024-04-22T13:26:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.597+0000 7fba367fc640 1 -- 172.21.15.5:0/1663647096 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22549 (secure 0 0 0) 0x7fba300bf400 con 0x7fba40101ff0 2024-04-22T13:26:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:32.593 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:26:15.450938+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fba2c0a3c40 msgr2=0x7fba2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fba2c0a3c40 0x7fba2c0a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009bd0 tx=0x7fba3c005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 msgr2=0x7fba40103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40103460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fba30000c00 tx=0x7fba30004040 comp rx=0 tx=0).stop 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 shutdown_connections 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fba2c0a3c40 0x7fba2c0a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba40101ff0 0x7fba40103460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba401029b0 0x7fba401039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 --2- 172.21.15.5:0/1663647096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba40100ea0 0x7fba40104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 >> 172.21.15.5:0/1663647096 conn(0x7fba400fc410 msgr2=0x7fba400fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 shutdown_connections 2024-04-22T13:26:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.599+0000 7fba4774b640 1 -- 172.21.15.5:0/1663647096 wait complete. 2024-04-22T13:26:32.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:26:32.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 -- 172.21.15.5:0/3427197606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c103190 msgr2=0x7f8d7c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 --2- 172.21.15.5:0/3427197606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c103190 0x7f8d7c103610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009900 tx=0x7f8d7002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 -- 172.21.15.5:0/3427197606 shutdown_connections 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 --2- 172.21.15.5:0/3427197606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 --2- 172.21.15.5:0/3427197606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c103190 0x7f8d7c103610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 --2- 172.21.15.5:0/3427197606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d7c101fa0 0x7f8d7c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 -- 172.21.15.5:0/3427197606 >> 172.21.15.5:0/3427197606 conn(0x7f8d7c0fd510 msgr2=0x7f8d7c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 -- 172.21.15.5:0/3427197606 shutdown_connections 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.715+0000 7f8d8205c640 1 -- 172.21.15.5:0/3427197606 wait complete. 2024-04-22T13:26:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 Processor -- start 2024-04-22T13:26:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 -- start start 2024-04-22T13:26:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c101fa0 0x7f8d7c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d7c103190 0x7f8d7c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c10dae0 con 0x7f8d7c103b50 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c10d960 con 0x7f8d7c101fa0 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d8205c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c10dc60 con 0x7f8d7c103190 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d7b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c101fa0 0x7f8d7c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55782/0 (socket says 172.21.15.5:55782) 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.716+0000 7f8d7bfff640 1 -- 172.21.15.5:0/2090783013 learned_addr learned my addr 172.21.15.5:0/2090783013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7affd640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d7c103190 0x7f8d7c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7bfff640 1 -- 172.21.15.5:0/2090783013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d7c103190 msgr2=0x7f8d7c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7bfff640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d7c103190 0x7f8d7c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7bfff640 1 -- 172.21.15.5:0/2090783013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c101fa0 msgr2=0x7f8d7c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7bfff640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c101fa0 0x7f8d7c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7bfff640 1 -- 172.21.15.5:0/2090783013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c19fdf0 con 0x7f8d7c103b50 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7b7fe640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c101fa0 0x7f8d7c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d7bfff640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c19f780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400eb10 tx=0x7f8d7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74004b20 con 0x7f8d7c103b50 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c1a00e0 con 0x7f8d7c103b50 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d74004cc0 con 0x7f8d7c103b50 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74010670 con 0x7f8d7c103b50 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.717+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1a05e0 con 0x7f8d7c103b50 2024-04-22T13:26:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.718+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c109390 con 0x7f8d7c103b50 2024-04-22T13:26:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.719+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f8d74005020 con 0x7f8d7c103b50 2024-04-22T13:26:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.720+0000 7f8d78ff9640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8d640a3c40 0x7f8d640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.720+0000 7f8d7b7fe640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8d640a3c40 0x7f8d640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.720+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f8d740d0c40 con 0x7f8d7c103b50 2024-04-22T13:26:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.720+0000 7f8d7b7fe640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8d640a3c40 0x7f8d640a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c102ff0 tx=0x7f8d6c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:32.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.724+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d7409d470 con 0x7f8d7c103b50 2024-04-22T13:26:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.910+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d7c062b50 con 0x7f8d7c103b50 2024-04-22T13:26:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.910+0000 7f8d78ff9640 1 -- 172.21.15.5:0/2090783013 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f8d7409cd70 con 0x7f8d7c103b50 2024-04-22T13:26:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:26:32.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8d640a3c40 msgr2=0x7f8d640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8d640a3c40 0x7f8d640a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c102ff0 tx=0x7f8d6c00b390 comp rx=0 tx=0).stop 2024-04-22T13:26:32.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 msgr2=0x7f8d7c19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c19f780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400eb10 tx=0x7f8d7400c2d0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 shutdown_connections 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8d640a3c40 0x7f8d640a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c103b50 0x7f8d7c19f780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d7c103190 0x7f8d7c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 --2- 172.21.15.5:0/2090783013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d7c101fa0 0x7f8d7c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 >> 172.21.15.5:0/2090783013 conn(0x7f8d7c0fd510 msgr2=0x7f8d7c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 shutdown_connections 2024-04-22T13:26:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:32.912+0000 7f8d8205c640 1 -- 172.21.15.5:0/2090783013 wait complete. 2024-04-22T13:26:32.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 -- 172.21.15.5:0/2874407208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b74101fb0 msgr2=0x7f9b741023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/2874407208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b74101fb0 0x7f9b741023d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009900 tx=0x7f9b7002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 -- 172.21.15.5:0/2874407208 shutdown_connections 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/2874407208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74103b60 0x7f9b7410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/2874407208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b741031a0 0x7f9b74103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/2874407208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b74101fb0 0x7f9b741023d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 -- 172.21.15.5:0/2874407208 >> 172.21.15.5:0/2874407208 conn(0x7f9b740fd520 msgr2=0x7f9b740ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 -- 172.21.15.5:0/2874407208 shutdown_connections 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.030+0000 7f9b7bef2640 1 -- 172.21.15.5:0/2874407208 wait complete. 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 Processor -- start 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 -- start start 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74101fb0 0x7f9b74198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 0x7f9b741987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74103b60 0x7f9b7419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7410db30 con 0x7f9b741031a0 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7410d9b0 con 0x7f9b74101fb0 2024-04-22T13:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7bef2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7410dcb0 con 0x7f9b74103b60 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 0x7f9b741987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 0x7f9b741987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55800/0 (socket says 172.21.15.5:55800) 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 -- 172.21.15.5:0/444922221 learned_addr learned my addr 172.21.15.5:0/444922221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b79c8e640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74101fb0 0x7f9b74198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7a48f640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74103b60 0x7f9b7419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 -- 172.21.15.5:0/444922221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74103b60 msgr2=0x7f9b7419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74103b60 0x7f9b7419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 -- 172.21.15.5:0/444922221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74101fb0 msgr2=0x7f9b74198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74101fb0 0x7f9b74198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b7948d640 1 -- 172.21.15.5:0/444922221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7419fe80 con 0x7f9b741031a0 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.031+0000 7f9b79c8e640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74101fb0 0x7f9b74198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.032+0000 7f9b7948d640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 0x7f9b741987a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400c890 tx=0x7f9b6400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.032+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64004220 con 0x7f9b741031a0 2024-04-22T13:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.032+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b640043c0 con 0x7f9b741031a0 2024-04-22T13:26:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.032+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b741a0170 con 0x7f9b741031a0 2024-04-22T13:26:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.032+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64004a00 con 0x7f9b741031a0 2024-04-22T13:26:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.032+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b741a0740 con 0x7f9b741031a0 2024-04-22T13:26:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.033+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b741093a0 con 0x7f9b741031a0 2024-04-22T13:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.034+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9b640079b0 con 0x7f9b741031a0 2024-04-22T13:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.035+0000 7f9b6affd640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b600a3c40 0x7f9b600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.035+0000 7f9b79c8e640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b600a3c40 0x7f9b600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.035+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f9b640d0370 con 0x7f9b741031a0 2024-04-22T13:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.035+0000 7f9b79c8e640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b600a3c40 0x7f9b600a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006010 tx=0x7f9b700095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.039+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b64014030 con 0x7f9b741031a0 2024-04-22T13:26:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.228+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b7402d810 con 0x7f9b741031a0 2024-04-22T13:26:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.229+0000 7f9b6affd640 1 -- 172.21.15.5:0/444922221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f9b6409cc00 con 0x7f9b741031a0 2024-04-22T13:26:33.225 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:26:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b600a3c40 msgr2=0x7f9b600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b600a3c40 0x7f9b600a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006010 tx=0x7f9b700095f0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 msgr2=0x7f9b741987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 0x7f9b741987a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400c890 tx=0x7f9b6400cd50 comp rx=0 tx=0).stop 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 shutdown_connections 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9b600a3c40 0x7f9b600a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b741031a0 0x7f9b741987a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74103b60 0x7f9b7419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 --2- 172.21.15.5:0/444922221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b74101fb0 0x7f9b74198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 >> 172.21.15.5:0/444922221 conn(0x7f9b740fd520 msgr2=0x7f9b74106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 shutdown_connections 2024-04-22T13:26:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.231+0000 7f9b7bef2640 1 -- 172.21.15.5:0/444922221 wait complete. 2024-04-22T13:26:33.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:32 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1663647096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:33.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:32 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2090783013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:33.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:26:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:32 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1663647096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:32 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2090783013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:32 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1663647096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:32 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2090783013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/409262285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8100e90 msgr2=0x7f8fc81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/409262285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8100e90 0x7f8fc81012b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009900 tx=0x7f8fbc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/409262285 shutdown_connections 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/409262285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc81029a0 0x7f8fc81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/409262285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc8101fe0 0x7f8fc8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/409262285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8100e90 0x7f8fc81012b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/409262285 >> 172.21.15.5:0/409262285 conn(0x7f8fc80fc420 msgr2=0x7f8fc80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/409262285 shutdown_connections 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/409262285 wait complete. 2024-04-22T13:26:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 Processor -- start 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 -- start start 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.348+0000 7f8fcf0a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100e90 0x7f8fc8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcf0a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 0x7f8fc81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcf0a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc81029a0 0x7f8fc819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcf0a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810daf0 con 0x7f8fc8101fe0 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcf0a8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810d970 con 0x7f8fc81029a0 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcf0a8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810dc70 con 0x7f8fc8100e90 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 0x7f8fc81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 0x7f8fc81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55826/0 (socket says 172.21.15.5:55826) 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 -- 172.21.15.5:0/447728601 learned_addr learned my addr 172.21.15.5:0/447728601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcce44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100e90 0x7f8fc8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcd645640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc81029a0 0x7f8fc819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 -- 172.21.15.5:0/447728601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100e90 msgr2=0x7f8fc8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100e90 0x7f8fc8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 -- 172.21.15.5:0/447728601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc81029a0 msgr2=0x7f8fc819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc81029a0 0x7f8fc819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 -- 172.21.15.5:0/447728601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc819fe30 con 0x7f8fc8101fe0 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc7fff640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 0x7f8fc81987a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400e9c0 tx=0x7f8fb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400cd80 con 0x7f8fc8101fe0 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb40035f0 con 0x7f8fc8101fe0 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc81a0120 con 0x7f8fc8101fe0 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.349+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4010690 con 0x7f8fc8101fe0 2024-04-22T13:26:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.350+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc81a0670 con 0x7f8fc8101fe0 2024-04-22T13:26:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.350+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc8108310 con 0x7f8fc8101fe0 2024-04-22T13:26:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.351+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f8fb40037a0 con 0x7f8fc8101fe0 2024-04-22T13:26:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.352+0000 7f8fc5ffb640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8fb80a3c40 0x7f8fb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.352+0000 7f8fcce44640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8fb80a3c40 0x7f8fb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.352+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f8fb40d08e0 con 0x7f8fc8101fe0 2024-04-22T13:26:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.353+0000 7f8fcce44640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8fb80a3c40 0x7f8fb80a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009230 tx=0x7f8fbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.357+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fb409d170 con 0x7f8fc8101fe0 2024-04-22T13:26:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.543+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8fc802d810 con 0x7f8fc8101fe0 2024-04-22T13:26:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.543+0000 7f8fc5ffb640 1 -- 172.21.15.5:0/447728601 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f8fb409ca70 con 0x7f8fc8101fe0 2024-04-22T13:26:33.540 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:26:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8fb80a3c40 msgr2=0x7f8fb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8fb80a3c40 0x7f8fb80a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009230 tx=0x7f8fbc005fd0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 msgr2=0x7f8fc81987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 0x7f8fc81987a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400e9c0 tx=0x7f8fb400ee80 comp rx=0 tx=0).stop 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 shutdown_connections 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8fb80a3c40 0x7f8fb80a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc8101fe0 0x7f8fc81987a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fc81029a0 0x7f8fc819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 --2- 172.21.15.5:0/447728601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100e90 0x7f8fc8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 >> 172.21.15.5:0/447728601 conn(0x7f8fc80fc420 msgr2=0x7f8fc81054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 shutdown_connections 2024-04-22T13:26:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.545+0000 7f8fcf0a8640 1 -- 172.21.15.5:0/447728601 wait complete. 2024-04-22T13:26:33.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- 172.21.15.5:0/78472773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 msgr2=0x7f9688104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- 172.21.15.5:0/78472773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f9688104e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9684009900 tx=0x7f968402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- 172.21.15.5:0/78472773 shutdown_connections 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- 172.21.15.5:0/78472773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9688107c70 0x7f968810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- 172.21.15.5:0/78472773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9688105340 0x7f9688107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- 172.21.15.5:0/78472773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f9688104e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- 172.21.15.5:0/78472773 >> 172.21.15.5:0/78472773 conn(0x7f96880fc420 msgr2=0x7f96880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- 172.21.15.5:0/78472773 shutdown_connections 2024-04-22T13:26:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- 172.21.15.5:0/78472773 wait complete. 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 Processor -- start 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- start start 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f96881981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9688105340 0x7f9688198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9688107c70 0x7f968819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f968810dae0 con 0x7f9688102a10 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f968810d960 con 0x7f9688105340 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.659+0000 7f969033d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f968810dc60 con 0x7f9688107c70 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f96881981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f96881981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55846/0 (socket says 172.21.15.5:55846) 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 -- 172.21.15.5:0/2481653348 learned_addr learned my addr 172.21.15.5:0/2481653348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968d8d8640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9688105340 0x7f9688198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e8da640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9688107c70 0x7f968819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 -- 172.21.15.5:0/2481653348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9688107c70 msgr2=0x7f968819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9688107c70 0x7f968819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 -- 172.21.15.5:0/2481653348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9688105340 msgr2=0x7f9688198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9688105340 0x7f9688198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 -- 172.21.15.5:0/2481653348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968819fe00 con 0x7f9688102a10 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f968e0d9640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f96881981e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684043020 con 0x7f9688102a10 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96881a0090 con 0x7f9688102a10 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9684004440 con 0x7f9688102a10 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684003bf0 con 0x7f9688102a10 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.660+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96881a0630 con 0x7f9688102a10 2024-04-22T13:26:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.661+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96880fdf10 con 0x7f9688102a10 2024-04-22T13:26:33.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.662+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9684046020 con 0x7f9688102a10 2024-04-22T13:26:33.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.663+0000 7f967f7fe640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96740a3c40 0x7f96740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:33.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.663+0000 7f968d8d8640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96740a3c40 0x7f96740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:33.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.663+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f96840f2830 con 0x7f9688102a10 2024-04-22T13:26:33.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.663+0000 7f968d8d8640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96740a3c40 0x7f96740a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96780095f0 tx=0x7f9678005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.667+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f968404d020 con 0x7f9688102a10 2024-04-22T13:26:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.893+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f968802d5c0 con 0x7f9688102a10 2024-04-22T13:26:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.894+0000 7f967f7fe640 1 -- 172.21.15.5:0/2481653348 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11905 (secure 0 0 0) 0x7f96840bf1d0 con 0x7f9688102a10 2024-04-22T13:26:33.890 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:33.890 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:25:17.229740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24485,24469,24496,24557,24512]},"id":1}]} 2024-04-22T13:26:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96740a3c40 msgr2=0x7f96740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96740a3c40 0x7f96740a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96780095f0 tx=0x7f9678005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 msgr2=0x7f96881981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f96881981e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684004040 comp rx=0 tx=0).stop 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 shutdown_connections 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f96740a3c40 0x7f96740a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9688107c70 0x7f968819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9688105340 0x7f9688198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 --2- 172.21.15.5:0/2481653348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9688102a10 0x7f96881981e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 >> 172.21.15.5:0/2481653348 conn(0x7f96880fc420 msgr2=0x7f9688103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 shutdown_connections 2024-04-22T13:26:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:33.896+0000 7f969033d640 1 -- 172.21.15.5:0/2481653348 wait complete. 2024-04-22T13:26:33.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:26:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 -- 172.21.15.5:0/1901810850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8102990 msgr2=0x7ff8a81092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/1901810850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8102990 0x7ff8a81092b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009ca0 tx=0x7ff8a002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 -- 172.21.15.5:0/1901810850 shutdown_connections 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/1901810850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8102990 0x7ff8a81092b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/1901810850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8a8101fd0 0x7ff8a8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/1901810850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8100e80 0x7ff8a81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 -- 172.21.15.5:0/1901810850 >> 172.21.15.5:0/1901810850 conn(0x7ff8a80fc3f0 msgr2=0x7ff8a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 -- 172.21.15.5:0/1901810850 shutdown_connections 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.010+0000 7ff8ae46d640 1 -- 172.21.15.5:0/1901810850 wait complete. 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 Processor -- start 2024-04-22T13:26:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 -- start start 2024-04-22T13:26:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 0x7ff8a8069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101fd0 0x7ff8a819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8a8102990 0x7ff8a819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a810c9c0 con 0x7ff8a8100e80 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a810c840 con 0x7ff8a8101fd0 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8ae46d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a810cb40 con 0x7ff8a8102990 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 0x7ff8a8069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 0x7ff8a8069dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55866/0 (socket says 172.21.15.5:55866) 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3027225660 learned_addr learned my addr 172.21.15.5:0/3027225660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a77fe640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101fd0 0x7ff8a819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8aca0a640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8a8102990 0x7ff8a819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3027225660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8a8102990 msgr2=0x7ff8a819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8a8102990 0x7ff8a819f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3027225660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101fd0 msgr2=0x7ff8a819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101fd0 0x7ff8a819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.011+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3027225660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a819fcd0 con 0x7ff8a8100e80 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.012+0000 7ff8a7fff640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 0x7ff8a8069dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff89800d950 tx=0x7ff89800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.012+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898017070 con 0x7ff8a8100e80 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.012+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff898009c30 con 0x7ff8a8100e80 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.012+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a819ff60 con 0x7ff8a8100e80 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.012+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89800f620 con 0x7ff8a8100e80 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.012+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a81a03c0 con 0x7ff8a8100e80 2024-04-22T13:26:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.013+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a81080c0 con 0x7ff8a8100e80 2024-04-22T13:26:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.014+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7ff898010460 con 0x7ff8a8100e80 2024-04-22T13:26:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.015+0000 7ff8a57fa640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff8900a3c40 0x7ff8900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.015+0000 7ff8a77fe640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff8900a3c40 0x7ff8900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.015+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7ff8980d0210 con 0x7ff8a8100e80 2024-04-22T13:26:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.015+0000 7ff8a77fe640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff8900a3c40 0x7ff8900a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8069b30 tx=0x7ff89c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.019+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff89809ca40 con 0x7ff8a8100e80 2024-04-22T13:26:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.208+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8a802d870 con 0x7ff8a8100e80 2024-04-22T13:26:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.209+0000 7ff8a57fa640 1 -- 172.21.15.5:0/3027225660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22549 (secure 0 0 0) 0x7ff898081830 con 0x7ff8a8100e80 2024-04-22T13:26:34.206 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:34.206 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:26:15.450938+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:26:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff8900a3c40 msgr2=0x7ff8900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff8900a3c40 0x7ff8900a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8069b30 tx=0x7ff89c005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 msgr2=0x7ff8a8069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 0x7ff8a8069dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff89800d950 tx=0x7ff89800de10 comp rx=0 tx=0).stop 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 shutdown_connections 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7ff8900a3c40 0x7ff8900a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8a8102990 0x7ff8a819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101fd0 0x7ff8a819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 --2- 172.21.15.5:0/3027225660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a8100e80 0x7ff8a8069dc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 >> 172.21.15.5:0/3027225660 conn(0x7ff8a80fc3f0 msgr2=0x7ff8a8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 shutdown_connections 2024-04-22T13:26:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.211+0000 7ff8ae46d640 1 -- 172.21.15.5:0/3027225660 wait complete. 2024-04-22T13:26:34.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:33 smithi005 ceph-mon[28006]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:26:34.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:33 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/444922221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:34.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:33 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/447728601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:34.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:33 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2481653348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:34.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:26:34.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:26:34.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:33 smithi049 ceph-mon[31379]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:26:34.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:33 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/444922221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:34.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:33 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/447728601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:34.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:33 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2481653348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:34.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:33 smithi107 ceph-mon[31859]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:26:34.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:33 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/444922221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:34.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:33 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/447728601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:34.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:33 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2481653348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 -- 172.21.15.5:0/3895071424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc474101fb0 msgr2=0x7fc4741023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 --2- 172.21.15.5:0/3895071424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc474101fb0 0x7fc4741023d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009900 tx=0x7fc46c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 -- 172.21.15.5:0/3895071424 shutdown_connections 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 --2- 172.21.15.5:0/3895071424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc474103b60 0x7fc47410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 --2- 172.21.15.5:0/3895071424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4741031a0 0x7fc474103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 --2- 172.21.15.5:0/3895071424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc474101fb0 0x7fc4741023d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 -- 172.21.15.5:0/3895071424 >> 172.21.15.5:0/3895071424 conn(0x7fc4740fd520 msgr2=0x7fc4740ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 -- 172.21.15.5:0/3895071424 shutdown_connections 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.339+0000 7fc47aefd640 1 -- 172.21.15.5:0/3895071424 wait complete. 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 Processor -- start 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 -- start start 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc474101fb0 0x7fc474075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 0x7fc474074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc474103b60 0x7fc474074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410db40 con 0x7fc4741031a0 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410d9c0 con 0x7fc474101fb0 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47aefd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410dcc0 con 0x7fc474103b60 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 0x7fc474074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 0x7fc474074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55884/0 (socket says 172.21.15.5:55884) 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc473fff640 1 -- 172.21.15.5:0/2608122605 learned_addr learned my addr 172.21.15.5:0/2608122605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:34.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc478c99640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc474101fb0 0x7fc474075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc47949a640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc474103b60 0x7fc474074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc473fff640 1 -- 172.21.15.5:0/2608122605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc474103b60 msgr2=0x7fc474074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.340+0000 7fc473fff640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc474103b60 0x7fc474074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc473fff640 1 -- 172.21.15.5:0/2608122605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc474101fb0 msgr2=0x7fc474075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc473fff640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc474101fb0 0x7fc474075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc473fff640 1 -- 172.21.15.5:0/2608122605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc474075110 con 0x7fc4741031a0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc473fff640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 0x7fc474074340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc46400db80 tx=0x7fc46400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464017070 con 0x7fc4741031a0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc464003b70 con 0x7fc4741031a0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a2340 con 0x7fc4741031a0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4640106c0 con 0x7fc4741031a0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.341+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a2890 con 0x7fc4741031a0 2024-04-22T13:26:34.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.342+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4741093f0 con 0x7fc4741031a0 2024-04-22T13:26:34.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.343+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fc464004020 con 0x7fc4741031a0 2024-04-22T13:26:34.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.344+0000 7fc471ffb640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc4600a3c40 0x7fc4600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.344+0000 7fc478c99640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc4600a3c40 0x7fc4600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.344+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fc4640d0660 con 0x7fc4741031a0 2024-04-22T13:26:34.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.344+0000 7fc478c99640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc4600a3c40 0x7fc4600a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc46c02c240 tx=0x7fc46c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.348+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc46409ce90 con 0x7fc4741031a0 2024-04-22T13:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.535+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc47402d770 con 0x7fc4741031a0 2024-04-22T13:26:34.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.536+0000 7fc471ffb640 1 -- 172.21.15.5:0/2608122605 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fc46409c790 con 0x7fc4741031a0 2024-04-22T13:26:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc4600a3c40 msgr2=0x7fc4600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc4600a3c40 0x7fc4600a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc46c02c240 tx=0x7fc46c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 msgr2=0x7fc474074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 0x7fc474074340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc46400db80 tx=0x7fc46400b630 comp rx=0 tx=0).stop 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 shutdown_connections 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc4600a3c40 0x7fc4600a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4741031a0 0x7fc474074340 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc474103b60 0x7fc474074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 --2- 172.21.15.5:0/2608122605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc474101fb0 0x7fc474075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 >> 172.21.15.5:0/2608122605 conn(0x7fc4740fd520 msgr2=0x7fc4741065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 shutdown_connections 2024-04-22T13:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.538+0000 7fc47aefd640 1 -- 172.21.15.5:0/2608122605 wait complete. 2024-04-22T13:26:34.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:26:34.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1200871762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 msgr2=0x7f55bc10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1200871762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc10a3f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009ca0 tx=0x7f55ac02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1200871762 shutdown_connections 2024-04-22T13:26:34.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1200871762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc10a3f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1200871762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55bc1031a0 0x7f55bc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1200871762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc101fb0 0x7f55bc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1200871762 >> 172.21.15.5:0/1200871762 conn(0x7f55bc0fd520 msgr2=0x7f55bc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1200871762 shutdown_connections 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.654+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1200871762 wait complete. 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 Processor -- start 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 -- start start 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55bc101fb0 0x7f55bc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1031a0 0x7f55bc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10daf0 con 0x7f55bc103b60 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10d970 con 0x7f55bc101fb0 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c3ab2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10dc70 con 0x7f55bc1031a0 2024-04-22T13:26:34.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55896/0 (socket says 172.21.15.5:55896) 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 -- 172.21.15.5:0/1901214384 learned_addr learned my addr 172.21.15.5:0/1901214384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c104d640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1031a0 0x7f55bc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c184e640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55bc101fb0 0x7f55bc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 -- 172.21.15.5:0/1901214384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1031a0 msgr2=0x7f55bc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1031a0 0x7f55bc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 -- 172.21.15.5:0/1901214384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55bc101fb0 msgr2=0x7f55bc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55bc101fb0 0x7f55bc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 -- 172.21.15.5:0/1901214384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc19fdc0 con 0x7f55bc103b60 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.655+0000 7f55c204f640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc19f750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55ac02c700 tx=0x7f55ac0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.656+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac041020 con 0x7f55bc103b60 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.656+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac034d60 con 0x7f55bc103b60 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.656+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1a0050 con 0x7f55bc103b60 2024-04-22T13:26:34.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.656+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac039040 con 0x7f55bc103b60 2024-04-22T13:26:34.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.656+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a0540 con 0x7f55bc103b60 2024-04-22T13:26:34.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.656+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc109320 con 0x7f55bc103b60 2024-04-22T13:26:34.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.658+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f55ac044020 con 0x7f55bc103b60 2024-04-22T13:26:34.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.658+0000 7f55baffd640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55a40a3c40 0x7f55a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.659+0000 7f55c184e640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55a40a3c40 0x7f55a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.659+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f55ac0f28e0 con 0x7f55bc103b60 2024-04-22T13:26:34.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.659+0000 7f55c184e640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55a40a3c40 0x7f55a40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55bc103000 tx=0x7f55b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.662+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55ac04b020 con 0x7f55bc103b60 2024-04-22T13:26:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.847+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55bc02d5c0 con 0x7f55bc103b60 2024-04-22T13:26:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.848+0000 7f55baffd640 1 -- 172.21.15.5:0/1901214384 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f55ac0bf280 con 0x7f55bc103b60 2024-04-22T13:26:34.844 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55a40a3c40 msgr2=0x7f55a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55a40a3c40 0x7f55a40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55bc103000 tx=0x7f55b4005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 msgr2=0x7f55bc19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc19f750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55ac02c700 tx=0x7f55ac0093f0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 shutdown_connections 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f55a40a3c40 0x7f55a40a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc103b60 0x7f55bc19f750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1031a0 0x7f55bc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 --2- 172.21.15.5:0/1901214384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55bc101fb0 0x7f55bc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 >> 172.21.15.5:0/1901214384 conn(0x7f55bc0fd520 msgr2=0x7f55bc1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 shutdown_connections 2024-04-22T13:26:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.850+0000 7f55c3ab2640 1 -- 172.21.15.5:0/1901214384 wait complete. 2024-04-22T13:26:34.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 -- 172.21.15.5:0/1721852795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 msgr2=0x7fcac81077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 --2- 172.21.15.5:0/1721852795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81077b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009900 tx=0x7fcab802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 -- 172.21.15.5:0/1721852795 shutdown_connections 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 --2- 172.21.15.5:0/1721852795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81077b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 --2- 172.21.15.5:0/1721852795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8107cf0 0x7fcac810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 --2- 172.21.15.5:0/1721852795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcac8102a90 0x7fcac8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.967+0000 7fcacfe06640 1 -- 172.21.15.5:0/1721852795 >> 172.21.15.5:0/1721852795 conn(0x7fcac80fc420 msgr2=0x7fcac80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 -- 172.21.15.5:0/1721852795 shutdown_connections 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 -- 172.21.15.5:0/1721852795 wait complete. 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 Processor -- start 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 -- start start 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8102a90 0x7fcac8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcac8107cf0 0x7fcac819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810dab0 con 0x7fcac81053c0 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810d930 con 0x7fcac8107cf0 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacfe06640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810dc30 con 0x7fcac8102a90 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacd3a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacd3a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55928/0 (socket says 172.21.15.5:55928) 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacd3a1640 1 -- 172.21.15.5:0/3773209656 learned_addr learned my addr 172.21.15.5:0/3773209656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcacdba2640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8102a90 0x7fcac8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.968+0000 7fcace3a3640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcac8107cf0 0x7fcac819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacd3a1640 1 -- 172.21.15.5:0/3773209656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8102a90 msgr2=0x7fcac8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacd3a1640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8102a90 0x7fcac8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacd3a1640 1 -- 172.21.15.5:0/3773209656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcac8107cf0 msgr2=0x7fcac819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacd3a1640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcac8107cf0 0x7fcac819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacd3a1640 1 -- 172.21.15.5:0/3773209656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac819fda0 con 0x7fcac81053c0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacdba2640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8102a90 0x7fcac8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacd3a1640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81986c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000c00 tx=0x7fcab8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8043020 con 0x7fcac81053c0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81a0030 con 0x7fcac81053c0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab8004440 con 0x7fcac81053c0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8003c90 con 0x7fcac81053c0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.969+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81a0580 con 0x7fcac81053c0 2024-04-22T13:26:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.970+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac80fdf10 con 0x7fcac81053c0 2024-04-22T13:26:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.971+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fcab8046020 con 0x7fcac81053c0 2024-04-22T13:26:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.972+0000 7fcabeffd640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab40a3c40 0x7fcab40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.972+0000 7fcacdba2640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab40a3c40 0x7fcab40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.972+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fcab80f2780 con 0x7fcac81053c0 2024-04-22T13:26:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.972+0000 7fcacdba2640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab40a3c40 0x7fcab40a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcac400abe0 tx=0x7fcac400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:34.976+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcab804d020 con 0x7fcac81053c0 2024-04-22T13:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.165+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcac802d810 con 0x7fcac81053c0 2024-04-22T13:26:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.166+0000 7fcabeffd640 1 -- 172.21.15.5:0/3773209656 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fcab803c350 con 0x7fcac81053c0 2024-04-22T13:26:35.162 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab40a3c40 msgr2=0x7fcab40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab40a3c40 0x7fcab40a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcac400abe0 tx=0x7fcac400a5c0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 msgr2=0x7fcac81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81986c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000c00 tx=0x7fcab8004040 comp rx=0 tx=0).stop 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 shutdown_connections 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fcab40a3c40 0x7fcab40a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac81053c0 0x7fcac81986c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcac8107cf0 0x7fcac819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 --2- 172.21.15.5:0/3773209656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac8102a90 0x7fcac8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 >> 172.21.15.5:0/3773209656 conn(0x7fcac80fc420 msgr2=0x7fcac8103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:35.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 shutdown_connections 2024-04-22T13:26:35.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.168+0000 7fcacfe06640 1 -- 172.21.15.5:0/3773209656 wait complete. 2024-04-22T13:26:35.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3027225660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:35.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2608122605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:35.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:34 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1901214384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:35.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:35.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3027225660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:35.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2608122605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:35.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:34 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1901214384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.280+0000 7f4d24463640 1 -- 172.21.15.5:0/2050939329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c101fa0 msgr2=0x7f4d1c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.280+0000 7f4d24463640 1 --2- 172.21.15.5:0/2050939329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c101fa0 0x7f4d1c1023c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- 172.21.15.5:0/2050939329 shutdown_connections 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 --2- 172.21.15.5:0/2050939329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c103b50 0x7f4d1c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 --2- 172.21.15.5:0/2050939329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c103190 0x7f4d1c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 --2- 172.21.15.5:0/2050939329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c101fa0 0x7f4d1c1023c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- 172.21.15.5:0/2050939329 >> 172.21.15.5:0/2050939329 conn(0x7f4d1c0fd570 msgr2=0x7f4d1c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- 172.21.15.5:0/2050939329 shutdown_connections 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- 172.21.15.5:0/2050939329 wait complete. 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 Processor -- start 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- start start 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c101fa0 0x7f4d1c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c103190 0x7f4d1c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 0x7f4d1c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10dae0 con 0x7f4d1c103b50 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10d960 con 0x7f4d1c101fa0 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.281+0000 7f4d24463640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10dc60 con 0x7f4d1c103190 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 0x7f4d1c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 0x7f4d1c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55936/0 (socket says 172.21.15.5:55936) 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 -- 172.21.15.5:0/2105177384 learned_addr learned my addr 172.21.15.5:0/2105177384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d221ff640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c101fa0 0x7f4d1c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d219fe640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c103190 0x7f4d1c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 -- 172.21.15.5:0/2105177384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c103190 msgr2=0x7f4d1c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c103190 0x7f4d1c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 -- 172.21.15.5:0/2105177384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c101fa0 msgr2=0x7f4d1c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c101fa0 0x7f4d1c075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 -- 172.21.15.5:0/2105177384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c075110 con 0x7f4d1c103b50 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d22a00640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 0x7f4d1c074880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400eaa0 tx=0x7f4d1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400cdb0 con 0x7f4d1c103b50 2024-04-22T13:26:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a2280 con 0x7f4d1c103b50 2024-04-22T13:26:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d14005020 con 0x7f4d1c103b50 2024-04-22T13:26:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140116d0 con 0x7f4d1c103b50 2024-04-22T13:26:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.282+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a2820 con 0x7f4d1c103b50 2024-04-22T13:26:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.283+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c1092c0 con 0x7f4d1c103b50 2024-04-22T13:26:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.284+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f4d140054a0 con 0x7f4d1c103b50 2024-04-22T13:26:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.285+0000 7f4d137fe640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4d080a3c40 0x7f4d080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.285+0000 7f4d221ff640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4d080a3c40 0x7f4d080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.285+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f4d140d0690 con 0x7f4d1c103b50 2024-04-22T13:26:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.285+0000 7f4d221ff640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4d080a3c40 0x7f4d080a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005ab0 tx=0x7f4d18005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:35.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3027225660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:35.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2608122605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:35.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:34 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1901214384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.289+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d14016030 con 0x7f4d1c103b50 2024-04-22T13:26:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.516+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d1c062b50 con 0x7f4d1c103b50 2024-04-22T13:26:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.517+0000 7f4d137fe640 1 -- 172.21.15.5:0/2105177384 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11905 (secure 0 0 0) 0x7f4d1409c8d0 con 0x7f4d1c103b50 2024-04-22T13:26:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:35.513 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:25:17.229740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24485,24469,24496,24557,24512]},"id":1}]} 2024-04-22T13:26:35.514 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:26:35.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4d080a3c40 msgr2=0x7f4d080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4d080a3c40 0x7f4d080a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005ab0 tx=0x7f4d18005fd0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 msgr2=0x7f4d1c074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 0x7f4d1c074880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400eaa0 tx=0x7f4d1400ef60 comp rx=0 tx=0).stop 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 shutdown_connections 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4d080a3c40 0x7f4d080a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c103b50 0x7f4d1c074880 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c103190 0x7f4d1c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 --2- 172.21.15.5:0/2105177384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c101fa0 0x7f4d1c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 >> 172.21.15.5:0/2105177384 conn(0x7f4d1c0fd570 msgr2=0x7f4d1c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.519+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 shutdown_connections 2024-04-22T13:26:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.520+0000 7f4d24463640 1 -- 172.21.15.5:0/2105177384 wait complete. 2024-04-22T13:26:35.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.632+0000 7fef13c8e640 1 -- 172.21.15.5:0/36264755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 msgr2=0x7fef0c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.632+0000 7fef13c8e640 1 --2- 172.21.15.5:0/36264755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c1092b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef04009ca0 tx=0x7fef0402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.632+0000 7fef13c8e640 1 -- 172.21.15.5:0/36264755 shutdown_connections 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.632+0000 7fef13c8e640 1 --2- 172.21.15.5:0/36264755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c1092b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 --2- 172.21.15.5:0/36264755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef0c101fd0 0x7fef0c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 --2- 172.21.15.5:0/36264755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 0x7fef0c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- 172.21.15.5:0/36264755 >> 172.21.15.5:0/36264755 conn(0x7fef0c0fc3f0 msgr2=0x7fef0c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- 172.21.15.5:0/36264755 shutdown_connections 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- 172.21.15.5:0/36264755 wait complete. 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 Processor -- start 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- start start 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 0x7fef0c069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef0c101fd0 0x7fef0c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10c9c0 con 0x7fef0c102990 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10c840 con 0x7fef0c101fd0 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef13c8e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10cb40 con 0x7fef0c100e80 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.633+0000 7fef1222b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55944/0 (socket says 172.21.15.5:55944) 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 -- 172.21.15.5:0/2379567221 learned_addr learned my addr 172.21.15.5:0/2379567221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef11229640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef0c101fd0 0x7fef0c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef11a2a640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 0x7fef0c069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 -- 172.21.15.5:0/2379567221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 msgr2=0x7fef0c069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 0x7fef0c069dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 -- 172.21.15.5:0/2379567221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef0c101fd0 msgr2=0x7fef0c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef0c101fd0 0x7fef0c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 -- 172.21.15.5:0/2379567221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c19fd00 con 0x7fef0c102990 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef11a2a640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 0x7fef0c069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef1222b640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c19f3f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fef04032520 tx=0x7fef040347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04039040 con 0x7fef0c102990 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef04044070 con 0x7fef0c102990 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c19ff30 con 0x7fef0c102990 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040042a0 con 0x7fef0c102990 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.634+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c1a0390 con 0x7fef0c102990 2024-04-22T13:26:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.635+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c108190 con 0x7fef0c102990 2024-04-22T13:26:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.636+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fef04003970 con 0x7fef0c102990 2024-04-22T13:26:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.637+0000 7fef02ffd640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7feef80a3c40 0x7feef80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.637+0000 7fef11a2a640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7feef80a3c40 0x7feef80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.637+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fef040f28c0 con 0x7fef0c102990 2024-04-22T13:26:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.637+0000 7fef11a2a640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7feef80a3c40 0x7feef80a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fef08009b40 tx=0x7fef08005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.641+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef040333a0 con 0x7fef0c102990 2024-04-22T13:26:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.827+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef0c02d810 con 0x7fef0c102990 2024-04-22T13:26:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.829+0000 7fef02ffd640 1 -- 172.21.15.5:0/2379567221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22549 (secure 0 0 0) 0x7fef040bf200 con 0x7fef0c102990 2024-04-22T13:26:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:35.825 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:26:15.450938+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:26:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7feef80a3c40 msgr2=0x7feef80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7feef80a3c40 0x7feef80a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fef08009b40 tx=0x7fef08005e50 comp rx=0 tx=0).stop 2024-04-22T13:26:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 msgr2=0x7fef0c19f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c19f3f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fef04032520 tx=0x7fef040347b0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 shutdown_connections 2024-04-22T13:26:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7feef80a3c40 0x7feef80a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef0c102990 0x7fef0c19f3f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef0c101fd0 0x7fef0c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 --2- 172.21.15.5:0/2379567221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef0c100e80 0x7fef0c069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 >> 172.21.15.5:0/2379567221 conn(0x7fef0c0fc3f0 msgr2=0x7fef0c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 shutdown_connections 2024-04-22T13:26:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.831+0000 7fef13c8e640 1 -- 172.21.15.5:0/2379567221 wait complete. 2024-04-22T13:26:35.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 -- 172.21.15.5:0/2529071047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824101fd0 msgr2=0x7f4824102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 --2- 172.21.15.5:0/2529071047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824101fd0 0x7f4824102450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4818009900 tx=0x7f481802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 -- 172.21.15.5:0/2529071047 shutdown_connections 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 --2- 172.21.15.5:0/2529071047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824101fd0 0x7f4824102450 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 --2- 172.21.15.5:0/2529071047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4824102990 0x7f48241092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 --2- 172.21.15.5:0/2529071047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4824100e80 0x7f48241012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 -- 172.21.15.5:0/2529071047 >> 172.21.15.5:0/2529071047 conn(0x7f48240fc3f0 msgr2=0x7f48240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 -- 172.21.15.5:0/2529071047 shutdown_connections 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.944+0000 7f4829d6d640 1 -- 172.21.15.5:0/2529071047 wait complete. 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 Processor -- start 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 -- start start 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4824100e80 0x7f4824198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4824101fd0 0x7f48241987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 0x7f482419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f482410ca30 con 0x7f4824102990 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c8b0 con 0x7f4824100e80 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4829d6d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f482410cbb0 con 0x7f4824101fd0 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4823fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 0x7f482419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4823fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 0x7f482419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55956/0 (socket says 172.21.15.5:55956) 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.945+0000 7f4823fff640 1 -- 172.21.15.5:0/23421489 learned_addr learned my addr 172.21.15.5:0/23421489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f48237fe640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4824100e80 0x7f4824198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4822ffd640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4824101fd0 0x7f48241987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4823fff640 1 -- 172.21.15.5:0/23421489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4824101fd0 msgr2=0x7f48241987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4823fff640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4824101fd0 0x7f48241987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4823fff640 1 -- 172.21.15.5:0/23421489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4824100e80 msgr2=0x7f4824198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4823fff640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4824100e80 0x7f4824198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4823fff640 1 -- 172.21.15.5:0/23421489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482419fe50 con 0x7f4824102990 2024-04-22T13:26:35.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4823fff640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 0x7f482419f7e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f481c00b970 tx=0x7f481c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c019070 con 0x7f4824102990 2024-04-22T13:26:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481c014440 con 0x7f4824102990 2024-04-22T13:26:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c0075d0 con 0x7f4824102990 2024-04-22T13:26:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a0140 con 0x7f4824102990 2024-04-22T13:26:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.946+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a06e0 con 0x7f4824102990 2024-04-22T13:26:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.947+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48241a09a0 con 0x7f4824102990 2024-04-22T13:26:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.948+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f481c01c070 con 0x7f4824102990 2024-04-22T13:26:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.949+0000 7f4820ff9640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f480c0a3c40 0x7f480c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.949+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f481c0cf9c0 con 0x7f4824102990 2024-04-22T13:26:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.949+0000 7f48237fe640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f480c0a3c40 0x7f480c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.949+0000 7f48237fe640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f480c0a3c40 0x7f480c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4814009bd0 tx=0x7f4814005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:35.954+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481c09bd10 con 0x7f4824102990 2024-04-22T13:26:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.180+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48241a0b70 con 0x7f4824102990 2024-04-22T13:26:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.181+0000 7f4820ff9640 1 -- 172.21.15.5:0/23421489 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11905 (secure 0 0 0) 0x7f481c0bae20 con 0x7f4824102990 2024-04-22T13:26:36.177 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:36.177 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:25:17.229740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24485,24469,24496,24557,24512]},"id":1}]} 2024-04-22T13:26:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f480c0a3c40 msgr2=0x7f480c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f480c0a3c40 0x7f480c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4814009bd0 tx=0x7f4814005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 msgr2=0x7f482419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 0x7f482419f7e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f481c00b970 tx=0x7f481c00be30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 shutdown_connections 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f480c0a3c40 0x7f480c0a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4824102990 0x7f482419f7e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4824101fd0 0x7f48241987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 --2- 172.21.15.5:0/23421489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4824100e80 0x7f4824198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 >> 172.21.15.5:0/23421489 conn(0x7f48240fc3f0 msgr2=0x7f48240fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 shutdown_connections 2024-04-22T13:26:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.183+0000 7f4829d6d640 1 -- 172.21.15.5:0/23421489 wait complete. 2024-04-22T13:26:36.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:35 smithi005 ceph-mon[28006]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:26:36.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:35 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3773209656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:36.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:35 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2105177384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:36.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:35 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2379567221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:36.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T13:25:10.226899+0000', 'modified': '2024-04-22T13:25:17.229740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24512, 'mds_2': 24496, 'mds_3': 24485, 'mds_4': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14550': {'gid': 14550, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/1096407808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 1096407808}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 1096407808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/373669901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 373669901}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 373669901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/4083506255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 4083506255}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 4083506255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6833/1584198825', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 1584198825}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 1584198825}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/2603097758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2603097758}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2603097758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.107:6837/207360580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 207360580}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 207360580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.49:6835/827214764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 827214764}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 827214764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.107:6839/3107882391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3107882391}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3107882391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6837/164218196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 164218196}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 164218196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6839/1393810832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 1393810832}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 1393810832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24485, 24469, 24496, 24557, 24512]} 2024-04-22T13:26:36.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T13:26:36.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T13:26:36.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:35 smithi049 ceph-mon[31379]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:26:36.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:35 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3773209656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:36.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:35 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2105177384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:36.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:35 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2379567221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:36.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:35 smithi107 ceph-mon[31859]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:26:36.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:35 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3773209656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:36.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:35 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2105177384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:36.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:35 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2379567221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 -- 172.21.15.5:0/3580026319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 msgr2=0x7fe1201012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 --2- 172.21.15.5:0/3580026319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe1201012b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe114009900 tx=0x7fe11402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 -- 172.21.15.5:0/3580026319 shutdown_connections 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 --2- 172.21.15.5:0/3580026319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1201029a0 0x7fe1201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 --2- 172.21.15.5:0/3580026319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe120101fe0 0x7fe120102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 --2- 172.21.15.5:0/3580026319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe1201012b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 -- 172.21.15.5:0/3580026319 >> 172.21.15.5:0/3580026319 conn(0x7fe1200fc420 msgr2=0x7fe1200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 -- 172.21.15.5:0/3580026319 shutdown_connections 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.297+0000 7fe12719f640 1 -- 172.21.15.5:0/3580026319 wait complete. 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 Processor -- start 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 -- start start 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe12019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120101fe0 0x7fe12019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1201029a0 0x7fe1201a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe12010ca40 con 0x7fe120100e90 2024-04-22T13:26:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe12010c8c0 con 0x7fe120101fe0 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12719f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe12010cbc0 con 0x7fe1201029a0 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe12019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe12019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55968/0 (socket says 172.21.15.5:55968) 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 -- 172.21.15.5:0/1934053782 learned_addr learned my addr 172.21.15.5:0/1934053782 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe12573c640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1201029a0 0x7fe1201a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 -- 172.21.15.5:0/1934053782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1201029a0 msgr2=0x7fe1201a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1201029a0 0x7fe1201a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 -- 172.21.15.5:0/1934053782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120101fe0 msgr2=0x7fe12019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120101fe0 0x7fe12019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.298+0000 7fe124f3b640 1 -- 172.21.15.5:0/1934053782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1201a2080 con 0x7fe120100e90 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.299+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe12019a4b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe114036040 tx=0x7fe1140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.299+0000 7fe11dffb640 1 -- 172.21.15.5:0/1934053782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114043020 con 0x7fe120100e90 2024-04-22T13:26:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.299+0000 7fe11dffb640 1 -- 172.21.15.5:0/1934053782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1140054c0 con 0x7fe120100e90 2024-04-22T13:26:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.299+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1201a2310 con 0x7fe120100e90 2024-04-22T13:26:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.299+0000 7fe11dffb640 1 -- 172.21.15.5:0/1934053782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114003db0 con 0x7fe120100e90 2024-04-22T13:26:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.299+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201a2930 con 0x7fe120100e90 2024-04-22T13:26:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.301+0000 7fe11dffb640 1 -- 172.21.15.5:0/1934053782 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fe114046020 con 0x7fe120100e90 2024-04-22T13:26:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.301+0000 7fe129d8b740 1 Processor -- start 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.301+0000 7fe129d8b740 1 -- start start 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe129d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e542b10 con 0x55cc2e5dde40 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe129d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e4e0240 con 0x55cc2e5de830 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe129d8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e53ead0 con 0x55cc2e5df2d0 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5de260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55980/0 (socket says 172.21.15.5:55980) 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 -- 172.21.15.5:0/2510222386 learned_addr learned my addr 172.21.15.5:0/2510222386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe11ffff640 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe124f3b640 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe11dffb640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1100a3cc0 0x7fe1100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 -- 172.21.15.5:0/2510222386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5df2d0 msgr2=0x55cc2e5e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 -- 172.21.15.5:0/2510222386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 msgr2=0x55cc2e5decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 -- 172.21.15.5:0/2510222386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2e2552d0 con 0x55cc2e5dde40 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe11dffb640 1 -- 172.21.15.5:0/1934053782 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe1140f2940 con 0x7fe120100e90 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1100a3cc0 0x7fe1100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.302+0000 7fe12573c640 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5de260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c4a0 tx=0x7fe11800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83f78a6bd191f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe10affd640 1 -- 172.21.15.5:0/2510222386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118018640 con 0x55cc2e5dde40 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe10affd640 1 -- 172.21.15.5:0/2510222386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11801e070 con 0x55cc2e5dde40 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe12573c640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1100a3cc0 0x7fe1100a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe11801d000 tx=0x7fe118017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe10affd640 1 -- 172.21.15.5:0/2510222386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11800a900 con 0x55cc2e5dde40 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 -- 172.21.15.5:0/2510222386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 msgr2=0x55cc2e5de260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5de260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c4a0 tx=0x7fe11800c960 comp rx=0 tx=0).stop 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 -- 172.21.15.5:0/2510222386 shutdown_connections 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 --2- 172.21.15.5:0/2510222386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5de260 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 -- 172.21.15.5:0/2510222386 >> 172.21.15.5:0/2510222386 conn(0x55cc2e5d9f40 msgr2=0x55cc2e5d8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 -- 172.21.15.5:0/2510222386 shutdown_connections 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.303+0000 7fe129d8b740 1 -- 172.21.15.5:0/2510222386 wait complete. 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 -- start start 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 Processor -- start 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5e64b0 con 0x55cc2e5df2d0 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e4cb190 con 0x55cc2e5de830 2024-04-22T13:26:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5e6630 con 0x55cc2e5dde40 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe129d8b740 10 client.0 _ll_register_callbacks cb 0x55cc2e4c68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe12573c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe12573c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55994/0 (socket says 172.21.15.5:55994) 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe12573c640 1 -- 172.21.15.5:0/1047755349 learned_addr learned my addr 172.21.15.5:0/1047755349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.304+0000 7fe11ffff640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe12573c640 1 -- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 msgr2=0x55cc2e5e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe12573c640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe12573c640 1 -- 172.21.15.5:0/1047755349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 msgr2=0x55cc2e5e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe12573c640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe12573c640 1 -- 172.21.15.5:0/1047755349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2e31f6b0 con 0x55cc2e5df2d0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe11ffff640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe12573c640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ef400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe118004b90 tx=0x7fe118007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4311aa896f1007 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe0fffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe0fffff640 1 -- 172.21.15.5:0/1047755349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118009c20 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe0fffff640 1 -- 172.21.15.5:0/1047755349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe11800a6c0 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11801e070 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe0fffff640 1 -- 172.21.15.5:0/1047755349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe118018650 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 10 client.15054 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc2e5f2770 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe0fffff640 1 -- 172.21.15.5:0/1047755349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11801e070 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe0fffff640 1 -- 172.21.15.5:0/1047755349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe118022550 con 0x55cc2e5df2d0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 10 client.15054 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 10 client.15054 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 20 client.15054 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.305+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e5f4c50 0x55cc2e5f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.306+0000 7fe129d8b740 4 client.15054 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.306+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc2e550530 con 0x55cc2e5f4c50 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.306+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e5f4c50 0x55cc2e5f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.306+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e5f4c50 0x55cc2e5f60c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4f6aa2f6866fb94 server_cookie=bfea6a0f6e0b6907 in_seq=0 out_seq=0 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.306+0000 7fe0fffff640 10 client.15054 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe0fffff640 1 -- 172.21.15.5:0/1047755349 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc2e550530 con 0x55cc2e5f4c50 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe0fffff640 10 client.15054 handle_command_reply: tid=0 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 client.15054 shutdown 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 msgr2=0x55cc2e5ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ef400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe118004b90 tx=0x7fe118007cf0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 shutdown_connections 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e5f4c50 0x55cc2e5f60c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ef400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5de830 0x55cc2e5e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1047755349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe0fffff640 0 client.15054 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 >> 172.21.15.5:0/1047755349 conn(0x55cc2e5d9f40 msgr2=0x55cc2e5d7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 shutdown_connections 2024-04-22T13:26:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 1 -- 172.21.15.5:0/1047755349 wait complete. 2024-04-22T13:26:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.307+0000 7fe129d8b740 20 client.15054 trim_cache size 0 max 16384 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 Processor -- start 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 -- start start 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 0x55cc2e5ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5f4400 con 0x55cc2e5de830 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5f4ad0 con 0x55cc2e5df2d0 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.325+0000 7fe129d8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5f51a0 con 0x55cc2e5dde40 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 0x55cc2e5ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 0x55cc2e5ef3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55998/0 (socket says 172.21.15.5:55998) 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 -- 172.21.15.5:0/1039897428 learned_addr learned my addr 172.21.15.5:0/1039897428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe12573c640 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 -- 172.21.15.5:0/1039897428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 msgr2=0x55cc2e5e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 -- 172.21.15.5:0/1039897428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 msgr2=0x55cc2e5e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 -- 172.21.15.5:0/1039897428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2e5e9000 con 0x55cc2e5de830 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe124f3b640 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe11ffff640 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 0x55cc2e5ef3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00aa70 tx=0x7fe10c00af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ac19aa02e20df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe10a7fc640 1 -- 172.21.15.5:0/1039897428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c005020 con 0x55cc2e5de830 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe10a7fc640 1 -- 172.21.15.5:0/1039897428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe10c0047a0 con 0x55cc2e5de830 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.326+0000 7fe10a7fc640 1 -- 172.21.15.5:0/1039897428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00f780 con 0x55cc2e5de830 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 -- 172.21.15.5:0/1039897428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 msgr2=0x55cc2e5ef3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 0x55cc2e5ef3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00aa70 tx=0x7fe10c00af30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 -- 172.21.15.5:0/1039897428 shutdown_connections 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5de830 0x55cc2e5ef3c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 --2- 172.21.15.5:0/1039897428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5dde40 0x55cc2e5e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 -- 172.21.15.5:0/1039897428 >> 172.21.15.5:0/1039897428 conn(0x55cc2e5d9f40 msgr2=0x55cc2e5e27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 -- 172.21.15.5:0/1039897428 shutdown_connections 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 -- 172.21.15.5:0/1039897428 wait complete. 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 -- start start 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.327+0000 7fe129d8b740 1 Processor -- start 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5eace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5de830 0x55cc2e5eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5f5f40 con 0x55cc2e5dde40 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5f5870 con 0x55cc2e5df2d0 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e5f6610 con 0x55cc2e5de830 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 10 client.0 _ll_register_callbacks cb 0x7fe1140baf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5eace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56004/0 (socket says 172.21.15.5:56004) 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 -- 172.21.15.5:0/3484325988 learned_addr learned my addr 172.21.15.5:0/3484325988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe12573c640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe11ffff640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5de830 0x55cc2e5eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 -- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5de830 msgr2=0x55cc2e5eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5de830 0x55cc2e5eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 -- 172.21.15.5:0/3484325988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 msgr2=0x55cc2e5ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 -- 172.21.15.5:0/3484325988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2e5ee830 con 0x55cc2e5dde40 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe124f3b640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5eace0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe1140f0440 tx=0x7fe1140df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b02e626ca9de03a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe0fffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe0fffff640 1 -- 172.21.15.5:0/3484325988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140d7070 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe0fffff640 1 -- 172.21.15.5:0/3484325988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe1140035f0 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.328+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c0047a0 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe0fffff640 1 -- 172.21.15.5:0/3484325988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe1140a9040 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 10 client.15060 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc2e65acc0 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe0fffff640 1 -- 172.21.15.5:0/3484325988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe10c0047a0 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe0fffff640 1 -- 172.21.15.5:0/3484325988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe1140bb250 con 0x55cc2e5dde40 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 20 client.15060 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e65f9b0 0x55cc2e65e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 4 client.15060 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "status"}) -- 0x55cc2e54a3a0 con 0x55cc2e65f9b0 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe11ffff640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e65f9b0 0x55cc2e65e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe11ffff640 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e65f9b0 0x55cc2e65e6c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2979111d70c1b8de server_cookie=21ecb86270c8186c in_seq=0 out_seq=0 2024-04-22T13:26:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.329+0000 7fe0fffff640 10 client.15060 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe0fffff640 1 -- 172.21.15.5:0/3484325988 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cc2e54a3a0 con 0x55cc2e65f9b0 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe0fffff640 10 client.15060 handle_command_reply: tid=0 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 client.15060 shutdown 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 msgr2=0x55cc2e5eace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5eace0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe1140f0440 tx=0x7fe1140df040 comp rx=0 tx=0).stop 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 shutdown_connections 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc2e5df2d0 0x55cc2e5ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc2e5de830 0x55cc2e5eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55cc2e65f9b0 0x55cc2e65e6c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 --2- 172.21.15.5:0/3484325988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc2e5dde40 0x55cc2e5eace0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe0fffff640 0 client.15060 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 >> 172.21.15.5:0/3484325988 conn(0x55cc2e5d9f40 msgr2=0x55cc2e5e59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 shutdown_connections 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 1 -- 172.21.15.5:0/3484325988 wait complete. 2024-04-22T13:26:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe129d8b740 20 client.15060 trim_cache size 0 max 16384 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24469, 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 79.393168138999997, 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:26:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 104.57723636999999 2024-04-22T13:26:36.334 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:26:36.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1100a3cc0 msgr2=0x7fe1100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.330+0000 7fe12719f640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1100a3cc0 0x7fe1100a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe11801d000 tx=0x7fe118017000 comp rx=0 tx=0).stop 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 msgr2=0x7fe12019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe12019a4b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe114036040 tx=0x7fe1140042f0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 shutdown_connections 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe1100a3cc0 0x7fe1100a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1201029a0 0x7fe1201a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120101fe0 0x7fe12019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 --2- 172.21.15.5:0/1934053782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe120100e90 0x7fe12019a4b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 >> 172.21.15.5:0/1934053782 conn(0x7fe1200fc420 msgr2=0x7fe120069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 shutdown_connections 2024-04-22T13:26:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.331+0000 7fe12719f640 1 -- 172.21.15.5:0/1934053782 wait complete. 2024-04-22T13:26:36.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T13:26:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.443+0000 7fd84c146640 1 -- 172.21.15.5:0/2174985479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844101fa0 msgr2=0x7fd8441023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.443+0000 7fd84c146640 1 --2- 172.21.15.5:0/2174985479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844101fa0 0x7fd8441023c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd840009900 tx=0x7fd84002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.443+0000 7fd84c146640 1 -- 172.21.15.5:0/2174985479 shutdown_connections 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.443+0000 7fd84c146640 1 --2- 172.21.15.5:0/2174985479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103b50 0x7fd84410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.443+0000 7fd84c146640 1 --2- 172.21.15.5:0/2174985479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844103190 0x7fd844103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 --2- 172.21.15.5:0/2174985479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844101fa0 0x7fd8441023c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- 172.21.15.5:0/2174985479 >> 172.21.15.5:0/2174985479 conn(0x7fd8440fd530 msgr2=0x7fd8440ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- 172.21.15.5:0/2174985479 shutdown_connections 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- 172.21.15.5:0/2174985479 wait complete. 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 Processor -- start 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- start start 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844101fa0 0x7fd8441982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103190 0x7fd8441987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 0x7fd84419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd84410dae0 con 0x7fd844103b50 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd84410d960 con 0x7fd844103190 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84c146640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd84410dc60 con 0x7fd844101fa0 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84a6e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 0x7fd84419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84a6e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 0x7fd84419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56030/0 (socket says 172.21.15.5:56030) 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.444+0000 7fd84a6e3640 1 -- 172.21.15.5:0/2304809283 learned_addr learned my addr 172.21.15.5:0/2304809283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd849ee2640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844101fa0 0x7fd8441982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd8496e1640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103190 0x7fd8441987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84a6e3640 1 -- 172.21.15.5:0/2304809283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844101fa0 msgr2=0x7fd8441982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84a6e3640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844101fa0 0x7fd8441982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84a6e3640 1 -- 172.21.15.5:0/2304809283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103190 msgr2=0x7fd8441987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84a6e3640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103190 0x7fd8441987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84a6e3640 1 -- 172.21.15.5:0/2304809283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84419fe80 con 0x7fd844103b50 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd8496e1640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103190 0x7fd8441987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd849ee2640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844101fa0 0x7fd8441982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84a6e3640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 0x7fd84419f810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009fd0 tx=0x7fd83c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd83affd640 1 -- 172.21.15.5:0/2304809283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c017070 con 0x7fd844103b50 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd83affd640 1 -- 172.21.15.5:0/2304809283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c003e20 con 0x7fd844103b50 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd83affd640 1 -- 172.21.15.5:0/2304809283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c010460 con 0x7fd844103b50 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8441a0170 con 0x7fd844103b50 2024-04-22T13:26:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.445+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441a0670 con 0x7fd844103b50 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.447+0000 7fd83affd640 1 -- 172.21.15.5:0/2304809283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd83c003970 con 0x7fd844103b50 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 Processor -- start 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 -- start start 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 0x55d0091ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091efc40 0x55d0091f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0091f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d00914fb10 con 0x55d0091ef250 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d0090ed240 con 0x55d0091f06e0 2024-04-22T13:26:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d00914bad0 con 0x55d0091efc40 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd83affd640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd8300a3c40 0x7fd8300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd8496e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 0x55d0091ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd8496e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 0x55d0091ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56032/0 (socket says 172.21.15.5:56032) 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd8496e1640 1 -- 172.21.15.5:0/2761317776 learned_addr learned my addr 172.21.15.5:0/2761317776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd84a6e3640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd8300a3c40 0x7fd8300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd849ee2640 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091efc40 0x55d0091f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd83affd640 1 -- 172.21.15.5:0/2304809283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd83c0d03f0 con 0x7fd844103b50 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd8496e1640 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0091f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd8496e1640 1 -- 172.21.15.5:0/2761317776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091efc40 msgr2=0x55d0091f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.448+0000 7fd8496e1640 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091efc40 0x55d0091f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd8496e1640 1 -- 172.21.15.5:0/2761317776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 msgr2=0x55d0091f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd8496e1640 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0091f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd8496e1640 1 -- 172.21.15.5:0/2761317776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d008e622d0 con 0x55d0091ef250 2024-04-22T13:26:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd8496e1640 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 0x55d0091ef670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8340058c0 tx=0x7fd83400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46d336768e1469f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd84a6e3640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd8300a3c40 0x7fd8300a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0989e0 tx=0x7fd83c087040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd82ffff640 1 -- 172.21.15.5:0/2761317776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834019030 con 0x55d0091ef250 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd82ffff640 1 -- 172.21.15.5:0/2761317776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8340119f0 con 0x55d0091ef250 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.449+0000 7fd82ffff640 1 -- 172.21.15.5:0/2761317776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83401d8c0 con 0x55d0091ef250 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- 172.21.15.5:0/2761317776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 msgr2=0x55d0091ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 0x55d0091ef670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8340058c0 tx=0x7fd83400b730 comp rx=0 tx=0).stop 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- 172.21.15.5:0/2761317776 shutdown_connections 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0091f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091efc40 0x55d0091f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/2761317776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091ef250 0x55d0091ef670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- 172.21.15.5:0/2761317776 >> 172.21.15.5:0/2761317776 conn(0x55d0091e6f40 msgr2=0x55d0091e5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- 172.21.15.5:0/2761317776 shutdown_connections 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- 172.21.15.5:0/2761317776 wait complete. 2024-04-22T13:26:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- start start 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 Processor -- start 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 0x55d0091f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0092007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0091f78c0 con 0x55d0091efc40 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d0090d8190 con 0x55d0091f06e0 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.450+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d0091f7a40 con 0x55d0091ef250 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 0x55d0091f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 0x55d0091f9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56034/0 (socket says 172.21.15.5:56034) 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd84ee0b740 10 client.0 _ll_register_callbacks cb 0x55d0090d38b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 -- 172.21.15.5:0/609285482 learned_addr learned my addr 172.21.15.5:0/609285482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd8496e1640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd8496e1640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0092007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 -- 172.21.15.5:0/609285482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 msgr2=0x55d0091f92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 -- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 msgr2=0x55d0092007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0092007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 -- 172.21.15.5:0/609285482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d008f2c6b0 con 0x55d0091efc40 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd849ee2640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 0x55d0091f9810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd840005940 tx=0x7fd840004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13de243e6aef6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd82d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd82d7fa640 1 -- 172.21.15.5:0/609285482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840005230 con 0x55d0091efc40 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8340119f0 con 0x55d0091efc40 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd82d7fa640 1 -- 172.21.15.5:0/609285482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd8400053d0 con 0x55d0091efc40 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd82d7fa640 1 -- 172.21.15.5:0/609285482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd8400055d0 con 0x55d0091efc40 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd84ee0b740 10 client.15072 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.451+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d009203bd0 con 0x55d0091efc40 2024-04-22T13:26:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd82d7fa640 1 -- 172.21.15.5:0/609285482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8340119f0 con 0x55d0091efc40 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd82d7fa640 1 -- 172.21.15.5:0/609285482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd840003960 con 0x55d0091efc40 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd84ee0b740 10 client.15072 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd84ee0b740 10 client.15072 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd84ee0b740 20 client.15072 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d0092062b0 0x55d009207720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd84ee0b740 4 client.15072 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d00915d530 con 0x55d0092062b0 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.452+0000 7fd8496e1640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d0092062b0 0x55d009207720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.453+0000 7fd8496e1640 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d0092062b0 0x55d009207720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8547dc8f4ca54297 server_cookie=e05db48fce7f5842 in_seq=0 out_seq=0 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.453+0000 7fd82d7fa640 10 client.15072 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd82d7fa640 1 -- 172.21.15.5:0/609285482 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d00915d530 con 0x55d0092062b0 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd82d7fa640 10 client.15072 handle_command_reply: tid=0 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 client.15072 shutdown 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 msgr2=0x55d0091f9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 0x55d0091f9810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd840005940 tx=0x7fd840004270 comp rx=0 tx=0).stop 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 shutdown_connections 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d0092062b0 0x55d009207720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091efc40 0x55d0091f9810 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091f06e0 0x55d0092007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/609285482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd82d7fa640 0 client.15072 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 >> 172.21.15.5:0/609285482 conn(0x55d0091e6f40 msgr2=0x55d0091e4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 shutdown_connections 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 1 -- 172.21.15.5:0/609285482 wait complete. 2024-04-22T13:26:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.454+0000 7fd84ee0b740 20 client.15072 trim_cache size 0 max 16384 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 Processor -- start 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 -- start start 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0092007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d009205a60 con 0x55d0091f06e0 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d009206130 con 0x55d0091efc40 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d009206800 con 0x55d0091ef250 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0092007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0092007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56036/0 (socket says 172.21.15.5:56036) 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 -- 172.21.15.5:0/241585326 learned_addr learned my addr 172.21.15.5:0/241585326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd849ee2640 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 -- 172.21.15.5:0/241585326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 msgr2=0x55d0091f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 -- 172.21.15.5:0/241585326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 msgr2=0x55d0091f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 -- 172.21.15.5:0/241585326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0091e0e70 con 0x55d0091f06e0 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd849ee2640 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd8496e1640 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0092007a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd834013710 tx=0x7fd83400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64954502f5782b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd82f7fe640 1 -- 172.21.15.5:0/241585326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834034070 con 0x55d0091f06e0 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd82f7fe640 1 -- 172.21.15.5:0/241585326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83402f930 con 0x55d0091f06e0 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.473+0000 7fd82f7fe640 1 -- 172.21.15.5:0/241585326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834019040 con 0x55d0091f06e0 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 -- 172.21.15.5:0/241585326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 msgr2=0x55d0092007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0092007a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd834013710 tx=0x7fd83400b730 comp rx=0 tx=0).stop 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 -- 172.21.15.5:0/241585326 shutdown_connections 2024-04-22T13:26:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0092007a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/241585326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 -- 172.21.15.5:0/241585326 >> 172.21.15.5:0/241585326 conn(0x55d0091e6f40 msgr2=0x55d0091e5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 -- 172.21.15.5:0/241585326 shutdown_connections 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 -- 172.21.15.5:0/241585326 wait complete. 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 -- start start 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.474+0000 7fd84ee0b740 1 Processor -- start 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0091ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0092075a0 con 0x55d0091f06e0 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d009206ed0 con 0x55d0091efc40 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d009207c70 con 0x55d0091ef250 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 10 client.0 _ll_register_callbacks cb 0x7fd840009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0091ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0091ff210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56052/0 (socket says 172.21.15.5:56052) 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 -- 172.21.15.5:0/3293743680 learned_addr learned my addr 172.21.15.5:0/3293743680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 -- 172.21.15.5:0/3293743680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 msgr2=0x55d0091fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 -- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 msgr2=0x55d0091fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd849ee2640 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091fecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 -- 172.21.15.5:0/3293743680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0091ff950 con 0x55d0091f06e0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd8496e1640 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0091ff210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd834013f60 tx=0x7fd83400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8ba54853f5b8b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd82d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd82d7fa640 1 -- 172.21.15.5:0/3293743680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834034020 con 0x55d0091f06e0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd82d7fa640 1 -- 172.21.15.5:0/3293743680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd8340118f0 con 0x55d0091f06e0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.475+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83402f930 con 0x55d0091f06e0 2024-04-22T13:26:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd82d7fa640 1 -- 172.21.15.5:0/3293743680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd834019070 con 0x55d0091f06e0 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 10 client.15078 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d00926c260 con 0x55d0091f06e0 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd82d7fa640 1 -- 172.21.15.5:0/3293743680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83402f930 con 0x55d0091f06e0 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd82d7fa640 1 -- 172.21.15.5:0/3293743680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd83402b030 con 0x55d0091f06e0 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 10 client.15078 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 20 client.15078 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d009270570 0x55d00926f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 4 client.15078 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "status"}) -- 0x55d0091573a0 con 0x55d009270570 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.476+0000 7fd849ee2640 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d009270570 0x55d00926f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd82d7fa640 10 client.15078 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd849ee2640 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d009270570 0x55d00926f280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98e1ee8b4e62bf30 server_cookie=44cd6aae00f92d78 in_seq=0 out_seq=0 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd82d7fa640 1 -- 172.21.15.5:0/3293743680 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d0091573a0 con 0x55d009270570 2024-04-22T13:26:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd82d7fa640 10 client.15078 handle_command_reply: tid=0 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 client.15078 shutdown 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 msgr2=0x55d0091ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0091ff210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd834013f60 tx=0x7fd83400bd60 comp rx=0 tx=0).stop 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 shutdown_connections 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55d009270570 0x55d00926f280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0091f06e0 0x55d0091ff210 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0091efc40 0x55d0091fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 --2- 172.21.15.5:0/3293743680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0091ef250 0x55d0091fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd82d7fa640 0 client.15078 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 >> 172.21.15.5:0/3293743680 conn(0x55d0091e6f40 msgr2=0x55d0091f6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 shutdown_connections 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 1 -- 172.21.15.5:0/3293743680 wait complete. 2024-04-22T13:26:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.477+0000 7fd84ee0b740 20 client.15078 trim_cache size 0 max 16384 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24485, 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 80.550410798000001, 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 110.058085326 2024-04-22T13:26:36.481 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:26:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd8300a3c40 msgr2=0x7fd8300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd8300a3c40 0x7fd8300a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0989e0 tx=0x7fd83c087040 comp rx=0 tx=0).stop 2024-04-22T13:26:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 msgr2=0x7fd84419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 0x7fd84419f810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009fd0 tx=0x7fd83c00b260 comp rx=0 tx=0).stop 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 shutdown_connections 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd8300a3c40 0x7fd8300a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844103b50 0x7fd84419f810 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd844103190 0x7fd8441987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 --2- 172.21.15.5:0/2304809283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd844101fa0 0x7fd8441982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 >> 172.21.15.5:0/2304809283 conn(0x7fd8440fd530 msgr2=0x7fd844106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 shutdown_connections 2024-04-22T13:26:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.478+0000 7fd84c146640 1 -- 172.21.15.5:0/2304809283 wait complete. 2024-04-22T13:26:36.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T13:26:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 -- 172.21.15.5:0/1194275888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8100e90 msgr2=0x7fe3d81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 --2- 172.21.15.5:0/1194275888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8100e90 0x7fe3d81012b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 -- 172.21.15.5:0/1194275888 shutdown_connections 2024-04-22T13:26:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 --2- 172.21.15.5:0/1194275888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d81029a0 0x7fe3d81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 --2- 172.21.15.5:0/1194275888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 --2- 172.21.15.5:0/1194275888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8100e90 0x7fe3d81012b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 -- 172.21.15.5:0/1194275888 >> 172.21.15.5:0/1194275888 conn(0x7fe3d80fc420 msgr2=0x7fe3d80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 -- 172.21.15.5:0/1194275888 shutdown_connections 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 -- 172.21.15.5:0/1194275888 wait complete. 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.592+0000 7fe3dce11640 1 Processor -- start 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 -- start start 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d8100e90 0x7fe3d8104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d81029a0 0x7fe3d8103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d810c9e0 con 0x7fe3d8101fe0 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d810c860 con 0x7fe3d81029a0 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3dce11640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d810cb60 con 0x7fe3d8100e90 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56078/0 (socket says 172.21.15.5:56078) 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 -- 172.21.15.5:0/3453942884 learned_addr learned my addr 172.21.15.5:0/3453942884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d659c640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d8100e90 0x7fe3d8104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d81029a0 0x7fe3d8103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 -- 172.21.15.5:0/3453942884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d8100e90 msgr2=0x7fe3d8104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d8100e90 0x7fe3d8104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 -- 172.21.15.5:0/3453942884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d81029a0 msgr2=0x7fe3d8103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d81029a0 0x7fe3d8103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 -- 172.21.15.5:0/3453942884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d8104220 con 0x7fe3d8101fe0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d659c640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d8100e90 0x7fe3d8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.593+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8103450 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000d950 tx=0x7fe3c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.594+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3453942884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0017020 con 0x7fe3d8101fe0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.594+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3453942884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c0009c30 con 0x7fe3d8101fe0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.594+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d81a2350 con 0x7fe3d8101fe0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.594+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3453942884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000f6b0 con 0x7fe3d8101fe0 2024-04-22T13:26:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.594+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a28a0 con 0x7fe3d8101fe0 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.596+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3453942884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fe3c001a020 con 0x7fe3d8101fe0 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 Processor -- start 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 -- start start 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c16a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c16acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c171eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e74c0ceb10 con 0x55e74c169e40 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e74c06c240 con 0x55e74c16b2d0 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e74c0caad0 con 0x55e74c16a830 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c16a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c16a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56080/0 (socket says 172.21.15.5:56080) 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/3028247084 learned_addr learned my addr 172.21.15.5:0/3028247084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3cf7fe640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe3c40a3c40 0x7fe3c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d659c640 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c16acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c171eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/3028247084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 msgr2=0x55e74c16acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c16acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/3028247084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 msgr2=0x55e74c171eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c171eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/3028247084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74bde12d0 con 0x55e74c169e40 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c171eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3453942884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe3c00d02f0 con 0x7fe3d8101fe0 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.597+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe3c40a3c40 0x7fe3c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.598+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c16a260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8011590 tx=0x7fe3c8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbd65f5c3f03185 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.598+0000 7fe3ccff9640 1 -- 172.21.15.5:0/3028247084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8017050 con 0x55e74c169e40 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.598+0000 7fe3ccff9640 1 -- 172.21.15.5:0/3028247084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c801bb90 con 0x55e74c169e40 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.598+0000 7fe3ccff9640 1 -- 172.21.15.5:0/3028247084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8025d80 con 0x55e74c169e40 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- 172.21.15.5:0/3028247084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 msgr2=0x55e74c16a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe3c40a3c40 0x7fe3c40a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8022000 tx=0x7fe3c8023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c16a260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8011590 tx=0x7fe3c8011a50 comp rx=0 tx=0).stop 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- 172.21.15.5:0/3028247084 shutdown_connections 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c171eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c16acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/3028247084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c16a260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- 172.21.15.5:0/3028247084 >> 172.21.15.5:0/3028247084 conn(0x55e74c165f40 msgr2=0x55e74c164490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- 172.21.15.5:0/3028247084 shutdown_connections 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- 172.21.15.5:0/3028247084 wait complete. 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- start start 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 Processor -- start 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c173e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c1743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c17b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e74c1724b0 con 0x55e74c16b2d0 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e74c057190 con 0x55e74c169e40 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e74c172630 con 0x55e74c16a830 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3dfa00740 10 client.0 _ll_register_callbacks cb 0x55e74c0528b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c17b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c17b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56084/0 (socket says 172.21.15.5:56084) 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.599+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/650156258 learned_addr learned my addr 172.21.15.5:0/650156258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c1743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d659c640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 msgr2=0x55e74c1743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c1743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/650156258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 msgr2=0x55e74c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/650156258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74beab6b0 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c1743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d659c640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c173e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c17b410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800f600 tx=0x7fe3c800f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243dc5c216273a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3bdffb640 1 -- 172.21.15.5:0/650156258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801b990 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c801bb90 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3bdffb640 1 -- 172.21.15.5:0/650156258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe3c8025640 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3bdffb640 1 -- 172.21.15.5:0/650156258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3c80049c0 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3dfa00740 10 client.15090 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e74c17e760 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.600+0000 7fe3bdffb640 1 -- 172.21.15.5:0/650156258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c801bb90 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3bdffb640 1 -- 172.21.15.5:0/650156258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3c8005590 con 0x55e74c16b2d0 2024-04-22T13:26:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3dfa00740 10 client.15090 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3dfa00740 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3dfa00740 20 client.15090 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c181ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3dfa00740 4 client.15090 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e74c0dc530 con 0x55e74c180a30 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3d659c640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c181ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3d659c640 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c181ea0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=591a4fb5b8668e9b server_cookie=29a907058974861f in_seq=0 out_seq=0 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.601+0000 7fe3bdffb640 10 client.15090 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3bdffb640 1 -- 172.21.15.5:0/650156258 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e74c0dc530 con 0x55e74c180a30 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3bdffb640 10 client.15090 handle_command_reply: tid=0 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 client.15090 shutdown 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 msgr2=0x55e74c17b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c17b410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800f600 tx=0x7fe3c800f3e0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 shutdown_connections 2024-04-22T13:26:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c181ea0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c17b410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c1743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3bdffb640 0 client.15090 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/650156258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c173e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 >> 172.21.15.5:0/650156258 conn(0x55e74c165f40 msgr2=0x55e74c163a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 shutdown_connections 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.602+0000 7fe3dfa00740 1 -- 172.21.15.5:0/650156258 wait complete. 2024-04-22T13:26:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.603+0000 7fe3dfa00740 20 client.15090 trim_cache size 0 max 16384 2024-04-22T13:26:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 Processor -- start 2024-04-22T13:26:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 -- start start 2024-04-22T13:26:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c15f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c17b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c16dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e74c1801e0 con 0x55e74c16b2d0 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e74c1808b0 con 0x55e74c169e40 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e74c180f80 con 0x55e74c16a830 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c16dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c16dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56096/0 (socket says 172.21.15.5:56096) 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/2105259742 learned_addr learned my addr 172.21.15.5:0/2105259742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c17b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/2105259742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 msgr2=0x55e74c17b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c17b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/2105259742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 msgr2=0x55e74c15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c15f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 -- 172.21.15.5:0/2105259742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74c174d30 con 0x55e74c16b2d0 2024-04-22T13:26:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c16dc60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800f470 tx=0x7fe3c8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a467be271c4da21a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3bffff640 1 -- 172.21.15.5:0/2105259742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8017070 con 0x55e74c16b2d0 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3bffff640 1 -- 172.21.15.5:0/2105259742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8032070 con 0x55e74c16b2d0 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.621+0000 7fe3bffff640 1 -- 172.21.15.5:0/2105259742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800e910 con 0x55e74c16b2d0 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2105259742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 msgr2=0x55e74c16dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c16dc60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800f470 tx=0x7fe3c8009f80 comp rx=0 tx=0).stop 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2105259742 shutdown_connections 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c16b2d0 0x55e74c16dc60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c17b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2105259742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c169e40 0x55e74c15f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2105259742 >> 172.21.15.5:0/2105259742 conn(0x55e74c165f40 msgr2=0x55e74c1729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2105259742 shutdown_connections 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2105259742 wait complete. 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 -- start start 2024-04-22T13:26:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.622+0000 7fe3dfa00740 1 Processor -- start 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c1773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c179900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c179e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e74c181d20 con 0x55e74c169e40 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e74c181650 con 0x55e74c16b2d0 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e74c1823f0 con 0x55e74c16a830 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 10 client.0 _ll_register_callbacks cb 0x7fe3c8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c1773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c1773b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56110/0 (socket says 172.21.15.5:56110) 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 -- 172.21.15.5:0/2987851040 learned_addr learned my addr 172.21.15.5:0/2987851040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c179900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 -- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 msgr2=0x55e74c179900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d6d9d640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c179e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c179900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 -- 172.21.15.5:0/2987851040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 msgr2=0x55e74c179e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c179e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 -- 172.21.15.5:0/2987851040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74c17a480 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3d659c640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c1773b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005ab0 tx=0x7fe3d002cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96437eb3795ae010 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3bdffb640 1 -- 172.21.15.5:0/2987851040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d004c020 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3bdffb640 1 -- 172.21.15.5:0/2987851040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe3d0004250 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.623+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c8032070 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3bdffb640 1 -- 172.21.15.5:0/2987851040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3d0004450 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 10 client.15096 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e74c1e69e0 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3bdffb640 1 -- 172.21.15.5:0/2987851040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8032070 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3bdffb640 1 -- 172.21.15.5:0/2987851040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3d0047070 con 0x55e74c169e40 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:26:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 20 client.15096 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c1ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 4 client.15096 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "status"}) -- 0x55e74c0d63a0 con 0x55e74c180a30 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.624+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c1ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3bdffb640 10 client.15096 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3d5d9b640 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c1ea230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2262a16a691c8d35 server_cookie=761f5fadb30973b5 in_seq=0 out_seq=0 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3bdffb640 1 -- 172.21.15.5:0/2987851040 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e74c0d63a0 con 0x55e74c180a30 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3bdffb640 10 client.15096 handle_command_reply: tid=0 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 client.15096 shutdown 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 msgr2=0x55e74c1773b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c1773b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005ab0 tx=0x7fe3d002cb40 comp rx=0 tx=0).stop 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 shutdown_connections 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55e74c180a30 0x55e74c1ea230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74c16b2d0 0x55e74c179e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3bdffb640 0 client.15096 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e74c16a830 0x55e74c179900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 --2- 172.21.15.5:0/2987851040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74c169e40 0x55e74c1773b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 >> 172.21.15.5:0/2987851040 conn(0x55e74c165f40 msgr2=0x55e74c171a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 shutdown_connections 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 1 -- 172.21.15.5:0/2987851040 wait complete. 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dfa00740 20 client.15096 trim_cache size 0 max 16384 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24496, 2024-04-22T13:26:36.628 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 81.705102198999995, 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 99.402157686999999 2024-04-22T13:26:36.629 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe3c40a3c40 msgr2=0x7fe3c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.625+0000 7fe3dce11640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe3c40a3c40 0x7fe3c40a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8022000 tx=0x7fe3c8023000 comp rx=0 tx=0).stop 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 msgr2=0x7fe3d8103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8103450 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000d950 tx=0x7fe3c000de10 comp rx=0 tx=0).stop 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 shutdown_connections 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fe3c40a3c40 0x7fe3c40a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d8101fe0 0x7fe3d8103450 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3d81029a0 0x7fe3d8103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 --2- 172.21.15.5:0/3453942884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d8100e90 0x7fe3d8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 >> 172.21.15.5:0/3453942884 conn(0x7fe3d80fc420 msgr2=0x7fe3d8069300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 shutdown_connections 2024-04-22T13:26:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.626+0000 7fe3dce11640 1 -- 172.21.15.5:0/3453942884 wait complete. 2024-04-22T13:26:36.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 -- 172.21.15.5:0/1493594169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed841053a0 msgr2=0x7fed84107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 --2- 172.21.15.5:0/1493594169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed841053a0 0x7fed84107790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fed70009900 tx=0x7fed7002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 -- 172.21.15.5:0/1493594169 shutdown_connections 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 --2- 172.21.15.5:0/1493594169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed841053a0 0x7fed84107790 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 --2- 172.21.15.5:0/1493594169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed84107cd0 0x7fed8410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 --2- 172.21.15.5:0/1493594169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84102a70 0x7fed84104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 -- 172.21.15.5:0/1493594169 >> 172.21.15.5:0/1493594169 conn(0x7fed840fc420 msgr2=0x7fed840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 -- 172.21.15.5:0/1493594169 shutdown_connections 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.739+0000 7fed8ab23640 1 -- 172.21.15.5:0/1493594169 wait complete. 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 Processor -- start 2024-04-22T13:26:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 -- start start 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 0x7fed84198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed841053a0 0x7fed84198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84107cd0 0x7fed8419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed8410dac0 con 0x7fed84102a70 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed8410d940 con 0x7fed841053a0 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed8ab23640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed8410dc40 con 0x7fed84107cd0 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 0x7fed84198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 0x7fed84198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56132/0 (socket says 172.21.15.5:56132) 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 -- 172.21.15.5:0/2209872232 learned_addr learned my addr 172.21.15.5:0/2209872232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed83fff640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed841053a0 0x7fed84198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed890c0640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84107cd0 0x7fed8419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 -- 172.21.15.5:0/2209872232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84107cd0 msgr2=0x7fed8419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84107cd0 0x7fed8419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 -- 172.21.15.5:0/2209872232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed841053a0 msgr2=0x7fed84198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed841053a0 0x7fed84198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed888bf640 1 -- 172.21.15.5:0/2209872232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8419fe40 con 0x7fed84102a70 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.740+0000 7fed890c0640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84107cd0 0x7fed8419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.741+0000 7fed888bf640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 0x7fed84198220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c970 tx=0x7fed7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.741+0000 7fed81ffb640 1 -- 172.21.15.5:0/2209872232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78007bb0 con 0x7fed84102a70 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.741+0000 7fed81ffb640 1 -- 172.21.15.5:0/2209872232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed78007d50 con 0x7fed84102a70 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.741+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed841a0130 con 0x7fed84102a70 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.741+0000 7fed81ffb640 1 -- 172.21.15.5:0/2209872232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7800f450 con 0x7fed84102a70 2024-04-22T13:26:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.741+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed841a0600 con 0x7fed84102a70 2024-04-22T13:26:36.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.743+0000 7fed81ffb640 1 -- 172.21.15.5:0/2209872232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fed78004f20 con 0x7fed84102a70 2024-04-22T13:26:36.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.743+0000 7fed8d7eb740 1 Processor -- start 2024-04-22T13:26:36.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.743+0000 7fed8d7eb740 1 -- start start 2024-04-22T13:26:36.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.743+0000 7fed81ffb640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed740a3c40 0x7fed740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5efa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5e59b10 con 0x55a0a5ef9250 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5df7240 con 0x55a0a5efa6e0 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5e55ad0 con 0x55a0a5ef9c40 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed890c0640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed740a3c40 0x7fed740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56134/0 (socket says 172.21.15.5:56134) 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 -- 172.21.15.5:0/1768862885 learned_addr learned my addr 172.21.15.5:0/1768862885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed81ffb640 1 -- 172.21.15.5:0/2209872232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fed780d02f0 con 0x7fed84102a70 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed890c0640 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed83fff640 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5efa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 -- 172.21.15.5:0/1768862885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 msgr2=0x55a0a5efa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5efa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 -- 172.21.15.5:0/1768862885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 msgr2=0x55a0a5f012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 -- 172.21.15.5:0/1768862885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a5b6c2d0 con 0x55a0a5ef9250 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed890c0640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed740a3c40 0x7fed740a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009270 tx=0x7fed7c013a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed888bf640 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef9670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed780a4bc0 tx=0x7fed78008ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acecc3232454ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed6effd640 1 -- 172.21.15.5:0/1768862885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780641c0 con 0x55a0a5ef9250 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed6effd640 1 -- 172.21.15.5:0/1768862885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed780055f0 con 0x55a0a5ef9250 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.744+0000 7fed6effd640 1 -- 172.21.15.5:0/1768862885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7806e1c0 con 0x55a0a5ef9250 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 -- 172.21.15.5:0/1768862885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 msgr2=0x55a0a5ef9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef9670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed780a4bc0 tx=0x7fed78008ea0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 -- 172.21.15.5:0/1768862885 shutdown_connections 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5efa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/1768862885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef9670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 -- 172.21.15.5:0/1768862885 >> 172.21.15.5:0/1768862885 conn(0x55a0a5ef0f40 msgr2=0x55a0a5eef490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 -- 172.21.15.5:0/1768862885 shutdown_connections 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 -- 172.21.15.5:0/1768862885 wait complete. 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 -- start start 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.745+0000 7fed8d7eb740 1 Processor -- start 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5f03330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f03870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f018c0 con 0x55a0a5ef9c40 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5de2190 con 0x55a0a5efa6e0 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f01a40 con 0x55a0a5ef9250 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed8d7eb740 10 client.0 _ll_register_callbacks cb 0x55a0a5ddd8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f03870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f03870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56150/0 (socket says 172.21.15.5:56150) 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 -- 172.21.15.5:0/2156219345 learned_addr learned my addr 172.21.15.5:0/2156219345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed888bf640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5f03330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed890c0640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 -- 172.21.15.5:0/2156219345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 msgr2=0x55a0a5f03330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5f03330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 -- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 msgr2=0x55a0a5f0a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f0a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 -- 172.21.15.5:0/2156219345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a5c366b0 con 0x55a0a5ef9c40 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed83fff640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f03870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed84069d50 tx=0x7fed70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7475f59e0705d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.746+0000 7fed63fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed63fff640 1 -- 172.21.15.5:0/2156219345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700044a0 con 0x55a0a5ef9c40 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed63fff640 1 -- 172.21.15.5:0/2156219345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fed70005250 con 0x55a0a5ef9c40 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0a5e67530 con 0x55a0a5ef9c40 2024-04-22T13:26:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed63fff640 1 -- 172.21.15.5:0/2156219345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed70005450 con 0x55a0a5ef9c40 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0a5f0dc20 con 0x55a0a5ef9c40 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed63fff640 1 -- 172.21.15.5:0/2156219345 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a0a5e67530 con 0x55a0a5ef9c40 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed63fff640 1 -- 172.21.15.5:0/2156219345 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed70003850 con 0x55a0a5ef9c40 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 20 client.15108 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f102b0 0x55a0a5f11720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 4 client.15108 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a5e613a0 con 0x55a0a5f102b0 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.747+0000 7fed888bf640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f102b0 0x55a0a5f11720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.748+0000 7fed888bf640 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f102b0 0x55a0a5f11720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32047d5e41744f6c server_cookie=42f409f7bba5d351 in_seq=0 out_seq=0 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.748+0000 7fed63fff640 10 client.15108 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed63fff640 1 -- 172.21.15.5:0/2156219345 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0a5e613a0 con 0x55a0a5f102b0 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed63fff640 10 client.15108 handle_command_reply: tid=0 2024-04-22T13:26:36.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 client.15108 shutdown 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 msgr2=0x55a0a5f03870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f03870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed84069d50 tx=0x7fed70004270 comp rx=0 tx=0).stop 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 shutdown_connections 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f102b0 0x55a0a5f11720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f03870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5f0a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed63fff640 0 client.15108 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2156219345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5f03330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 >> 172.21.15.5:0/2156219345 conn(0x55a0a5ef0f40 msgr2=0x55a0a5eeea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 shutdown_connections 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2156219345 wait complete. 2024-04-22T13:26:36.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.749+0000 7fed8d7eb740 20 client.15108 trim_cache size 0 max 16384 2024-04-22T13:26:36.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 Processor -- start 2024-04-22T13:26:36.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 -- start start 2024-04-22T13:26:36.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5efdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f0a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5efe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f0fa60 con 0x55a0a5ef9c40 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f10130 con 0x55a0a5efa6e0 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f10800 con 0x55a0a5ef9250 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.767+0000 7fed83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f0a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f0a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56166/0 (socket says 172.21.15.5:56166) 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 -- 172.21.15.5:0/3366843493 learned_addr learned my addr 172.21.15.5:0/3366843493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed888bf640 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5efdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed890c0640 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5efe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 -- 172.21.15.5:0/3366843493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 msgr2=0x55a0a5efdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5efdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 -- 172.21.15.5:0/3366843493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 msgr2=0x55a0a5efe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5efe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 -- 172.21.15.5:0/3366843493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a5f00fb0 con 0x55a0a5ef9c40 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed890c0640 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5efe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed83fff640 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f0a800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed70009850 tx=0x7fed7002cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3cfe387fb91682 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed6e7fc640 1 -- 172.21.15.5:0/3366843493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70005d50 con 0x55a0a5ef9c40 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed6e7fc640 1 -- 172.21.15.5:0/3366843493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed700044a0 con 0x55a0a5ef9c40 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.768+0000 7fed6e7fc640 1 -- 172.21.15.5:0/3366843493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70004b20 con 0x55a0a5ef9c40 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- 172.21.15.5:0/3366843493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 msgr2=0x55a0a5f0a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f0a800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed70009850 tx=0x7fed7002cc80 comp rx=0 tx=0).stop 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- 172.21.15.5:0/3366843493 shutdown_connections 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5ef9c40 0x55a0a5f0a800 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5efa6e0 0x55a0a5efe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/3366843493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9250 0x55a0a5efdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- 172.21.15.5:0/3366843493 >> 172.21.15.5:0/3366843493 conn(0x55a0a5ef0f40 msgr2=0x55a0a5efd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- 172.21.15.5:0/3366843493 shutdown_connections 2024-04-22T13:26:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- 172.21.15.5:0/3366843493 wait complete. 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- start start 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 Processor -- start 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5ef73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 0x55a0a5ef7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f115a0 con 0x55a0a5efa6e0 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f10ed0 con 0x55a0a5ef9250 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0a5f11c70 con 0x55a0a5ef9c40 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.769+0000 7fed8d7eb740 10 client.0 _ll_register_callbacks cb 0x7fed70009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 0x55a0a5ef7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 0x55a0a5ef7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56174/0 (socket says 172.21.15.5:56174) 2024-04-22T13:26:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 -- 172.21.15.5:0/2237034003 learned_addr learned my addr 172.21.15.5:0/2237034003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed888bf640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed83fff640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5ef73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 -- 172.21.15.5:0/2237034003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 msgr2=0x55a0a5ef73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5ef73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 -- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5ef9250 msgr2=0x55a0a5ef6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 -- 172.21.15.5:0/2237034003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a5ef81a0 con 0x55a0a5efa6e0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed890c0640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 0x55a0a5ef7970 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed7c013160 tx=0x7fed7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa0fc12e587eec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed63fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed63fff640 1 -- 172.21.15.5:0/2237034003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c022820 con 0x55a0a5efa6e0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed63fff640 1 -- 172.21.15.5:0/2237034003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fed7c0229c0 con 0x55a0a5efa6e0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed700044a0 con 0x55a0a5efa6e0 2024-04-22T13:26:36.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed63fff640 1 -- 172.21.15.5:0/2237034003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed7c022bc0 con 0x55a0a5efa6e0 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed8d7eb740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.770+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0a5f08b70 con 0x55a0a5efa6e0 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed63fff640 1 -- 172.21.15.5:0/2237034003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed700044a0 con 0x55a0a5efa6e0 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed63fff640 1 -- 172.21.15.5:0/2237034003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed7c01e070 con 0x55a0a5efa6e0 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed8d7eb740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed8d7eb740 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed8d7eb740 20 client.15114 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f7f4d0 0x55a0a5f7e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed8d7eb740 4 client.15114 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "status"}) -- 0x55a0a5ddd960 con 0x55a0a5f7f4d0 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed888bf640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f7f4d0 0x55a0a5f7e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed63fff640 10 client.15114 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:26:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.771+0000 7fed888bf640 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f7f4d0 0x55a0a5f7e1e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5375eeb9f9418bf9 server_cookie=1b81b147d13574ae in_seq=0 out_seq=0 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed63fff640 1 -- 172.21.15.5:0/2237034003 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a0a5ddd960 con 0x55a0a5f7f4d0 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed63fff640 10 client.15114 handle_command_reply: tid=0 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 client.15114 shutdown 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 msgr2=0x55a0a5ef7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 0x55a0a5ef7970 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed7c013160 tx=0x7fed7c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 shutdown_connections 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55a0a5f7f4d0 0x55a0a5f7e1e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0a5efa6e0 0x55a0a5ef7970 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0a5ef9c40 0x55a0a5ef73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed63fff640 0 client.15114 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 --2- 172.21.15.5:0/2237034003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0a5ef9250 0x55a0a5ef6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 >> 172.21.15.5:0/2237034003 conn(0x55a0a5ef0f40 msgr2=0x55a0a5eee170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 shutdown_connections 2024-04-22T13:26:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 1 -- 172.21.15.5:0/2237034003 wait complete. 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.772+0000 7fed8d7eb740 20 client.15114 trim_cache size 0 max 16384 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24512, 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 82.861477520999998, 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:26:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 102.17429072199999 2024-04-22T13:26:36.777 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:26:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed740a3c40 msgr2=0x7fed740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed740a3c40 0x7fed740a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009270 tx=0x7fed7c013a80 comp rx=0 tx=0).stop 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 msgr2=0x7fed84198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 0x7fed84198220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c970 tx=0x7fed7800ce30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 shutdown_connections 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed740a3c40 0x7fed740a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84107cd0 0x7fed8419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed841053a0 0x7fed84198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 --2- 172.21.15.5:0/2209872232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84102a70 0x7fed84198220 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 >> 172.21.15.5:0/2209872232 conn(0x7fed840fc420 msgr2=0x7fed841048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 shutdown_connections 2024-04-22T13:26:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.773+0000 7fed8ab23640 1 -- 172.21.15.5:0/2209872232 wait complete. 2024-04-22T13:26:36.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 -- 172.21.15.5:0/56995080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454105350 msgr2=0x7fd454107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 --2- 172.21.15.5:0/56995080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454105350 0x7fd454107740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd444009900 tx=0x7fd44402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 -- 172.21.15.5:0/56995080 shutdown_connections 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 --2- 172.21.15.5:0/56995080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454105350 0x7fd454107740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 --2- 172.21.15.5:0/56995080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 0x7fd45410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 --2- 172.21.15.5:0/56995080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd454102a20 0x7fd454104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 -- 172.21.15.5:0/56995080 >> 172.21.15.5:0/56995080 conn(0x7fd4540fc430 msgr2=0x7fd4540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 -- 172.21.15.5:0/56995080 shutdown_connections 2024-04-22T13:26:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.886+0000 7fd45b26d640 1 -- 172.21.15.5:0/56995080 wait complete. 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 Processor -- start 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 -- start start 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 0x7fd454198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd454105350 0x7fd454198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 0x7fd45419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd45410daa0 con 0x7fd454102a20 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd45410d920 con 0x7fd454107c80 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45b26d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd45410dc20 con 0x7fd454105350 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd459009640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 0x7fd454198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd459009640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 0x7fd454198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56188/0 (socket says 172.21.15.5:56188) 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd459009640 1 -- 172.21.15.5:0/543957108 learned_addr learned my addr 172.21.15.5:0/543957108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd458808640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd454105350 0x7fd454198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd45980a640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 0x7fd45419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd459009640 1 -- 172.21.15.5:0/543957108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd454105350 msgr2=0x7fd454198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.887+0000 7fd459009640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd454105350 0x7fd454198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd459009640 1 -- 172.21.15.5:0/543957108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 msgr2=0x7fd45419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd459009640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 0x7fd45419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd459009640 1 -- 172.21.15.5:0/543957108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45419fe40 con 0x7fd454102a20 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd45980a640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 0x7fd45419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd459009640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 0x7fd454198220 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd45000d950 tx=0x7fd45000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd44a7fc640 1 -- 172.21.15.5:0/543957108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450017030 con 0x7fd454102a20 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd44a7fc640 1 -- 172.21.15.5:0/543957108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd450004020 con 0x7fd454102a20 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4541a0130 con 0x7fd454102a20 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd44a7fc640 1 -- 172.21.15.5:0/543957108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4500047a0 con 0x7fd454102a20 2024-04-22T13:26:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.888+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4541a0650 con 0x7fd454102a20 2024-04-22T13:26:36.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.890+0000 7fd44a7fc640 1 -- 172.21.15.5:0/543957108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd450009a30 con 0x7fd454102a20 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.890+0000 7fd45de68740 1 Processor -- start 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.890+0000 7fd45de68740 1 -- start start 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45de68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d95b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45de68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d95c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45de68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d9632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45de68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a41d8bbb10 con 0x55a41d95c6e0 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45de68740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a41d859240 con 0x55a41d95b250 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45de68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a41d8b7ad0 con 0x55a41d95bc40 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d9632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d9632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56192/0 (socket says 172.21.15.5:56192) 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 -- 172.21.15.5:0/3550982041 learned_addr learned my addr 172.21.15.5:0/3550982041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd459009640 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d95c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd45980a640 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d95b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 -- 172.21.15.5:0/3550982041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 msgr2=0x55a41d95c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d95c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 -- 172.21.15.5:0/3550982041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 msgr2=0x55a41d95b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d95b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd44a7fc640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd4400a3c40 0x7fd4400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 -- 172.21.15.5:0/3550982041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a41d5ce2d0 con 0x55a41d95c6e0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd459009640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd4400a3c40 0x7fd4400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd458808640 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d9632c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd45406b180 tx=0x7fd444005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9cd6f8a5595a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd44a7fc640 1 -- 172.21.15.5:0/543957108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd4500cffa0 con 0x7fd454102a20 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.891+0000 7fd43f7fe640 1 -- 172.21.15.5:0/3550982041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444045020 con 0x55a41d95c6e0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd43f7fe640 1 -- 172.21.15.5:0/3550982041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44402ca20 con 0x55a41d95c6e0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd43f7fe640 1 -- 172.21.15.5:0/3550982041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4440046c0 con 0x55a41d95c6e0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 -- 172.21.15.5:0/3550982041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 msgr2=0x55a41d9632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd459009640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd4400a3c40 0x7fd4400a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd45009bed0 tx=0x7fd4500d3b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d9632c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd45406b180 tx=0x7fd444005c00 comp rx=0 tx=0).stop 2024-04-22T13:26:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 -- 172.21.15.5:0/3550982041 shutdown_connections 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d9632c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d95c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 --2- 172.21.15.5:0/3550982041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d95b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 -- 172.21.15.5:0/3550982041 >> 172.21.15.5:0/3550982041 conn(0x55a41d952f40 msgr2=0x55a41d951490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 -- 172.21.15.5:0/3550982041 shutdown_connections 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.892+0000 7fd45de68740 1 -- 172.21.15.5:0/3550982041 wait complete. 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 -- start start 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 Processor -- start 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 0x55a41d965200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d965740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a41d963860 con 0x55a41d95b250 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a41d844190 con 0x55a41d95c6e0 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a41d9639e0 con 0x55a41d95bc40 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45de68740 10 client.0 _ll_register_callbacks cb 0x55a41d83f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 0x55a41d965200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 0x55a41d965200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56204/0 (socket says 172.21.15.5:56204) 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 -- 172.21.15.5:0/1952802312 learned_addr learned my addr 172.21.15.5:0/1952802312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd458808640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d965740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 -- 172.21.15.5:0/1952802312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 msgr2=0x55a41d965740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d965740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 -- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 msgr2=0x55a41d96c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.893+0000 7fd45980a640 1 -- 172.21.15.5:0/1952802312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41d6986b0 con 0x55a41d95b250 2024-04-22T13:26:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45980a640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 0x55a41d965200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd44c013bc0 tx=0x7fd44c009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6e1b4805cd8eda server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd43cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd43cff9640 1 -- 172.21.15.5:0/1952802312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c004b20 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd43cff9640 1 -- 172.21.15.5:0/1952802312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd44c004cc0 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44402ca20 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd43cff9640 1 -- 172.21.15.5:0/1952802312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd44c003970 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 10 client.15126 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a41d96fb60 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd43cff9640 1 -- 172.21.15.5:0/1952802312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44402ca20 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd43cff9640 1 -- 172.21.15.5:0/1952802312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd44c01d060 con 0x55a41d95b250 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 20 client.15126 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d972240 0x55a41d9736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 4 client.15126 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.894+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a41d8c9530 con 0x55a41d972240 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.895+0000 7fd458808640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d972240 0x55a41d9736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.895+0000 7fd43cff9640 10 client.15126 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.895+0000 7fd458808640 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d972240 0x55a41d9736b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97043f66d3d83bf server_cookie=59436b784d7d1c7e in_seq=0 out_seq=0 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd43cff9640 1 -- 172.21.15.5:0/1952802312 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a41d8c9530 con 0x55a41d972240 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd43cff9640 10 client.15126 handle_command_reply: tid=0 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 client.15126 shutdown 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 msgr2=0x55a41d965200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 0x55a41d965200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd44c013bc0 tx=0x7fd44c009a00 comp rx=0 tx=0).stop 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 shutdown_connections 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d972240 0x55a41d9736b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d965740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 --2- 172.21.15.5:0/1952802312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95b250 0x55a41d965200 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd43cff9640 0 client.15126 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:26:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 >> 172.21.15.5:0/1952802312 conn(0x55a41d952f40 msgr2=0x55a41d950a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 shutdown_connections 2024-04-22T13:26:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 1 -- 172.21.15.5:0/1952802312 wait complete. 2024-04-22T13:26:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.896+0000 7fd45de68740 20 client.15126 trim_cache size 0 max 16384 2024-04-22T13:26:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.914+0000 7fd45de68740 1 Processor -- start 2024-04-22T13:26:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.914+0000 7fd45de68740 1 -- start start 2024-04-22T13:26:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.914+0000 7fd45de68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d951400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.914+0000 7fd45de68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d96c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45de68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d951940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45de68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a41d9719f0 con 0x55a41d95c6e0 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45de68740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a41d9720c0 con 0x55a41d95b250 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45de68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a41d972790 con 0x55a41d95bc40 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d951940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d951940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56210/0 (socket says 172.21.15.5:56210) 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 -- 172.21.15.5:0/1783071271 learned_addr learned my addr 172.21.15.5:0/1783071271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d951400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd458808640 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d96c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 -- 172.21.15.5:0/1783071271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 msgr2=0x55a41d96c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d96c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 -- 172.21.15.5:0/1783071271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 msgr2=0x55a41d951400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d951400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 -- 172.21.15.5:0/1783071271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a41d94ce70 con 0x55a41d95c6e0 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd45980a640 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d951940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd44c002230 tx=0x7fd44c009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63831566d804e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd43effd640 1 -- 172.21.15.5:0/1783071271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0130d0 con 0x55a41d95c6e0 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd43effd640 1 -- 172.21.15.5:0/1783071271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44c011630 con 0x55a41d95c6e0 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.915+0000 7fd43effd640 1 -- 172.21.15.5:0/1783071271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c005260 con 0x55a41d95c6e0 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- 172.21.15.5:0/1783071271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 msgr2=0x55a41d951940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d951940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd44c002230 tx=0x7fd44c009790 comp rx=0 tx=0).stop 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- 172.21.15.5:0/1783071271 shutdown_connections 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95c6e0 0x55a41d951940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95bc40 0x55a41d96c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- 172.21.15.5:0/1783071271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95b250 0x55a41d951400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- 172.21.15.5:0/1783071271 >> 172.21.15.5:0/1783071271 conn(0x55a41d952f40 msgr2=0x55a41d95fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- 172.21.15.5:0/1783071271 shutdown_connections 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- 172.21.15.5:0/1783071271 wait complete. 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- start start 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 Processor -- start 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95b250 0x55a41d9680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 0x55a41d968620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a41d973530 con 0x55a41d95bc40 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.916+0000 7fd45de68740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a41d972e60 con 0x55a41d95c6e0 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45de68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a41d973c00 con 0x55a41d95b250 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45de68740 10 client.0 _ll_register_callbacks cb 0x7fd44c0098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 0x55a41d968620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 0x55a41d968620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56214/0 (socket says 172.21.15.5:56214) 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 -- 172.21.15.5:0/1903921585 learned_addr learned my addr 172.21.15.5:0/1903921585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45980a640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95b250 0x55a41d9680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45980a640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 -- 172.21.15.5:0/1903921585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95b250 msgr2=0x55a41d9680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95b250 0x55a41d9680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 -- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 msgr2=0x55a41d96b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 -- 172.21.15.5:0/1903921585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41d96bbe0 con 0x55a41d95bc40 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45980a640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd458808640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 0x55a41d968620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd4440057c0 tx=0x7fd44402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc910e6b7c52a04f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd43cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd43cff9640 1 -- 172.21.15.5:0/1903921585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444045040 con 0x55a41d95bc40 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd43cff9640 1 -- 172.21.15.5:0/1903921585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd444004240 con 0x55a41d95bc40 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44c011630 con 0x55a41d95bc40 2024-04-22T13:26:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd43cff9640 1 -- 172.21.15.5:0/1903921585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd444004440 con 0x55a41d95bc40 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45de68740 10 client.15132 fetch_fsmap learned FSMap version 26 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.917+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a41d9d8400 con 0x55a41d95bc40 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd43cff9640 1 -- 172.21.15.5:0/1903921585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44c011630 con 0x55a41d95bc40 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd43cff9640 1 -- 172.21.15.5:0/1903921585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd4440038a0 con 0x55a41d95bc40 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45de68740 10 client.15132 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45de68740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45de68740 20 client.15132 populate_metadata read hostname 'smithi005' 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45de68740 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d9dcbc0 0x55a41d9db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45de68740 4 client.15132 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "status"}) -- 0x55a41d8c33a0 con 0x55a41d9dcbc0 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45980a640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d9dcbc0 0x55a41d9db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd45980a640 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d9dcbc0 0x55a41d9db8d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf2ac936277141c server_cookie=be60905af1cb1bc6 in_seq=0 out_seq=0 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.918+0000 7fd43cff9640 10 client.15132 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd43cff9640 1 -- 172.21.15.5:0/1903921585 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a41d8c33a0 con 0x55a41d9dcbc0 2024-04-22T13:26:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd43cff9640 10 client.15132 handle_command_reply: tid=0 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 client.15132 shutdown 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 msgr2=0x55a41d968620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 0x55a41d968620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd4440057c0 tx=0x7fd44402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 shutdown_connections 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a41d9dcbc0 0x55a41d9db8d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a41d95bc40 0x55a41d968620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a41d95c6e0 0x55a41d96b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd43cff9640 0 client.15132 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 --2- 172.21.15.5:0/1903921585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a41d95b250 0x55a41d9680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 >> 172.21.15.5:0/1903921585 conn(0x55a41d952f40 msgr2=0x55a41d95fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 shutdown_connections 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 1 -- 172.21.15.5:0/1903921585 wait complete. 2024-04-22T13:26:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45de68740 20 client.15132 trim_cache size 0 max 16384 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24557, 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 86.029046536999999, 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:26:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:26:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 91.351015934000003 2024-04-22T13:26:36.924 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:26:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd4400a3c40 msgr2=0x7fd4400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45b26d640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd4400a3c40 0x7fd4400a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd45009bed0 tx=0x7fd4500d3b80 comp rx=0 tx=0).stop 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 msgr2=0x7fd454198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45b26d640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 0x7fd454198220 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd45000d950 tx=0x7fd45000de10 comp rx=0 tx=0).stop 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.919+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 shutdown_connections 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fd4400a3c40 0x7fd4400a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd454107c80 0x7fd45419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd454105350 0x7fd454198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 --2- 172.21.15.5:0/543957108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd454102a20 0x7fd454198220 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 >> 172.21.15.5:0/543957108 conn(0x7fd4540fc430 msgr2=0x7fd4541032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 shutdown_connections 2024-04-22T13:26:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:36.920+0000 7fd45b26d640 1 -- 172.21.15.5:0/543957108 wait complete. 2024-04-22T13:26:36.926 INFO:tasks.cephadm:Setup complete, yielding 2024-04-22T13:26:36.927 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T13:26:36.936 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:26:36.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-22T13:26:37.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:36 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/23421489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:37.188 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:37.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:36 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/23421489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:36 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/23421489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:38.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:37 smithi049 ceph-mon[31379]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:26:38.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:37 smithi107 ceph-mon[31859]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:26:38.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:37 smithi005 ceph-mon[28006]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:26:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 -- 172.21.15.5:0/1010779927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59141035f0 msgr2=0x7f59141059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 --2- 172.21.15.5:0/1010779927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59141035f0 0x7f59141059e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5904009990 tx=0x7f5904009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 -- 172.21.15.5:0/1010779927 shutdown_connections 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 --2- 172.21.15.5:0/1010779927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914108850 0x7f591410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 --2- 172.21.15.5:0/1010779927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59141035f0 0x7f59141059e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 --2- 172.21.15.5:0/1010779927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5914105f20 0x7f5914108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 -- 172.21.15.5:0/1010779927 >> 172.21.15.5:0/1010779927 conn(0x7f59140fcf60 msgr2=0x7f59140ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 -- 172.21.15.5:0/1010779927 shutdown_connections 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.865+0000 7f591b20c700 1 -- 172.21.15.5:0/1010779927 wait complete. 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 Processor -- start 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 -- start start 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59141035f0 0x7f5914198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914105f20 0x7f5914199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 0x7f59141a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f591410e520 con 0x7f5914108850 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f591410e3a0 con 0x7f5914105f20 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f591b20c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f591410e6a0 con 0x7f59141035f0 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 0x7f59141a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f5913fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914105f20 0x7f5914199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f5913fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914105f20 0x7f5914199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40052/0 (socket says 172.21.15.5:40052) 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f5918fa8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59141035f0 0x7f5914198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f5913fff700 1 -- 172.21.15.5:0/4075701749 learned_addr learned my addr 172.21.15.5:0/4075701749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 0x7f59141a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43350/0 (socket says 172.21.15.5:43350) 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 -- 172.21.15.5:0/4075701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59141035f0 msgr2=0x7f5914198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59141035f0 0x7f5914198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 -- 172.21.15.5:0/4075701749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914105f20 msgr2=0x7f5914199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914105f20 0x7f5914199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 -- 172.21.15.5:0/4075701749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59141a0820 con 0x7f5914108850 2024-04-22T13:26:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.866+0000 7f59197a9700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 0x7f59141a0180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f590c00efc0 tx=0x7f590c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.867+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0092e0 con 0x7f5914108850 2024-04-22T13:26:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.867+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59141a0ab0 con 0x7f5914108850 2024-04-22T13:26:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.867+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f590c004bb0 con 0x7f5914108850 2024-04-22T13:26:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.867+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c004020 con 0x7f5914108850 2024-04-22T13:26:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.867+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59141a1040 con 0x7f5914108850 2024-04-22T13:26:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.868+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5914069730 con 0x7f5914108850 2024-04-22T13:26:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.869+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f590c01b030 con 0x7f5914108850 2024-04-22T13:26:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.869+0000 7f5911ffb700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f58fc0a3cd0 0x7f58fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.869+0000 7f5918fa8700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f58fc0a3cd0 0x7f58fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.870+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f590c0cff80 con 0x7f5914108850 2024-04-22T13:26:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.870+0000 7f5918fa8700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f58fc0a3cd0 0x7f58fc0a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f590400b2e0 tx=0x7f5904005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:38.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:38.874+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f590c022020 con 0x7f5914108850 2024-04-22T13:26:39.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.106+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5914062e80 con 0x7f58fc0a3cd0 2024-04-22T13:26:39.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.107+0000 7f5911ffb700 1 -- 172.21.15.5:0/4075701749 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5914062e80 con 0x7f58fc0a3cd0 2024-04-22T13:26:39.103 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-04-22T13:26:39.103 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-04-22T13:26:39.103 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f58fc0a3cd0 msgr2=0x7f58fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f58fc0a3cd0 0x7f58fc0a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f590400b2e0 tx=0x7f5904005fb0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 msgr2=0x7f59141a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 0x7f59141a0180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f590c00efc0 tx=0x7f590c00c3a0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 shutdown_connections 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5914108850 0x7f59141a0180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59141035f0 0x7f5914198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f58fc0a3cd0 0x7f58fc0a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 --2- 172.21.15.5:0/4075701749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5914105f20 0x7f5914199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.109+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 >> 172.21.15.5:0/4075701749 conn(0x7f59140fcf60 msgr2=0x7f5914104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.110+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 shutdown_connections 2024-04-22T13:26:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:39.110+0000 7f591b20c700 1 -- 172.21.15.5:0/4075701749 wait complete. 2024-04-22T13:26:39.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-22T13:26:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:40.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:39 smithi005 ceph-mon[28006]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:26:40.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:39 smithi049 ceph-mon[31379]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:26:40.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:39 smithi107 ceph-mon[31859]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:26:41.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:40 smithi005 ceph-mon[28006]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:41.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:40 smithi049 ceph-mon[31379]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:41.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:40 smithi107 ceph-mon[31859]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/3035452819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04102ac0 msgr2=0x7f6f04102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/3035452819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04102ac0 0x7f6f04102ee0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00b0d0 tx=0x7f6efc00b3e0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/3035452819 shutdown_connections 2024-04-22T13:26:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/3035452819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f04104660 0x7f6f0410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/3035452819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04102ac0 0x7f6f04102ee0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/3035452819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f04103cc0 0x7f6f04104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/3035452819 >> 172.21.15.5:0/3035452819 conn(0x7f6f040fe060 msgr2=0x7f6f041004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/3035452819 shutdown_connections 2024-04-22T13:26:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.655+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/3035452819 wait complete. 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 Processor -- start 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 -- start start 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f04102ac0 0x7f6f04198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f04103cc0 0x7f6f041991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 0x7f6f041a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0410e5b0 con 0x7f6f04104660 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0410e430 con 0x7f6f04102ac0 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0c0bd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0410e730 con 0x7f6f04103cc0 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 0x7f6f041a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 0x7f6f041a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43370/0 (socket says 172.21.15.5:43370) 2024-04-22T13:26:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 -- 172.21.15.5:0/517657413 learned_addr learned my addr 172.21.15.5:0/517657413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f09e59700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f04102ac0 0x7f6f04198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f09658700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f04103cc0 0x7f6f041991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 -- 172.21.15.5:0/517657413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f04103cc0 msgr2=0x7f6f041991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f04103cc0 0x7f6f041991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 -- 172.21.15.5:0/517657413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f04102ac0 msgr2=0x7f6f04198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f04102ac0 0x7f6f04198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.656+0000 7f6f0a65a700 1 -- 172.21.15.5:0/517657413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f041a08a0 con 0x7f6f04104660 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.657+0000 7f6f0a65a700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 0x7f6f041a01d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400b550 tx=0x7f6ef400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.657+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4015020 con 0x7f6f04104660 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.657+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4011ce0 con 0x7f6f04104660 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.657+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4010dd0 con 0x7f6f04104660 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.657+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f041a0b90 con 0x7f6f04104660 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.657+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f041a10d0 con 0x7f6f04104660 2024-04-22T13:26:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.658+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f04069730 con 0x7f6f04104660 2024-04-22T13:26:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.660+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f6ef4018020 con 0x7f6f04104660 2024-04-22T13:26:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.660+0000 7f6f02ffd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6eec0a3d50 0x7f6eec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.660+0000 7f6f09e59700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6eec0a3d50 0x7f6eec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.660+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f6ef40cfe40 con 0x7f6f04104660 2024-04-22T13:26:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.661+0000 7f6f09e59700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6eec0a3d50 0x7f6eec0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f0419a130 tx=0x7f6efc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.664+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ef401f030 con 0x7f6f04104660 2024-04-22T13:26:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.882+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6f04062e80 con 0x7f6eec0a3d50 2024-04-22T13:26:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.896+0000 7f6f02ffd700 1 -- 172.21.15.5:0/517657413 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f6f04062e80 con 0x7f6eec0a3d50 2024-04-22T13:26:41.892 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T13:26:41.892 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (99s) 16s ago 99s 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f8402367009f 2024-04-22T13:26:41.892 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi049 running (96s) 30s ago 96s 25.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6271c96a529e 2024-04-22T13:26:41.892 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi107 running (110s) 30s ago 110s 17.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 cd1a96d06271 2024-04-22T13:26:41.892 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (118s) 16s ago 118s 17.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f871cdee6c7e 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi049 running (115s) 30s ago 115s 19.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 360b54678b5f 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi107 running (113s) 30s ago 112s 16.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 31f1c4059595 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (102s) 16s ago 101s 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d544728105a2 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi049 running (107s) 30s ago 107s 19.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fbb58709ca37 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi107 running (105s) 30s ago 104s 16.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3d90cf10b340 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) 16s ago 2m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 515ef78fe94a 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi049 running (2m) 30s ago 2m 15.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4e8bbe93aae0 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi107 running (2m) 30s ago 2m 16.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 dfc486f84b81 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (9m) 16s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e9cd05080807 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi049 *:8443,8765 running (7m) 30s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2852b3b02f72 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi107 *:8443,8765 running (7m) 30s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ddc7808f282d 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (9m) 16s ago 9m 51.6M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ecce5d1b31a9 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi049 running (7m) 30s ago 7m 39.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 b37e1c5be6ec 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi107 running (7m) 30s ago 7m 41.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 47e7084a938f 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (6m) 16s ago 6m 50.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 4dfba8397827 2024-04-22T13:26:41.893 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi049 running (6m) 30s ago 6m 49.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 99e9197c9ab3 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi107 running (6m) 30s ago 6m 58.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6533a63b67df 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (5m) 16s ago 5m 54.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 24ba3df5a772 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi049 running (5m) 30s ago 5m 53.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c8bb43c60ee2 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi107 running (4m) 30s ago 4m 54.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 99abbce2d72e 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) 16s ago 4m 45.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 44399764a50d 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi049 running (4m) 30s ago 4m 53.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fd7333ac1fe0 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi107 running (3m) 30s ago 3m 56.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 bc5f5326c3d5 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 16s ago 3m 50.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 136304d1c66b 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi049 running (2m) 30s ago 2m 55.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 917a5c6e2271 2024-04-22T13:26:41.894 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi107 running (2m) 30s ago 2m 44.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 dd3a4ed966e2 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6eec0a3d50 msgr2=0x7f6eec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6eec0a3d50 0x7f6eec0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f0419a130 tx=0x7f6efc00b040 comp rx=0 tx=0).stop 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 msgr2=0x7f6f041a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 0x7f6f041a01d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400b550 tx=0x7f6ef400b860 comp rx=0 tx=0).stop 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 shutdown_connections 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f6eec0a3d50 0x7f6eec0a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f04102ac0 0x7f6f04198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f04104660 0x7f6f041a01d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 --2- 172.21.15.5:0/517657413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f04103cc0 0x7f6f041991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 >> 172.21.15.5:0/517657413 conn(0x7f6f040fe060 msgr2=0x7f6f04106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 shutdown_connections 2024-04-22T13:26:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:41.898+0000 7f6f0c0bd700 1 -- 172.21.15.5:0/517657413 wait complete. 2024-04-22T13:26:42.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:41 smithi005 ceph-mon[28006]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-22T13:26:42.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:41 smithi049 ceph-mon[31379]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-22T13:26:42.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:41 smithi107 ceph-mon[31859]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-22T13:26:42.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-22T13:26:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:43.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:42 smithi005 ceph-mon[28006]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:43.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:42 smithi049 ceph-mon[31379]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:43.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:42 smithi107 ceph-mon[31859]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:44.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:43 smithi005 ceph-mon[28006]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:26:44.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:43 smithi049 ceph-mon[31379]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:26:44.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:43 smithi107 ceph-mon[31859]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:26:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1244596733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f41019a0 msgr2=0x7fb7f4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1244596733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f41019a0 0x7fb7f4101dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b0d0 tx=0x7fb7e400b3e0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1244596733 shutdown_connections 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1244596733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f4103540 0x7fb7f4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1244596733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f41019a0 0x7fb7f4101dc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1244596733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f4102ba0 0x7fb7f4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1244596733 >> 172.21.15.5:0/1244596733 conn(0x7fb7f40fcf40 msgr2=0x7fb7f40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1244596733 shutdown_connections 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.434+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1244596733 wait complete. 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 Processor -- start 2024-04-22T13:26:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 -- start start 2024-04-22T13:26:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f41019a0 0x7fb7f41058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f4102ba0 0x7fb7f4103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 0x7fb7f41044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f410d480 con 0x7fb7f4103540 2024-04-22T13:26:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f410d300 con 0x7fb7f41019a0 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f9d89700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f410d600 con 0x7fb7f4102ba0 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 0x7fb7f41044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 0x7fb7f41044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43398/0 (socket says 172.21.15.5:43398) 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 -- 172.21.15.5:0/1901161957 learned_addr learned my addr 172.21.15.5:0/1901161957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f2ffd700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f4102ba0 0x7fb7f4103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f37fe700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f41019a0 0x7fb7f41058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 -- 172.21.15.5:0/1901161957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f4102ba0 msgr2=0x7fb7f4103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f4102ba0 0x7fb7f4103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 -- 172.21.15.5:0/1901161957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f41019a0 msgr2=0x7fb7f41058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f41019a0 0x7fb7f41058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.435+0000 7fb7f3fff700 1 -- 172.21.15.5:0/1901161957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f4104d90 con 0x7fb7f4103540 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.436+0000 7fb7f3fff700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 0x7fb7f41044c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00d980 tx=0x7fb7ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.436+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec017070 con 0x7fb7f4103540 2024-04-22T13:26:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.436+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec005020 con 0x7fb7f4103540 2024-04-22T13:26:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.436+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f41a2de0 con 0x7fb7f4103540 2024-04-22T13:26:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.436+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00fa10 con 0x7fb7f4103540 2024-04-22T13:26:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.436+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f41a3230 con 0x7fb7f4103540 2024-04-22T13:26:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.437+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f4069760 con 0x7fb7f4103540 2024-04-22T13:26:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.438+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fb7ec004a30 con 0x7fb7f4103540 2024-04-22T13:26:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.439+0000 7fb7f0ff9700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb7dc0a3cd0 0x7fb7dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.439+0000 7fb7f37fe700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb7dc0a3cd0 0x7fb7dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.439+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fb7ec0d02f0 con 0x7fb7f4103540 2024-04-22T13:26:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.439+0000 7fb7f37fe700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb7dc0a3cd0 0x7fb7dc0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009fd0 tx=0x7fb7e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.443+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7f4069760 con 0x7fb7f4103540 2024-04-22T13:26:44.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.667+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb7f406ae30 con 0x7fb7dc0a3cd0 2024-04-22T13:26:44.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.673+0000 7fb7f0ff9700 1 -- 172.21.15.5:0/1901161957 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb7f406ae30 con 0x7fb7dc0a3cd0 2024-04-22T13:26:44.669 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-22T13:26:44.669 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 33s ago 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi049=b;smithi049=e;smithi049=h;smithi049=k;smithi107=c;smithi107=f;smithi107=i;smithi107=l;count:12 2024-04-22T13:26:44.670 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 33s ago 7m smithi005=x;smithi049=y;smithi107=z;count:3 2024-04-22T13:26:44.670 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 33s ago 7m smithi005:172.21.15.5=a;smithi049:172.21.15.49=b;smithi107:172.21.15.107=c;count:3 2024-04-22T13:26:44.670 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 33s ago - 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb7dc0a3cd0 msgr2=0x7fb7dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb7dc0a3cd0 0x7fb7dc0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009fd0 tx=0x7fb7e400b040 comp rx=0 tx=0).stop 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 msgr2=0x7fb7f41044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 0x7fb7f41044c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00d980 tx=0x7fb7ec00dc90 comp rx=0 tx=0).stop 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 shutdown_connections 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fb7dc0a3cd0 0x7fb7dc0a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f41019a0 0x7fb7f41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7f4103540 0x7fb7f41044c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 --2- 172.21.15.5:0/1901161957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f4102ba0 0x7fb7f4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:44.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 >> 172.21.15.5:0/1901161957 conn(0x7fb7f40fcf40 msgr2=0x7fb7f406a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:44.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 shutdown_connections 2024-04-22T13:26:44.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:44.676+0000 7fb7f9d89700 1 -- 172.21.15.5:0/1901161957 wait complete. 2024-04-22T13:26:45.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-22T13:26:45.604 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:45 smithi049 ceph-mon[31379]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:26:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:45 smithi049 ceph-mon[31379]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:45 smithi049 ceph-mon[31379]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T13:26:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:45 smithi107 ceph-mon[31859]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:26:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:45 smithi107 ceph-mon[31859]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:45 smithi107 ceph-mon[31859]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T13:26:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:45 smithi005 ceph-mon[28006]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:26:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:45 smithi005 ceph-mon[28006]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:45 smithi005 ceph-mon[28006]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T13:26:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 -- 172.21.15.5:0/427616539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c075420 msgr2=0x7f7e9c0758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/427616539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c075420 0x7f7e9c0758a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009990 tx=0x7f7e84009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 -- 172.21.15.5:0/427616539 shutdown_connections 2024-04-22T13:26:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/427616539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c0769f0 0x7f7e9c074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/427616539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c10ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/427616539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c075420 0x7f7e9c0758a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 -- 172.21.15.5:0/427616539 >> 172.21.15.5:0/427616539 conn(0x7f7e9c0fe060 msgr2=0x7f7e9c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 -- 172.21.15.5:0/427616539 shutdown_connections 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 -- 172.21.15.5:0/427616539 wait complete. 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.286+0000 7f7ea1af1700 1 Processor -- start 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 -- start start 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c075420 0x7f7e9c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c0769f0 0x7f7e9c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c110770 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1105f0 con 0x7f7e9c075420 2024-04-22T13:26:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7ea1af1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1108f0 con 0x7f7e9c0769f0 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c1a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43424/0 (socket says 172.21.15.5:43424) 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 -- 172.21.15.5:0/3135764268 learned_addr learned my addr 172.21.15.5:0/3135764268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9affd700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c0769f0 0x7f7e9c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9b7fe700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c075420 0x7f7e9c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 -- 172.21.15.5:0/3135764268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c0769f0 msgr2=0x7f7e9c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c0769f0 0x7f7e9c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 -- 172.21.15.5:0/3135764268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c075420 msgr2=0x7f7e9c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c075420 0x7f7e9c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.287+0000 7f7e9bfff700 1 -- 172.21.15.5:0/3135764268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c1a2a60 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.288+0000 7f7e9bfff700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c1a23c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d9b0 tx=0x7f7e8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.288+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c016070 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.288+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c004830 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.288+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c005520 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.288+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c1a2d50 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.288+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a31e0 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.289+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c069730 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.291+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f7e8c0049e0 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.291+0000 7f7e98ff9700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e880a3cc0 0x7f7e880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.291+0000 7f7e9b7fe700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e880a3cc0 0x7f7e880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.291+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f7e8c0d0410 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.292+0000 7f7e9b7fe700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e880a3cc0 0x7f7e880a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009d90 tx=0x7f7e94009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:47.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.295+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e8c01f030 con 0x7f7e9c10a9c0 2024-04-22T13:26:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.511+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7e9c062e80 con 0x7f7e880a3cc0 2024-04-22T13:26:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.513+0000 7f7e98ff9700 1 -- 172.21.15.5:0/3135764268 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7e9c062e80 con 0x7f7e880a3cc0 2024-04-22T13:26:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-04-22T13:26:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-04-22T13:26:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:smithi049 172.21.15.49 2024-04-22T13:26:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:smithi107 172.21.15.107 2024-04-22T13:26:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-04-22T13:26:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e880a3cc0 msgr2=0x7f7e880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e880a3cc0 0x7f7e880a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009d90 tx=0x7f7e94009040 comp rx=0 tx=0).stop 2024-04-22T13:26:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 msgr2=0x7f7e9c1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c1a23c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d9b0 tx=0x7f7e8c00dcc0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 shutdown_connections 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f7e880a3cc0 0x7f7e880a6180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e9c075420 0x7f7e9c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c10a9c0 0x7f7e9c1a23c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 --2- 172.21.15.5:0/3135764268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e9c0769f0 0x7f7e9c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 >> 172.21.15.5:0/3135764268 conn(0x7f7e9c0fe060 msgr2=0x7f7e9c106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 shutdown_connections 2024-04-22T13:26:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:47.515+0000 7f7ea1af1700 1 -- 172.21.15.5:0/3135764268 wait complete. 2024-04-22T13:26:48.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:47 smithi005 ceph-mon[28006]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:26:48.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-22T13:26:48.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:47 smithi049 ceph-mon[31379]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:26:48.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:47 smithi107 ceph-mon[31859]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:26:48.444 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:49.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:48 smithi049 ceph-mon[31379]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:49.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:49 smithi107 ceph-mon[31859]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:49.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:48 smithi005 ceph-mon[28006]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:50.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:50 smithi005 ceph-mon[28006]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T13:26:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 -- 172.21.15.5:0/2522850269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 msgr2=0x7f717c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 --2- 172.21.15.5:0/2522850269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c10ac20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f717400b0a0 tx=0x7f717400b3b0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 -- 172.21.15.5:0/2522850269 shutdown_connections 2024-04-22T13:26:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 --2- 172.21.15.5:0/2522850269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f717c1035d0 0x7f717c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 --2- 172.21.15.5:0/2522850269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c10ac20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 --2- 172.21.15.5:0/2522850269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c105f00 0x7f717c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 -- 172.21.15.5:0/2522850269 >> 172.21.15.5:0/2522850269 conn(0x7f717c0fcf40 msgr2=0x7f717c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 -- 172.21.15.5:0/2522850269 shutdown_connections 2024-04-22T13:26:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.152+0000 7f7182129700 1 -- 172.21.15.5:0/2522850269 wait complete. 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 Processor -- start 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 -- start start 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c1035d0 0x7f717c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f717c105f00 0x7f717c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10e530 con 0x7f717c108830 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10e3b0 con 0x7f717c105f00 2024-04-22T13:26:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f7182129700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10e6b0 con 0x7f717c1035d0 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c1a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60224/0 (socket says 172.21.15.5:60224) 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 -- 172.21.15.5:0/194895492 learned_addr learned my addr 172.21.15.5:0/194895492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717b7fe700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c1035d0 0x7f717c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717affd700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f717c105f00 0x7f717c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 -- 172.21.15.5:0/194895492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c1035d0 msgr2=0x7f717c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c1035d0 0x7f717c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 -- 172.21.15.5:0/194895492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f717c105f00 msgr2=0x7f717c1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.153+0000 7f717bfff700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f717c105f00 0x7f717c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f717bfff700 1 -- 172.21.15.5:0/194895492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c1a07d0 con 0x7f717c108830 2024-04-22T13:26:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f717b7fe700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c1035d0 0x7f717c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f717bfff700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c1a0130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7174006070 tx=0x7f7174004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174021070 con 0x7f717c108830 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7174004020 con 0x7f717c108830 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a0a60 con 0x7f717c108830 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717401d6b0 con 0x7f717c108830 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.154+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a0fa0 con 0x7f717c108830 2024-04-22T13:26:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.155+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c069730 con 0x7f717c108830 2024-04-22T13:26:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.156+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f71740044d0 con 0x7f717c108830 2024-04-22T13:26:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.157+0000 7f7178ff9700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71640a3c40 0x7f71640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.157+0000 7f717b7fe700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71640a3c40 0x7f71640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.157+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f717400f020 con 0x7f717c108830 2024-04-22T13:26:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.157+0000 7f717b7fe700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71640a3c40 0x7f71640a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f716c005950 tx=0x7f716c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.160+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f717402a020 con 0x7f717c108830 2024-04-22T13:26:50.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:50 smithi049 ceph-mon[31379]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T13:26:50.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:50 smithi107 ceph-mon[31859]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T13:26:50.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.379+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f717c062e80 con 0x7f71640a3c40 2024-04-22T13:26:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.381+0000 7f7178ff9700 1 -- 172.21.15.5:0/194895492 <== mgr.14152 v2:172.21.15.5:6800/2674774949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f717c062e80 con 0x7f71640a3c40 2024-04-22T13:26:50.377 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-22T13:26:50.377 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-22T13:26:50.377 INFO:teuthology.orchestra.run.smithi005.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-22T13:26:50.377 INFO:teuthology.orchestra.run.smithi005.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T13:26:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71640a3c40 msgr2=0x7f71640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71640a3c40 0x7f71640a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f716c005950 tx=0x7f716c005f50 comp rx=0 tx=0).stop 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 msgr2=0x7f717c1a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c1a0130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7174006070 tx=0x7f7174004ef0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 shutdown_connections 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f717c108830 0x7f717c1a0130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f717c1035d0 0x7f717c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f71640a3c40 0x7f71640a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 --2- 172.21.15.5:0/194895492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f717c105f00 0x7f717c1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 >> 172.21.15.5:0/194895492 conn(0x7f717c0fcf40 msgr2=0x7f717c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 shutdown_connections 2024-04-22T13:26:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:50.383+0000 7f7182129700 1 -- 172.21.15.5:0/194895492 wait complete. 2024-04-22T13:26:51.083 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T13:26:51.092 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:26:51.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T13:26:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:52.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:52 smithi049 ceph-mon[31379]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:26:52.274 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:52 smithi049 ceph-mon[31379]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:52.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:52 smithi107 ceph-mon[31859]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:26:52.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:52 smithi107 ceph-mon[31859]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:52.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:52 smithi005 ceph-mon[28006]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:26:52.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:52 smithi005 ceph-mon[28006]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.001+0000 7f91db7f6700 1 -- 172.21.15.5:0/3909229441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 msgr2=0x7f91d4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.001+0000 7f91db7f6700 1 --2- 172.21.15.5:0/3909229441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d4101dc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91d00099c0 tx=0x7f91d0009cd0 comp rx=0 tx=0).stop 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.001+0000 7f91db7f6700 1 -- 172.21.15.5:0/3909229441 shutdown_connections 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.001+0000 7f91db7f6700 1 --2- 172.21.15.5:0/3909229441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91d4103540 0x7f91d4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 --2- 172.21.15.5:0/3909229441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d4101dc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 --2- 172.21.15.5:0/3909229441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d4102ba0 0x7f91d4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 -- 172.21.15.5:0/3909229441 >> 172.21.15.5:0/3909229441 conn(0x7f91d40fcf60 msgr2=0x7f91d40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 -- 172.21.15.5:0/3909229441 shutdown_connections 2024-04-22T13:26:52.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 -- 172.21.15.5:0/3909229441 wait complete. 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 Processor -- start 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.002+0000 7f91db7f6700 1 -- start start 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91db7f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d41058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91db7f6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d4102ba0 0x7f91d4103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91db7f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91d4103540 0x7f91d41044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91db7f6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91d410d4b0 con 0x7f91d41019a0 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91db7f6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91d410d330 con 0x7f91d4103540 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91db7f6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91d410d630 con 0x7f91d4102ba0 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d41058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:53.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d41058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60232/0 (socket says 172.21.15.5:60232) 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 -- 172.21.15.5:0/1785523042 learned_addr learned my addr 172.21.15.5:0/1785523042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d8d91700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d4102ba0 0x7f91d4103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9d93700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91d4103540 0x7f91d41044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 -- 172.21.15.5:0/1785523042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d4102ba0 msgr2=0x7f91d4103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d4102ba0 0x7f91d4103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 -- 172.21.15.5:0/1785523042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91d4103540 msgr2=0x7f91d41044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91d4103540 0x7f91d41044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 -- 172.21.15.5:0/1785523042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4104d90 con 0x7f91d41019a0 2024-04-22T13:26:53.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.003+0000 7f91d9592700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d41058e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91d00038c0 tx=0x7f91d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.005+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0027070 con 0x7f91d41019a0 2024-04-22T13:26:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.005+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d00040e0 con 0x7f91d41019a0 2024-04-22T13:26:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.005+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d41a2d50 con 0x7f91d41019a0 2024-04-22T13:26:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.005+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0022df0 con 0x7f91d41019a0 2024-04-22T13:26:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.006+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d41a31a0 con 0x7f91d41019a0 2024-04-22T13:26:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.007+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d4069730 con 0x7f91d41019a0 2024-04-22T13:26:53.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.007+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f91d002a070 con 0x7f91d41019a0 2024-04-22T13:26:53.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.008+0000 7f91ca7fc700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91c00a3cc0 0x7f91c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:53.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.008+0000 7f91d8d91700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91c00a3cc0 0x7f91c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:53.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.008+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f91d00d0fe0 con 0x7f91d41019a0 2024-04-22T13:26:53.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.009+0000 7f91d8d91700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91c00a3cc0 0x7f91c00a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f91c4004630 tx=0x7f91c400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:53.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.013+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91d0013070 con 0x7f91d41019a0 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.270+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f91d41a3530 con 0x7f91d41019a0 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.271+0000 7f91ca7fc700 1 -- 172.21.15.5:0/1785523042 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3072 (secure 0 0 0) 0x7f91d009dc20 con 0x7f91d41019a0 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stdout:e26 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-04-22T13:26:53.267 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 26 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-04-22T13:25:10.226899+0000 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-04-22T13:25:17.229740+0000 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-04-22T13:26:53.268 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24557,1=24512,2=24496,3=24485,4=24469} 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-04-22T13:26:53.269 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24557 members: 24485,24469,24496,24557,24512 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24557} state up:active seq 3 addr [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.107:6834/2603097758,v1:172.21.15.107:6835/2603097758] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{1:24512} state up:active seq 6 addr [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{1:24412} state up:standby-replay seq 1 addr [v2:172.21.15.107:6832/1584198825,v1:172.21.15.107:6833/1584198825] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{2:24496} state up:active seq 6 addr [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{2:14637} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/4083506255,v1:172.21.15.5:6841/4083506255] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{3:24485} state up:active seq 9 addr [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.5:6838/373669901,v1:172.21.15.5:6839/373669901] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.270 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{4:24469} state up:active seq 8 addr [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{4:14550} state up:standby-replay seq 1 addr [v2:172.21.15.5:6836/1096407808,v1:172.21.15.5:6837/1096407808] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{-1:14520} state up:standby seq 1 addr [v2:172.21.15.49:6832/1569295173,v1:172.21.15.49:6833/1569295173] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.271 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.5:6834/219923343,v1:172.21.15.5:6835/219923343] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91c00a3cc0 msgr2=0x7f91c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91c00a3cc0 0x7f91c00a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f91c4004630 tx=0x7f91c400b3c0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 msgr2=0x7f91d41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d41058e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91d00038c0 tx=0x7f91d00047b0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 shutdown_connections 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f91c00a3cc0 0x7f91c00a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91d4103540 0x7f91d41044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91d41019a0 0x7f91d41058e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 --2- 172.21.15.5:0/1785523042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d4102ba0 0x7f91d4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 >> 172.21.15.5:0/1785523042 conn(0x7f91d40fcf60 msgr2=0x7f91d40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 shutdown_connections 2024-04-22T13:26:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:53.273+0000 7f91db7f6700 1 -- 172.21.15.5:0/1785523042 wait complete. 2024-04-22T13:26:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:26:54.057 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-22T13:26:54.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:54 smithi049 ceph-mon[31379]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:26:54.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:54 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1785523042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T13:26:54.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:54 smithi107 ceph-mon[31859]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:26:54.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:54 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1785523042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T13:26:54.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:54 smithi005 ceph-mon[28006]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:26:54.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:54 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1785523042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T13:26:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:26:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 -- 172.21.15.5:0/4157477179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 msgr2=0x7f9c8c1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/4157477179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c1041a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009990 tx=0x7f9c80009ca0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 -- 172.21.15.5:0/4157477179 shutdown_connections 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/4157477179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c100d60 0x7f9c8c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/4157477179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c1046e0 0x7f9c8c10af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/4157477179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c1041a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 -- 172.21.15.5:0/4157477179 >> 172.21.15.5:0/4157477179 conn(0x7f9c8c0fc6a0 msgr2=0x7f9c8c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 -- 172.21.15.5:0/4157477179 shutdown_connections 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.932+0000 7f9c91fcc700 1 -- 172.21.15.5:0/4157477179 wait complete. 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 Processor -- start 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 -- start start 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c100d60 0x7f9c8c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c1046e0 0x7f9c8c1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10e530 con 0x7f9c8c103d40 2024-04-22T13:26:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10e3b0 con 0x7f9c8c100d60 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c91fcc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10e6b0 con 0x7f9c8c1046e0 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60260/0 (socket says 172.21.15.5:60260) 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c8affd700 1 -- 172.21.15.5:0/2292558143 learned_addr learned my addr 172.21.15.5:0/2292558143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c8b7fe700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c100d60 0x7f9c8c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.933+0000 7f9c8bfff700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c1046e0 0x7f9c8c1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c8affd700 1 -- 172.21.15.5:0/2292558143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c1046e0 msgr2=0x7f9c8c1a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c8affd700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c1046e0 0x7f9c8c1a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c8affd700 1 -- 172.21.15.5:0/2292558143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c100d60 msgr2=0x7f9c8c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c8affd700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c100d60 0x7f9c8c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c8affd700 1 -- 172.21.15.5:0/2292558143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c1a6df0 con 0x7f9c8c103d40 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c8affd700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c19f6d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9c800036f0 tx=0x7f9c800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80027070 con 0x7f9c8c103d40 2024-04-22T13:26:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c80004000 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c1a7080 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80022c20 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.934+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c1a74d0 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.935+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c069730 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.936+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f9c8002a070 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.937+0000 7f9c88ff9700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3c40 0x7f9c740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.937+0000 7f9c8b7fe700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3c40 0x7f9c740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.937+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f9c800d1400 con 0x7f9c8c103d40 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.937+0000 7f9c8b7fe700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3c40 0x7f9c740a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c06b010 tx=0x7f9c7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:55.941+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c8005eff0 con 0x7f9c8c103d40 2024-04-22T13:26:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.153+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9c8c1a7730 con 0x7f9c8c103d40 2024-04-22T13:26:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.154+0000 7f9c88ff9700 1 -- 172.21.15.5:0/2292558143 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5244 (secure 0 0 0) 0x7f9c80013070 con 0x7f9c8c103d40 2024-04-22T13:26:56.150 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 91 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-04-22T13:17:12.208053+0000 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-04-22T13:26:15.450938+0000 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-22T13:26:56.151 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/236509498,v1:172.21.15.5:6803/236509498] [v2:172.21.15.5:6804/236509498,v1:172.21.15.5:6805/236509498] exists,up 39983697-bdf7-4808-8aaf-f2ae5083be8d 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6800/3148163706,v1:172.21.15.49:6801/3148163706] [v2:172.21.15.49:6802/3148163706,v1:172.21.15.49:6803/3148163706] exists,up 555f251b-3c1d-48a2-a456-d45129ace7df 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/1765086658,v1:172.21.15.107:6801/1765086658] [v2:172.21.15.107:6802/1765086658,v1:172.21.15.107:6803/1765086658] exists,up 7d526a13-cdf8-49ac-9871-b8a577898db3 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/466638912,v1:172.21.15.5:6811/466638912] [v2:172.21.15.5:6812/466638912,v1:172.21.15.5:6813/466638912] exists,up 8732d14a-c8a2-4cbe-ac81-8d7eccbecd08 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 34 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6808/939487791,v1:172.21.15.49:6809/939487791] [v2:172.21.15.49:6810/939487791,v1:172.21.15.49:6811/939487791] exists,up 9dc694d4-956f-424a-a8a7-e56d4e3b1213 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 40 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/2834490578,v1:172.21.15.107:6809/2834490578] [v2:172.21.15.107:6810/2834490578,v1:172.21.15.107:6811/2834490578] exists,up 2da21bf6-32ce-49f7-93ba-362c202d18cb 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 48 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/2491838519,v1:172.21.15.5:6819/2491838519] [v2:172.21.15.5:6820/2491838519,v1:172.21.15.5:6821/2491838519] exists,up b51d6e8a-1cb9-4856-9945-94e6ecb354d9 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 54 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6816/3440466171,v1:172.21.15.49:6817/3440466171] [v2:172.21.15.49:6818/3440466171,v1:172.21.15.49:6819/3440466171] exists,up 49b9738e-63b0-4716-8f64-d9508227837a 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 63 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/4036561619,v1:172.21.15.107:6817/4036561619] [v2:172.21.15.107:6818/4036561619,v1:172.21.15.107:6819/4036561619] exists,up 1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 71 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/3613366882,v1:172.21.15.5:6827/3613366882] [v2:172.21.15.5:6828/3613366882,v1:172.21.15.5:6829/3613366882] exists,up 2392b48b-bdb9-4875-8725-20edf192d686 2024-04-22T13:26:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 76 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6824/2579440718,v1:172.21.15.49:6825/2579440718] [v2:172.21.15.49:6826/2579440718,v1:172.21.15.49:6827/2579440718] exists,up aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 84 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/1856713900,v1:172.21.15.107:6825/1856713900] [v2:172.21.15.107:6826/1856713900,v1:172.21.15.107:6827/1856713900] exists,up 1c592b75-f30f-4e7f-b38c-5b09e1252184 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.a [2,11] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1d [8,11] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.0 [5,9] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,7] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.21 [4,8] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.22 [3,9] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.24 [4,9] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.28 [5,6] 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1744519677 expires 2024-04-23T13:18:14.098934+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1556657643 expires 2024-04-23T13:18:14.098934+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2261038189 expires 2024-04-23T13:17:46.974612+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/4095818300 expires 2024-04-23T13:18:14.098934+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1954020614 expires 2024-04-23T13:17:46.974612+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1797990010 expires 2024-04-23T13:18:14.098934+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/3443029620 expires 2024-04-23T13:17:46.974612+0000 2024-04-22T13:26:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/3443029620 expires 2024-04-23T13:17:46.974612+0000 2024-04-22T13:26:56.154 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1011902624 expires 2024-04-23T13:17:46.974612+0000 2024-04-22T13:26:56.154 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/4095818300 expires 2024-04-23T13:18:14.098934+0000 2024-04-22T13:26:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3c40 msgr2=0x7f9c740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3c40 0x7f9c740a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c06b010 tx=0x7f9c7c00b410 comp rx=0 tx=0).stop 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 msgr2=0x7f9c8c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c19f6d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9c800036f0 tx=0x7f9c800047b0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 shutdown_connections 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f9c740a3c40 0x7f9c740a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c100d60 0x7f9c8c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c8c1046e0 0x7f9c8c1a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 --2- 172.21.15.5:0/2292558143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8c103d40 0x7f9c8c19f6d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.156+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 >> 172.21.15.5:0/2292558143 conn(0x7f9c8c0fc6a0 msgr2=0x7f9c8c101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.157+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 shutdown_connections 2024-04-22T13:26:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:56.157+0000 7f9c91fcc700 1 -- 172.21.15.5:0/2292558143 wait complete. 2024-04-22T13:26:56.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:56 smithi049 ceph-mon[31379]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-22T13:26:56.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:56 smithi107 ceph-mon[31859]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-22T13:26:56.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:56 smithi005 ceph-mon[28006]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-22T13:26:56.938 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-22T13:26:56.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 -- 172.21.15.5:0/2718067182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 msgr2=0x7f4b88101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2718067182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b88101ff0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009ca0 tx=0x7f4b8002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 -- 172.21.15.5:0/2718067182 shutdown_connections 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2718067182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b88101ff0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2718067182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b881011d0 0x7f4b88101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2718067182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88100870 0x7f4b88100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 -- 172.21.15.5:0/2718067182 >> 172.21.15.5:0/2718067182 conn(0x7f4b880fc410 msgr2=0x7f4b880fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 -- 172.21.15.5:0/2718067182 shutdown_connections 2024-04-22T13:26:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 -- 172.21.15.5:0/2718067182 wait complete. 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 Processor -- start 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 -- start start 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88100870 0x7f4b8819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.069+0000 7f4b900a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b881011d0 0x7f4b8819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b900a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b881a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b900a7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810fcf0 con 0x7f4b88101b70 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b900a7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810fb70 con 0x7f4b88100870 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b900a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810fe70 con 0x7f4b881011d0 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b881a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b881a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60264/0 (socket says 172.21.15.5:60264) 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 -- 172.21.15.5:0/2773787329 learned_addr learned my addr 172.21.15.5:0/2773787329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8de43640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88100870 0x7f4b8819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8d642640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b881011d0 0x7f4b8819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 -- 172.21.15.5:0/2773787329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b881011d0 msgr2=0x7f4b8819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b881011d0 0x7f4b8819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 -- 172.21.15.5:0/2773787329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88100870 msgr2=0x7f4b8819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88100870 0x7f4b8819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 -- 172.21.15.5:0/2773787329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881a1f10 con 0x7f4b88101b70 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b8e644640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b881a18a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4b80032520 tx=0x7f4b800347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80039040 con 0x7f4b88101b70 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b80044070 con 0x7f4b88101b70 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a21a0 con 0x7f4b88101b70 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800042a0 con 0x7f4b88101b70 2024-04-22T13:26:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.070+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a2690 con 0x7f4b88101b70 2024-04-22T13:26:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.071+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8810b750 con 0x7f4b88101b70 2024-04-22T13:26:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.073+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f4b80003970 con 0x7f4b88101b70 2024-04-22T13:26:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.073+0000 7f4b7effd640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4b740a3bb0 0x7f4b740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.074+0000 7f4b8de43640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4b740a3bb0 0x7f4b740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.074+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f4b800f26a0 con 0x7f4b88101b70 2024-04-22T13:26:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.074+0000 7f4b8de43640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4b740a3bb0 0x7f4b740a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400bbe0 tx=0x7f4b8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.078+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b80035d30 con 0x7f4b88101b70 2024-04-22T13:26:57.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.267+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b88062b50 con 0x7f4b88101b70 2024-04-22T13:26:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.269+0000 7f4b7effd640 1 -- 172.21.15.5:0/2773787329 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22549 (secure 0 0 0) 0x7f4b800bf090 con 0x7f4b88101b70 2024-04-22T13:26:57.266 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:57.266 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:26:15.450938+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4b740a3bb0 msgr2=0x7f4b740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4b740a3bb0 0x7f4b740a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400bbe0 tx=0x7f4b8400b390 comp rx=0 tx=0).stop 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 msgr2=0x7f4b881a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b881a18a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4b80032520 tx=0x7f4b800347b0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 shutdown_connections 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4b740a3bb0 0x7f4b740a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b88101b70 0x7f4b881a18a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b881011d0 0x7f4b8819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 --2- 172.21.15.5:0/2773787329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b88100870 0x7f4b8819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 >> 172.21.15.5:0/2773787329 conn(0x7f4b880fc410 msgr2=0x7f4b880fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 shutdown_connections 2024-04-22T13:26:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.271+0000 7f4b900a7640 1 -- 172.21.15.5:0/2773787329 wait complete. 2024-04-22T13:26:57.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:57 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2292558143' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T13:26:57.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:57 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2292558143' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T13:26:57.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:26:57.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:26:57.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:57 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2292558143' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T13:26:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 -- 172.21.15.5:0/3654171441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 msgr2=0x7f5930107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 --2- 172.21.15.5:0/3654171441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f5930107730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5920009900 tx=0x7f592002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 -- 172.21.15.5:0/3654171441 shutdown_connections 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 --2- 172.21.15.5:0/3654171441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f5930107730 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 --2- 172.21.15.5:0/3654171441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5930107c70 0x7f593010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 --2- 172.21.15.5:0/3654171441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5930102a10 0x7f5930104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 -- 172.21.15.5:0/3654171441 >> 172.21.15.5:0/3654171441 conn(0x7f59300fc420 msgr2=0x7f59300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 -- 172.21.15.5:0/3654171441 shutdown_connections 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 -- 172.21.15.5:0/3654171441 wait complete. 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.388+0000 7f5938396640 1 Processor -- start 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 -- start start 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5930102a10 0x7f5930198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f59301986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5930107c70 0x7f593019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f593010daa0 con 0x7f5930105340 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f593010d920 con 0x7f5930107c70 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f593010dc20 con 0x7f5930102a10 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f59301986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f59301986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60280/0 (socket says 172.21.15.5:60280) 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 -- 172.21.15.5:0/3122912264 learned_addr learned my addr 172.21.15.5:0/3122912264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5936132640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5930102a10 0x7f5930198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 -- 172.21.15.5:0/3122912264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5930102a10 msgr2=0x7f5930198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5930102a10 0x7f5930198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 -- 172.21.15.5:0/3122912264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5930107c70 msgr2=0x7f593019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5930107c70 0x7f593019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 -- 172.21.15.5:0/3122912264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593019fda0 con 0x7f5930105340 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5935931640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f59301986c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5920036040 tx=0x7f59200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920043020 con 0x7f5930105340 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5920004aa0 con 0x7f5930105340 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920003de0 con 0x7f5930105340 2024-04-22T13:26:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.389+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59301a0030 con 0x7f5930105340 2024-04-22T13:26:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.390+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59301a0580 con 0x7f5930105340 2024-04-22T13:26:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.390+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59300fdf10 con 0x7f5930105340 2024-04-22T13:26:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.391+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f5920046020 con 0x7f5930105340 2024-04-22T13:26:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.392+0000 7f59277fe640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f591c0a3bb0 0x7f591c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.392+0000 7f5936132640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f591c0a3bb0 0x7f591c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.392+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f59200f27d0 con 0x7f5930105340 2024-04-22T13:26:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.393+0000 7f5936132640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f591c0a3bb0 0x7f591c0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f592c0058c0 tx=0x7f592c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.397+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f592004d020 con 0x7f5930105340 2024-04-22T13:26:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.583+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5930062b50 con 0x7f5930105340 2024-04-22T13:26:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.583+0000 7f59277fe640 1 -- 172.21.15.5:0/3122912264 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f59200bf1c0 con 0x7f5930105340 2024-04-22T13:26:57.580 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f591c0a3bb0 msgr2=0x7f591c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f591c0a3bb0 0x7f591c0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f592c0058c0 tx=0x7f592c00a2d0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 msgr2=0x7f59301986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f59301986c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5920036040 tx=0x7f59200042f0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 shutdown_connections 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f591c0a3bb0 0x7f591c0a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5930105340 0x7f59301986c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5930107c70 0x7f593019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 --2- 172.21.15.5:0/3122912264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5930102a10 0x7f5930198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 >> 172.21.15.5:0/3122912264 conn(0x7f59300fc420 msgr2=0x7f5930103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 shutdown_connections 2024-04-22T13:26:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.586+0000 7f5938396640 1 -- 172.21.15.5:0/3122912264 wait complete. 2024-04-22T13:26:57.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 -- 172.21.15.5:0/1012493399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344101fa0 msgr2=0x7f13441023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 --2- 172.21.15.5:0/1012493399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344101fa0 0x7f13441023c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f133c009900 tx=0x7f133c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 -- 172.21.15.5:0/1012493399 shutdown_connections 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 --2- 172.21.15.5:0/1012493399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344103b50 0x7f134410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 --2- 172.21.15.5:0/1012493399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1344103190 0x7f1344103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 --2- 172.21.15.5:0/1012493399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344101fa0 0x7f13441023c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 -- 172.21.15.5:0/1012493399 >> 172.21.15.5:0/1012493399 conn(0x7f13440fd510 msgr2=0x7f13440ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 -- 172.21.15.5:0/1012493399 shutdown_connections 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.702+0000 7f1348932640 1 -- 172.21.15.5:0/1012493399 wait complete. 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 Processor -- start 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 -- start start 2024-04-22T13:26:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344101fa0 0x7f1344075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 0x7f1344074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1344103b50 0x7f1344074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f134410dae0 con 0x7f1344103190 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f134410d960 con 0x7f1344103b50 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1348932640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f134410dc60 con 0x7f1344101fa0 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 0x7f1344074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 0x7f1344074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60292/0 (socket says 172.21.15.5:60292) 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 -- 172.21.15.5:0/760029944 learned_addr learned my addr 172.21.15.5:0/760029944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1342d9d640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1344103b50 0x7f1344074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f134259c640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344101fa0 0x7f1344075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 -- 172.21.15.5:0/760029944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344101fa0 msgr2=0x7f1344075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344101fa0 0x7f1344075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 -- 172.21.15.5:0/760029944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1344103b50 msgr2=0x7f1344074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1344103b50 0x7f1344074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f1341d9b640 1 -- 172.21.15.5:0/760029944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1344074dc0 con 0x7f1344103190 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.703+0000 7f134259c640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344101fa0 0x7f1344075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.704+0000 7f1341d9b640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 0x7f1344074340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f132c00e9f0 tx=0x7f132c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.704+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00cd80 con 0x7f1344103190 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.704+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13440750b0 con 0x7f1344103190 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.704+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132c0108e0 con 0x7f1344103190 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.704+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c018980 con 0x7f1344103190 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.704+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13441a6cc0 con 0x7f1344103190 2024-04-22T13:26:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.705+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1344069250 con 0x7f1344103190 2024-04-22T13:26:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.706+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f132c010430 con 0x7f1344103190 2024-04-22T13:26:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.707+0000 7f133b7fe640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13300a3bb0 0x7f13300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.707+0000 7f134259c640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13300a3bb0 0x7f13300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.707+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f132c0d07d0 con 0x7f1344103190 2024-04-22T13:26:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.707+0000 7f134259c640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13300a3bb0 0x7f13300a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f133c006010 tx=0x7f133c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.711+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f132c09d160 con 0x7f1344103190 2024-04-22T13:26:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.898+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1344062b50 con 0x7f1344103190 2024-04-22T13:26:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.899+0000 7f133b7fe640 1 -- 172.21.15.5:0/760029944 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f132c09ca60 con 0x7f1344103190 2024-04-22T13:26:57.896 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13300a3bb0 msgr2=0x7f13300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13300a3bb0 0x7f13300a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f133c006010 tx=0x7f133c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 msgr2=0x7f1344074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 0x7f1344074340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f132c00e9f0 tx=0x7f132c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 shutdown_connections 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f13300a3bb0 0x7f13300a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1344103190 0x7f1344074340 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1344103b50 0x7f1344074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 --2- 172.21.15.5:0/760029944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1344101fa0 0x7f1344075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 >> 172.21.15.5:0/760029944 conn(0x7f13440fd510 msgr2=0x7f13441065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 shutdown_connections 2024-04-22T13:26:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:57.901+0000 7f1348932640 1 -- 172.21.15.5:0/760029944 wait complete. 2024-04-22T13:26:57.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:26:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 -- 172.21.15.5:0/2587976499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64201029a0 msgr2=0x7f64201092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 --2- 172.21.15.5:0/2587976499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64201029a0 0x7f64201092c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6418009ca0 tx=0x7f641802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 -- 172.21.15.5:0/2587976499 shutdown_connections 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 --2- 172.21.15.5:0/2587976499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64201029a0 0x7f64201092c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 --2- 172.21.15.5:0/2587976499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6420101fe0 0x7f6420102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 --2- 172.21.15.5:0/2587976499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6420100e90 0x7f64201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 -- 172.21.15.5:0/2587976499 >> 172.21.15.5:0/2587976499 conn(0x7f64200fc420 msgr2=0x7f64200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 -- 172.21.15.5:0/2587976499 shutdown_connections 2024-04-22T13:26:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.018+0000 7f6424888640 1 -- 172.21.15.5:0/2587976499 wait complete. 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 Processor -- start 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 -- start start 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6420100e90 0x7f6420069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 0x7f642019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64201029a0 0x7f642019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f642010c9e0 con 0x7f6420101fe0 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f642010c860 con 0x7f64201029a0 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f6424888640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f642010cb60 con 0x7f6420100e90 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 0x7f642019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 0x7f642019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60304/0 (socket says 172.21.15.5:60304) 2024-04-22T13:26:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641dd9b640 1 -- 172.21.15.5:0/2776278084 learned_addr learned my addr 172.21.15.5:0/2776278084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641e59c640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6420100e90 0x7f6420069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641dd9b640 1 -- 172.21.15.5:0/2776278084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6420100e90 msgr2=0x7f6420069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641dd9b640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6420100e90 0x7f6420069dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.019+0000 7f641ed9d640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64201029a0 0x7f642019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f641dd9b640 1 -- 172.21.15.5:0/2776278084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64201029a0 msgr2=0x7f642019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f641dd9b640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64201029a0 0x7f642019f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f641dd9b640 1 -- 172.21.15.5:0/2776278084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642019fe10 con 0x7f6420101fe0 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f641ed9d640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64201029a0 0x7f642019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f641dd9b640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 0x7f642019cee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9c0 tx=0x7f641400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641400cd80 con 0x7f6420101fe0 2024-04-22T13:26:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64140035f0 con 0x7f6420101fe0 2024-04-22T13:26:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64201a00a0 con 0x7f6420101fe0 2024-04-22T13:26:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414010690 con 0x7f6420101fe0 2024-04-22T13:26:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.020+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64201a05a0 con 0x7f6420101fe0 2024-04-22T13:26:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.021+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64201081c0 con 0x7f6420101fe0 2024-04-22T13:26:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.022+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f6414004000 con 0x7f6420101fe0 2024-04-22T13:26:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.022+0000 7f640f7fe640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f64080a3bb0 0x7f64080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.023+0000 7f641e59c640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f64080a3bb0 0x7f64080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.023+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f64140d0a50 con 0x7f6420101fe0 2024-04-22T13:26:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.023+0000 7f641e59c640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f64080a3bb0 0x7f64080a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6410005d00 tx=0x7f6410005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.027+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f641409d3e0 con 0x7f6420101fe0 2024-04-22T13:26:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.213+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6420062b80 con 0x7f6420101fe0 2024-04-22T13:26:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.214+0000 7f640f7fe640 1 -- 172.21.15.5:0/2776278084 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f641409cce0 con 0x7f6420101fe0 2024-04-22T13:26:58.211 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:26:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.215+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f64080a3bb0 msgr2=0x7f64080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.215+0000 7f6424888640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f64080a3bb0 0x7f64080a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6410005d00 tx=0x7f6410005c50 comp rx=0 tx=0).stop 2024-04-22T13:26:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.215+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 msgr2=0x7f642019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.215+0000 7f6424888640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 0x7f642019cee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9c0 tx=0x7f641400ee80 comp rx=0 tx=0).stop 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 shutdown_connections 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f64080a3bb0 0x7f64080a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6420101fe0 0x7f642019cee0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64201029a0 0x7f642019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 --2- 172.21.15.5:0/2776278084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6420100e90 0x7f6420069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 >> 172.21.15.5:0/2776278084 conn(0x7f64200fc420 msgr2=0x7f6420105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 shutdown_connections 2024-04-22T13:26:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.216+0000 7f6424888640 1 -- 172.21.15.5:0/2776278084 wait complete. 2024-04-22T13:26:58.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:58 smithi005 ceph-mon[28006]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:26:58.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2773787329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:58.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3122912264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:58.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/760029944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:58.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:58.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:58 smithi049 ceph-mon[31379]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:26:58.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2773787329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:58.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3122912264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:58.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/760029944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:58.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:58 smithi107 ceph-mon[31859]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:26:58.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2773787329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:58.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3122912264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:26:58.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/760029944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:26:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 -- 172.21.15.5:0/330593184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 msgr2=0x7f35a8107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 --2- 172.21.15.5:0/330593184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a8107740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 -- 172.21.15.5:0/330593184 shutdown_connections 2024-04-22T13:26:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 --2- 172.21.15.5:0/330593184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a8107740 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 --2- 172.21.15.5:0/330593184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a8107c80 0x7f35a810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 --2- 172.21.15.5:0/330593184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35a8102a20 0x7f35a8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 -- 172.21.15.5:0/330593184 >> 172.21.15.5:0/330593184 conn(0x7f35a80fc410 msgr2=0x7f35a80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 -- 172.21.15.5:0/330593184 shutdown_connections 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.329+0000 7f35ad443640 1 -- 172.21.15.5:0/330593184 wait complete. 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 Processor -- start 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 -- start start 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35a8102a20 0x7f35a819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a8107c80 0x7f35a81a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8101590 con 0x7f35a8105350 2024-04-22T13:26:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8101410 con 0x7f35a8107c80 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35ad443640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8101710 con 0x7f35a8102a20 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60312/0 (socket says 172.21.15.5:60312) 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35a67fc640 1 -- 172.21.15.5:0/3722292235 learned_addr learned my addr 172.21.15.5:0/3722292235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35a77fe640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a8107c80 0x7f35a81a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.330+0000 7f35a6ffd640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35a8102a20 0x7f35a819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35a67fc640 1 -- 172.21.15.5:0/3722292235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35a8102a20 msgr2=0x7f35a819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35a67fc640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35a8102a20 0x7f35a819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35a67fc640 1 -- 172.21.15.5:0/3722292235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a8107c80 msgr2=0x7f35a81a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35a67fc640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a8107c80 0x7f35a81a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35a67fc640 1 -- 172.21.15.5:0/3722292235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81a1fb0 con 0x7f35a8105350 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35a67fc640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a819a8d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f359c000c00 tx=0x7f359c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c043020 con 0x7f35a8105350 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a2240 con 0x7f35a8105350 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f359c004440 con 0x7f35a8105350 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c003c90 con 0x7f35a8105350 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.331+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a2790 con 0x7f35a8105350 2024-04-22T13:26:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.332+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a81a2990 con 0x7f35a8105350 2024-04-22T13:26:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.333+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f359c046020 con 0x7f35a8105350 2024-04-22T13:26:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.334+0000 7f358ffff640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35900a3bb0 0x7f35900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.334+0000 7f35a6ffd640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35900a3bb0 0x7f35900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.334+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f359c0f2560 con 0x7f35a8105350 2024-04-22T13:26:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.334+0000 7f35a6ffd640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35900a3bb0 0x7f35900a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f35980058c0 tx=0x7f359800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.338+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f359c04d020 con 0x7f35a8105350 2024-04-22T13:26:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.566+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35a81a2b60 con 0x7f35a8105350 2024-04-22T13:26:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.566+0000 7f358ffff640 1 -- 172.21.15.5:0/3722292235 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11905 (secure 0 0 0) 0x7f359c0bef50 con 0x7f35a8105350 2024-04-22T13:26:58.564 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:58.564 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:25:17.229740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24485,24469,24496,24557,24512]},"id":1}]} 2024-04-22T13:26:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:26:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35900a3bb0 msgr2=0x7f35900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35900a3bb0 0x7f35900a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f35980058c0 tx=0x7f359800a2d0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 msgr2=0x7f35a819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a819a8d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f359c000c00 tx=0x7f359c004040 comp rx=0 tx=0).stop 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 shutdown_connections 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f35900a3bb0 0x7f35900a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a8105350 0x7f35a819a8d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a8107c80 0x7f35a81a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 --2- 172.21.15.5:0/3722292235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35a8102a20 0x7f35a819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 >> 172.21.15.5:0/3722292235 conn(0x7f35a80fc410 msgr2=0x7f35a81080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 shutdown_connections 2024-04-22T13:26:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.569+0000 7f35ad443640 1 -- 172.21.15.5:0/3722292235 wait complete. 2024-04-22T13:26:58.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 -- 172.21.15.5:0/2290381149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 msgr2=0x7f99a8104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/2290381149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a8104e90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9998009900 tx=0x7f999802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 -- 172.21.15.5:0/2290381149 shutdown_connections 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/2290381149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a8107d00 0x7f99a810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/2290381149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a81053d0 0x7f99a81077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/2290381149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a8104e90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 -- 172.21.15.5:0/2290381149 >> 172.21.15.5:0/2290381149 conn(0x7f99a80fc3f0 msgr2=0x7f99a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.682+0000 7f99ad1f4640 1 -- 172.21.15.5:0/2290381149 shutdown_connections 2024-04-22T13:26:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.683+0000 7f99ad1f4640 1 -- 172.21.15.5:0/2290381149 wait complete. 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.683+0000 7f99ad1f4640 1 Processor -- start 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.683+0000 7f99ad1f4640 1 -- start start 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a81053d0 0x7f99a8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a8107d00 0x7f99a819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810dae0 con 0x7f99a8102aa0 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810d960 con 0x7f99a8107d00 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810dc60 con 0x7f99a81053d0 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60330/0 (socket says 172.21.15.5:60330) 2024-04-22T13:26:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 -- 172.21.15.5:0/3662523862 learned_addr learned my addr 172.21.15.5:0/3662523862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a659c640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a81053d0 0x7f99a8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a759e640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a8107d00 0x7f99a819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 -- 172.21.15.5:0/3662523862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a81053d0 msgr2=0x7f99a8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a81053d0 0x7f99a8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 -- 172.21.15.5:0/3662523862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a8107d00 msgr2=0x7f99a819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a8107d00 0x7f99a819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 -- 172.21.15.5:0/3662523862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a819fde0 con 0x7f99a8102aa0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99a6d9d640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a81981c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9998006010 tx=0x7f99980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998043020 con 0x7f99a8102aa0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9998004aa0 con 0x7f99a8102aa0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a0070 con 0x7f99a8102aa0 2024-04-22T13:26:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998003c70 con 0x7f99a8102aa0 2024-04-22T13:26:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.684+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a05c0 con 0x7f99a8102aa0 2024-04-22T13:26:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.685+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a80fdee0 con 0x7f99a8102aa0 2024-04-22T13:26:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.686+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f9998046020 con 0x7f99a8102aa0 2024-04-22T13:26:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.687+0000 7f998ffff640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99900a3bb0 0x7f99900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.687+0000 7f99a659c640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99900a3bb0 0x7f99900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.687+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f99980f2780 con 0x7f99a8102aa0 2024-04-22T13:26:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.688+0000 7f99a659c640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99900a3bb0 0x7f99900a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99a81996d0 tx=0x7f999c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.692+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f999804d020 con 0x7f99a8102aa0 2024-04-22T13:26:58.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.878+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99a8062b50 con 0x7f99a8102aa0 2024-04-22T13:26:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.880+0000 7f998ffff640 1 -- 172.21.15.5:0/3662523862 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22549 (secure 0 0 0) 0x7f99980bf170 con 0x7f99a8102aa0 2024-04-22T13:26:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:26:58.877 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:26:15.450938+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:26:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99900a3bb0 msgr2=0x7f99900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99900a3bb0 0x7f99900a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99a81996d0 tx=0x7f999c00b390 comp rx=0 tx=0).stop 2024-04-22T13:26:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 msgr2=0x7f99a81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a81981c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9998006010 tx=0x7f99980042f0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 shutdown_connections 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f99900a3bb0 0x7f99900a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a8107d00 0x7f99a819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a81053d0 0x7f99a8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 --2- 172.21.15.5:0/3662523862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a8102aa0 0x7f99a81981c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 >> 172.21.15.5:0/3662523862 conn(0x7f99a80fc3f0 msgr2=0x7f99a8103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 shutdown_connections 2024-04-22T13:26:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.882+0000 7f99ad1f4640 1 -- 172.21.15.5:0/3662523862 wait complete. 2024-04-22T13:26:58.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:26:58.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:26:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 -- 172.21.15.5:0/3190766301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 msgr2=0x7f2b54107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3190766301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b54107740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009900 tx=0x7f2b5002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 -- 172.21.15.5:0/3190766301 shutdown_connections 2024-04-22T13:26:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3190766301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b54107740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3190766301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54107c80 0x7f2b5410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3190766301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54102a20 0x7f2b54104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 -- 172.21.15.5:0/3190766301 >> 172.21.15.5:0/3190766301 conn(0x7f2b540fc430 msgr2=0x7f2b540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 -- 172.21.15.5:0/3190766301 shutdown_connections 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:58.999+0000 7f2b5c606640 1 -- 172.21.15.5:0/3190766301 wait complete. 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 Processor -- start 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 -- start start 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54102a20 0x7f2b54102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b541009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54107c80 0x7f2b54100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410db70 con 0x7f2b54105350 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410d9f0 con 0x7f2b54107c80 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5c606640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410dcf0 con 0x7f2b54102a20 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b541009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b541009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56480/0 (socket says 172.21.15.5:56480) 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 -- 172.21.15.5:0/3445639295 learned_addr learned my addr 172.21.15.5:0/3445639295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5a3a2640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54102a20 0x7f2b54102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 -- 172.21.15.5:0/3445639295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54102a20 msgr2=0x7f2b54102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54102a20 0x7f2b54102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 -- 172.21.15.5:0/3445639295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54107c80 msgr2=0x7f2b54100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54107c80 0x7f2b54100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b59ba1640 1 -- 172.21.15.5:0/3445639295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b541017d0 con 0x7f2b54105350 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.000+0000 7f2b5a3a2640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54102a20 0x7f2b54102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.001+0000 7f2b59ba1640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b541009d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005ab0 tx=0x7f2b500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.001+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50043020 con 0x7f2b54105350 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.001+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b541a23a0 con 0x7f2b54105350 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.001+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b500054c0 con 0x7f2b54105350 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.001+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50003d10 con 0x7f2b54105350 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.001+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b541a28c0 con 0x7f2b54105350 2024-04-22T13:26:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.002+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b540fdf20 con 0x7f2b54105350 2024-04-22T13:26:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.003+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f2b50046020 con 0x7f2b54105350 2024-04-22T13:26:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.003+0000 7f2b4b7fe640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2b400a3bb0 0x7f2b400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.004+0000 7f2b5a3a2640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2b400a3bb0 0x7f2b400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.004+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f2b500f27e0 con 0x7f2b54105350 2024-04-22T13:26:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.004+0000 7f2b5a3a2640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2b400a3bb0 0x7f2b400a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009930 tx=0x7f2b44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.008+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b5004d020 con 0x7f2b54105350 2024-04-22T13:26:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.195+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b54062b50 con 0x7f2b54105350 2024-04-22T13:26:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.195+0000 7f2b4b7fe640 1 -- 172.21.15.5:0/3445639295 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f2b500bf1d0 con 0x7f2b54105350 2024-04-22T13:26:59.193 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2b400a3bb0 msgr2=0x7f2b400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2b400a3bb0 0x7f2b400a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009930 tx=0x7f2b44008040 comp rx=0 tx=0).stop 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 msgr2=0x7f2b541009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b541009d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005ab0 tx=0x7f2b500042f0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 shutdown_connections 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f2b400a3bb0 0x7f2b400a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b54105350 0x7f2b541009d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54107c80 0x7f2b54100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 --2- 172.21.15.5:0/3445639295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b54102a20 0x7f2b54102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 >> 172.21.15.5:0/3445639295 conn(0x7f2b540fc430 msgr2=0x7f2b54103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 shutdown_connections 2024-04-22T13:26:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.197+0000 7f2b5c606640 1 -- 172.21.15.5:0/3445639295 wait complete. 2024-04-22T13:26:59.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:59 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2776278084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:59.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:59 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3722292235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:59.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:26:59 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3662523862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:59.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:26:59.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:59 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2776278084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:59.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:59 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3722292235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:59.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:26:59 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3662523862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:59.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:59 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2776278084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:26:59.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:59 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3722292235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:59.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:26:59 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3662523862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 -- 172.21.15.5:0/1672975132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53201029a0 msgr2=0x7f53201092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 --2- 172.21.15.5:0/1672975132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53201029a0 0x7f53201092c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5318009ca0 tx=0x7f531802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 -- 172.21.15.5:0/1672975132 shutdown_connections 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 --2- 172.21.15.5:0/1672975132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53201029a0 0x7f53201092c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 --2- 172.21.15.5:0/1672975132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 0x7f5320102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 --2- 172.21.15.5:0/1672975132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5320100e90 0x7f53201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 -- 172.21.15.5:0/1672975132 >> 172.21.15.5:0/1672975132 conn(0x7f53200fc420 msgr2=0x7f53200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 -- 172.21.15.5:0/1672975132 shutdown_connections 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 -- 172.21.15.5:0/1672975132 wait complete. 2024-04-22T13:26:59.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 Processor -- start 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.315+0000 7f5325559640 1 -- start start 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 0x7f53201981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 0x7f5320198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53201029a0 0x7f532019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f532010db30 con 0x7f5320100e90 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f532010d9b0 con 0x7f53201029a0 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f532010dcb0 con 0x7f5320101fe0 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 0x7f53201981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 0x7f53201981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56486/0 (socket says 172.21.15.5:56486) 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 -- 172.21.15.5:0/2374056728 learned_addr learned my addr 172.21.15.5:0/2374056728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531e7fc640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 0x7f5320198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531f7fe640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53201029a0 0x7f532019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 -- 172.21.15.5:0/2374056728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 msgr2=0x7f5320198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 0x7f5320198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 -- 172.21.15.5:0/2374056728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53201029a0 msgr2=0x7f532019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53201029a0 0x7f532019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 -- 172.21.15.5:0/2374056728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532019fde0 con 0x7f5320100e90 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531e7fc640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 0x7f5320198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f531effd640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 0x7f53201981c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f531000d950 tx=0x7f531000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310017070 con 0x7f5320100e90 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201a00d0 con 0x7f5320100e90 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310009c30 con 0x7f5320100e90 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531000f620 con 0x7f5320100e90 2024-04-22T13:26:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.316+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201a0670 con 0x7f5320100e90 2024-04-22T13:26:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.317+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53201081c0 con 0x7f5320100e90 2024-04-22T13:26:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.318+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f5310010460 con 0x7f5320100e90 2024-04-22T13:26:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.319+0000 7f5307fff640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53080a3bb0 0x7f53080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.319+0000 7f531e7fc640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53080a3bb0 0x7f53080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.319+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f53100d0110 con 0x7f5320100e90 2024-04-22T13:26:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.320+0000 7f531e7fc640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53080a3bb0 0x7f53080a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53201996d0 tx=0x7f5314005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.323+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f531009caa0 con 0x7f5320100e90 2024-04-22T13:26:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.510+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f532002d5c0 con 0x7f5320100e90 2024-04-22T13:26:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.510+0000 7f5307fff640 1 -- 172.21.15.5:0/2374056728 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f5310009de0 con 0x7f5320100e90 2024-04-22T13:26:59.508 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.512+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53080a3bb0 msgr2=0x7f53080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.512+0000 7f5325559640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53080a3bb0 0x7f53080a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53201996d0 tx=0x7f5314005e70 comp rx=0 tx=0).stop 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.512+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 msgr2=0x7f53201981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.512+0000 7f5325559640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 0x7f53201981c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f531000d950 tx=0x7f531000de10 comp rx=0 tx=0).stop 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 shutdown_connections 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f53080a3bb0 0x7f53080a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53201029a0 0x7f532019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5320101fe0 0x7f5320198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 --2- 172.21.15.5:0/2374056728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5320100e90 0x7f53201981c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 >> 172.21.15.5:0/2374056728 conn(0x7f53200fc420 msgr2=0x7f5320105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 shutdown_connections 2024-04-22T13:26:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.513+0000 7f5325559640 1 -- 172.21.15.5:0/2374056728 wait complete. 2024-04-22T13:26:59.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:26:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 -- 172.21.15.5:0/3290637772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84101fb0 msgr2=0x7fdb841023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 --2- 172.21.15.5:0/3290637772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84101fb0 0x7fdb841023d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009900 tx=0x7fdb8002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 -- 172.21.15.5:0/3290637772 shutdown_connections 2024-04-22T13:26:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 --2- 172.21.15.5:0/3290637772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb84103b60 0x7fdb8410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 --2- 172.21.15.5:0/3290637772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb841031a0 0x7fdb84103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 --2- 172.21.15.5:0/3290637772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84101fb0 0x7fdb841023d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 -- 172.21.15.5:0/3290637772 >> 172.21.15.5:0/3290637772 conn(0x7fdb840fd520 msgr2=0x7fdb840ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 -- 172.21.15.5:0/3290637772 shutdown_connections 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.628+0000 7fdb8af52640 1 -- 172.21.15.5:0/3290637772 wait complete. 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 Processor -- start 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 -- start start 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84101fb0 0x7fdb84198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 0x7fdb84198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb84103b60 0x7fdb8419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410db40 con 0x7fdb841031a0 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410d9c0 con 0x7fdb84103b60 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb8af52640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410dcc0 con 0x7fdb84101fb0 2024-04-22T13:26:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 0x7fdb84198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 0x7fdb84198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56514/0 (socket says 172.21.15.5:56514) 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 -- 172.21.15.5:0/2026146221 learned_addr learned my addr 172.21.15.5:0/2026146221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb88cee640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84101fb0 0x7fdb84198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb894ef640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb84103b60 0x7fdb8419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 -- 172.21.15.5:0/2026146221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84101fb0 msgr2=0x7fdb84198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84101fb0 0x7fdb84198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 -- 172.21.15.5:0/2026146221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb84103b60 msgr2=0x7fdb8419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb84103b60 0x7fdb8419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.629+0000 7fdb7bfff640 1 -- 172.21.15.5:0/2026146221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8419fe60 con 0x7fdb841031a0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.630+0000 7fdb7bfff640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 0x7fdb84198780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400c970 tx=0x7fdb7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.630+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74007bb0 con 0x7fdb841031a0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.630+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb74007d50 con 0x7fdb841031a0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.630+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb841a0150 con 0x7fdb841031a0 2024-04-22T13:26:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.630+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400f450 con 0x7fdb841031a0 2024-04-22T13:26:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.630+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841a06a0 con 0x7fdb841031a0 2024-04-22T13:26:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.631+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84109370 con 0x7fdb841031a0 2024-04-22T13:26:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.632+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fdb74004f20 con 0x7fdb841031a0 2024-04-22T13:26:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.633+0000 7fdb79ffb640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.633+0000 7fdb88cee640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.633+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fdb740d0130 con 0x7fdb841031a0 2024-04-22T13:26:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.633+0000 7fdb88cee640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb8002c240 tx=0x7fdb8002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.637+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb74014030 con 0x7fdb841031a0 2024-04-22T13:26:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.821+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdb84062b50 con 0x7fdb841031a0 2024-04-22T13:26:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.822+0000 7fdb79ffb640 1 -- 172.21.15.5:0/2026146221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fdb7409cac0 con 0x7fdb841031a0 2024-04-22T13:26:59.819 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:26:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb6c0a3c30 msgr2=0x7fdb6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb8002c240 tx=0x7fdb8002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 msgr2=0x7fdb84198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 0x7fdb84198780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400c970 tx=0x7fdb7400ce30 comp rx=0 tx=0).stop 2024-04-22T13:26:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 shutdown_connections 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb841031a0 0x7fdb84198780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb84103b60 0x7fdb8419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 --2- 172.21.15.5:0/2026146221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84101fb0 0x7fdb84198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 >> 172.21.15.5:0/2026146221 conn(0x7fdb840fd520 msgr2=0x7fdb84106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 shutdown_connections 2024-04-22T13:26:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.824+0000 7fdb8af52640 1 -- 172.21.15.5:0/2026146221 wait complete. 2024-04-22T13:26:59.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:59.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 -- 172.21.15.5:0/2345699610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0100e80 msgr2=0x7f05b01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/2345699610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0100e80 0x7f05b01012a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009900 tx=0x7f05ac02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 -- 172.21.15.5:0/2345699610 shutdown_connections 2024-04-22T13:26:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/2345699610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05b0102990 0x7f05b01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/2345699610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b0101fd0 0x7f05b0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/2345699610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0100e80 0x7f05b01012a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 -- 172.21.15.5:0/2345699610 >> 172.21.15.5:0/2345699610 conn(0x7f05b00fc3f0 msgr2=0x7f05b00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 -- 172.21.15.5:0/2345699610 shutdown_connections 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 -- 172.21.15.5:0/2345699610 wait complete. 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 Processor -- start 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 -- start start 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05b0100e80 0x7f05b01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.937+0000 7f05b7c3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b0101fd0 0x7f05b0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b7c3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 0x7f05b019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b7c3a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010ca20 con 0x7f05b0102990 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b7c3a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010c8a0 con 0x7f05b0100e80 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b7c3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010cba0 con 0x7f05b0101fd0 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 0x7f05b019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 0x7f05b019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56530/0 (socket says 172.21.15.5:56530) 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 -- 172.21.15.5:0/687190096 learned_addr learned my addr 172.21.15.5:0/687190096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:26:59.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b51d5640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b0101fd0 0x7f05b0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b59d6640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05b0100e80 0x7f05b01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 -- 172.21.15.5:0/687190096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b0101fd0 msgr2=0x7f05b0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b0101fd0 0x7f05b0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 -- 172.21.15.5:0/687190096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05b0100e80 msgr2=0x7f05b01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05b0100e80 0x7f05b01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 -- 172.21.15.5:0/687190096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b019fe00 con 0x7f05b0102990 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b61d7640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 0x7f05b019f790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009fd0 tx=0x7f05a800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8017070 con 0x7f05b0102990 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8003e20 con 0x7f05b0102990 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b01a00f0 con 0x7f05b0102990 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8010460 con 0x7f05b0102990 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.938+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b01a0640 con 0x7f05b0102990 2024-04-22T13:26:59.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.939+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b01a08a0 con 0x7f05b0102990 2024-04-22T13:26:59.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.940+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f05a8003970 con 0x7f05b0102990 2024-04-22T13:26:59.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.941+0000 7f05a6ffd640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f059c0a3c30 0x7f059c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:59.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.941+0000 7f05b59d6640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f059c0a3c30 0x7f059c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:59.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.941+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f05a80cffa0 con 0x7f05b0102990 2024-04-22T13:26:59.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.942+0000 7f05b59d6640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f059c0a3c30 0x7f059c0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:59.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:26:59.945+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05a809c460 con 0x7f05b0102990 2024-04-22T13:27:00.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.174+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05b01a0b90 con 0x7f05b0102990 2024-04-22T13:27:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.175+0000 7f05a6ffd640 1 -- 172.21.15.5:0/687190096 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11905 (secure 0 0 0) 0x7f05a80a0760 con 0x7f05b0102990 2024-04-22T13:27:00.172 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:00.172 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:25:17.229740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24485,24469,24496,24557,24512]},"id":1}]} 2024-04-22T13:27:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-22T13:27:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f059c0a3c30 msgr2=0x7f059c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f059c0a3c30 0x7f059c0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 msgr2=0x7f05b019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 0x7f05b019f790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009fd0 tx=0x7f05a800b260 comp rx=0 tx=0).stop 2024-04-22T13:27:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 shutdown_connections 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f059c0a3c30 0x7f059c0a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0102990 0x7f05b019f790 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05b0101fd0 0x7f05b0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 --2- 172.21.15.5:0/687190096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05b0100e80 0x7f05b01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 >> 172.21.15.5:0/687190096 conn(0x7f05b00fc3f0 msgr2=0x7f05b0069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 shutdown_connections 2024-04-22T13:27:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.177+0000 7f05b7c3a640 1 -- 172.21.15.5:0/687190096 wait complete. 2024-04-22T13:27:00.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:00 smithi005 ceph-mon[28006]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:27:00.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:00 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3445639295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:00 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2374056728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:00 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2026146221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:00.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:00 smithi049 ceph-mon[31379]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:27:00.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:00 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3445639295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:00 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2374056728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:00 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2026146221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:00 smithi107 ceph-mon[31859]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:27:00.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:00 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3445639295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:00 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2374056728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:00 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2026146221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 -- 172.21.15.5:0/4172372325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4100e80 msgr2=0x7f4fa41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/4172372325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4100e80 0x7f4fa41012a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009900 tx=0x7f4f9402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 -- 172.21.15.5:0/4172372325 shutdown_connections 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/4172372325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa4102990 0x7f4fa41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/4172372325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fa4101fd0 0x7f4fa4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/4172372325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4100e80 0x7f4fa41012a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.290+0000 7f4faa3c3640 1 -- 172.21.15.5:0/4172372325 >> 172.21.15.5:0/4172372325 conn(0x7f4fa40fc3f0 msgr2=0x7f4fa40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 -- 172.21.15.5:0/4172372325 shutdown_connections 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 -- 172.21.15.5:0/4172372325 wait complete. 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 Processor -- start 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 -- start start 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fa4100e80 0x7f4fa41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa4101fd0 0x7f4fa4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 0x7f4fa419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa410c9d0 con 0x7f4fa4102990 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa410c850 con 0x7f4fa4101fd0 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.291+0000 7f4faa3c3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa410cb50 con 0x7f4fa4100e80 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 0x7f4fa419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 0x7f4fa419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56556/0 (socket says 172.21.15.5:56556) 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 -- 172.21.15.5:0/1845933429 learned_addr learned my addr 172.21.15.5:0/1845933429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa3fff640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fa4100e80 0x7f4fa41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa37fe640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa4101fd0 0x7f4fa4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 -- 172.21.15.5:0/1845933429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fa4100e80 msgr2=0x7f4fa41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fa4100e80 0x7f4fa41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 -- 172.21.15.5:0/1845933429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa4101fd0 msgr2=0x7f4fa4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa4101fd0 0x7f4fa4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 -- 172.21.15.5:0/1845933429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa419fdc0 con 0x7f4fa4102990 2024-04-22T13:27:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa8960640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 0x7f4fa419f750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00eb10 tx=0x7f4f9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c004b20 con 0x7f4fa4102990 2024-04-22T13:27:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa41a00b0 con 0x7f4fa4102990 2024-04-22T13:27:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9c004cc0 con 0x7f4fa4102990 2024-04-22T13:27:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c010670 con 0x7f4fa4102990 2024-04-22T13:27:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.292+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa41a0600 con 0x7f4fa4102990 2024-04-22T13:27:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.293+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa41a0920 con 0x7f4fa4102990 2024-04-22T13:27:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.295+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f4f9c005000 con 0x7f4fa4102990 2024-04-22T13:27:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.295+0000 7f4fa17fa640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f8c0a3bb0 0x7f4f8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.296+0000 7f4fa3fff640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f8c0a3bb0 0x7f4f8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.296+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f4f9c0d0cc0 con 0x7f4fa4102990 2024-04-22T13:27:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.296+0000 7f4fa3fff640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f8c0a3bb0 0x7f4f8c0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f94000c00 tx=0x7f4f94005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.299+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f9c053ff0 con 0x7f4fa4102990 2024-04-22T13:27:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.486+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fa41a0b90 con 0x7f4fa4102990 2024-04-22T13:27:00.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.488+0000 7f4fa17fa640 1 -- 172.21.15.5:0/1845933429 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22549 (secure 0 0 0) 0x7f4f9c07be20 con 0x7f4fa4102990 2024-04-22T13:27:00.485 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:00.485 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:26:15.450938+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f8c0a3bb0 msgr2=0x7f4f8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f8c0a3bb0 0x7f4f8c0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f94000c00 tx=0x7f4f94005be0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 msgr2=0x7f4fa419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 0x7f4fa419f750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00eb10 tx=0x7f4f9c00c2d0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 shutdown_connections 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f4f8c0a3bb0 0x7f4f8c0a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102990 0x7f4fa419f750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fa4101fd0 0x7f4fa4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 --2- 172.21.15.5:0/1845933429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fa4100e80 0x7f4fa41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 >> 172.21.15.5:0/1845933429 conn(0x7f4fa40fc3f0 msgr2=0x7f4fa40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 shutdown_connections 2024-04-22T13:27:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.490+0000 7f4faa3c3640 1 -- 172.21.15.5:0/1845933429 wait complete. 2024-04-22T13:27:00.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T13:25:10.226899+0000', 'modified': '2024-04-22T13:25:17.229740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24512, 'mds_2': 24496, 'mds_3': 24485, 'mds_4': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14550': {'gid': 14550, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/1096407808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 1096407808}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 1096407808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/373669901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 373669901}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 373669901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/4083506255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 4083506255}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 4083506255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6833/1584198825', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 1584198825}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 1584198825}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/2603097758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2603097758}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2603097758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.107:6837/207360580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 207360580}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 207360580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.49:6835/827214764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 827214764}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 827214764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.107:6839/3107882391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3107882391}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3107882391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6837/164218196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 164218196}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 164218196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6839/1393810832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 1393810832}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 1393810832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24485, 24469, 24496, 24557, 24512]} 2024-04-22T13:27:00.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T13:27:00.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T13:27:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 -- 172.21.15.5:0/4245674518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 msgr2=0x7f8f681092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/4245674518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f681092b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009900 tx=0x7f8f6002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 -- 172.21.15.5:0/4245674518 shutdown_connections 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/4245674518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f681092b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/4245674518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f68101fd0 0x7f8f68102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/4245674518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f68100e80 0x7f8f681012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 -- 172.21.15.5:0/4245674518 >> 172.21.15.5:0/4245674518 conn(0x7f8f680fc3f0 msgr2=0x7f8f680fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 -- 172.21.15.5:0/4245674518 shutdown_connections 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.605+0000 7f8f6cce0640 1 -- 172.21.15.5:0/4245674518 wait complete. 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 Processor -- start 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 -- start start 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f68100e80 0x7f8f681980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f68101fd0 0x7f8f68198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f6819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6810db80 con 0x7f8f68102990 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6810da00 con 0x7f8f68101fd0 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6cce0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6810dd00 con 0x7f8f68100e80 2024-04-22T13:27:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f6819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f6819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56562/0 (socket says 172.21.15.5:56562) 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f66d9d640 1 -- 172.21.15.5:0/136987604 learned_addr learned my addr 172.21.15.5:0/136987604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.606+0000 7f8f6659c640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f68100e80 0x7f8f681980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f68101fd0 0x7f8f68198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f66d9d640 1 -- 172.21.15.5:0/136987604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f68100e80 msgr2=0x7f8f681980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f66d9d640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f68100e80 0x7f8f681980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f66d9d640 1 -- 172.21.15.5:0/136987604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f68101fd0 msgr2=0x7f8f68198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f66d9d640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f68101fd0 0x7f8f68198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f66d9d640 1 -- 172.21.15.5:0/136987604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6819fcf0 con 0x7f8f68102990 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f66d9d640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f6819f680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f6002c170 tx=0x7f8f60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f577fe640 1 -- 172.21.15.5:0/136987604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60043020 con 0x7f8f68102990 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6819ff80 con 0x7f8f68102990 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f577fe640 1 -- 172.21.15.5:0/136987604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f600052b0 con 0x7f8f68102990 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f577fe640 1 -- 172.21.15.5:0/136987604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60003d00 con 0x7f8f68102990 2024-04-22T13:27:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.607+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f681a0440 con 0x7f8f68102990 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.609+0000 7f8f577fe640 1 -- 172.21.15.5:0/136987604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f8f60046020 con 0x7f8f68102990 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 Processor -- start 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 -- start start 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c753e40 0x56130c754260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 0x56130c754cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c7552d0 0x56130c75beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56130c6b8b10 con 0x56130c754830 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56130c656240 con 0x56130c753e40 2024-04-22T13:27:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56130c6b4ad0 con 0x56130c7552d0 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 0x56130c754cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 0x56130c754cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56564/0 (socket says 172.21.15.5:56564) 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f6659c640 1 -- 172.21.15.5:0/1415712381 learned_addr learned my addr 172.21.15.5:0/1415712381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f577fe640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8f500a3bb0 0x7f8f500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.610+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c7552d0 0x56130c75beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c753e40 0x56130c754260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f66d9d640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8f500a3bb0 0x7f8f500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f6659c640 1 -- 172.21.15.5:0/1415712381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c7552d0 msgr2=0x56130c75beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f6659c640 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c7552d0 0x56130c75beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f6659c640 1 -- 172.21.15.5:0/1415712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c753e40 msgr2=0x56130c754260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f6659c640 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c753e40 0x56130c754260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f6659c640 1 -- 172.21.15.5:0/1415712381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130c3cb2d0 con 0x56130c754830 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c753e40 0x56130c754260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f577fe640 1 -- 172.21.15.5:0/136987604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f8f600f2a10 con 0x7f8f68102990 2024-04-22T13:27:00.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f6659c640 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 0x56130c754cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800af40 tx=0x7f8f5800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c013231b5097d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f66d9d640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8f500a3bb0 0x7f8f500a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f600bb100 tx=0x7f8f600b96c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f54ff9640 1 -- 172.21.15.5:0/1415712381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5801d070 con 0x56130c754830 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f54ff9640 1 -- 172.21.15.5:0/1415712381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58010db0 con 0x56130c754830 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.611+0000 7f8f54ff9640 1 -- 172.21.15.5:0/1415712381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58018da0 con 0x56130c754830 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1415712381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 msgr2=0x56130c754cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 0x56130c754cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800af40 tx=0x7f8f5800f850 comp rx=0 tx=0).stop 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1415712381 shutdown_connections 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c754830 0x56130c754cb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c7552d0 0x56130c75beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1415712381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c753e40 0x56130c754260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1415712381 >> 172.21.15.5:0/1415712381 conn(0x56130c74ff40 msgr2=0x56130c74e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1415712381 shutdown_connections 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1415712381 wait complete. 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 -- start start 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.612+0000 7f8f6f9af740 1 Processor -- start 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c75de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 0x56130c765400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56130c75c4b0 con 0x56130c7552d0 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56130c641190 con 0x56130c754830 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56130c75c630 con 0x56130c753e40 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 10 client.0 _ll_register_callbacks cb 0x56130c63c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 0x56130c765400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 0x56130c765400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56576/0 (socket says 172.21.15.5:56576) 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 -- 172.21.15.5:0/3107243062 learned_addr learned my addr 172.21.15.5:0/3107243062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c75de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6659c640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 -- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 msgr2=0x56130c75de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c75de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 -- 172.21.15.5:0/3107243062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 msgr2=0x56130c75e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 -- 172.21.15.5:0/3107243062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130c4956b0 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6659c640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 0x56130c765400 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8f681995e0 tx=0x7f8f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7782d1f67d7e92c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f4dffb640 1 -- 172.21.15.5:0/3107243062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c010840 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f4dffb640 1 -- 172.21.15.5:0/3107243062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f8f5c0109e0 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f58010db0 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f4dffb640 1 -- 172.21.15.5:0/3107243062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f5c026050 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.613+0000 7f8f6f9af740 10 client.15252 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56130c768890 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f4dffb640 1 -- 172.21.15.5:0/3107243062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58010db0 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f4dffb640 1 -- 172.21.15.5:0/3107243062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f5c00fae0 con 0x56130c7552d0 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 10 client.15252 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 20 client.15252 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c76afb0 0x56130c76c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 4 client.15252 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56130c6c6530 con 0x56130c76afb0 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.614+0000 7f8f6659c640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c76afb0 0x56130c76c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.615+0000 7f8f6659c640 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c76afb0 0x56130c76c420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b36f69a961490cbe server_cookie=d5f2086db24a25be in_seq=0 out_seq=0 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.615+0000 7f8f4dffb640 10 client.15252 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.615+0000 7f8f4dffb640 1 -- 172.21.15.5:0/3107243062 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56130c6c6530 con 0x56130c76afb0 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.615+0000 7f8f4dffb640 10 client.15252 handle_command_reply: tid=0 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.615+0000 7f8f6f9af740 1 client.15252 shutdown 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 msgr2=0x56130c765400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 0x56130c765400 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8f681995e0 tx=0x7f8f5c0046c0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 shutdown_connections 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c76afb0 0x56130c76c420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c7552d0 0x56130c765400 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f4dffb640 0 client.15252 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:00.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/3107243062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c75de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 >> 172.21.15.5:0/3107243062 conn(0x56130c74ff40 msgr2=0x56130c74da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 shutdown_connections 2024-04-22T13:27:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 1 -- 172.21.15.5:0/3107243062 wait complete. 2024-04-22T13:27:00.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.616+0000 7f8f6f9af740 20 client.15252 trim_cache size 0 max 16384 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 Processor -- start 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 -- start start 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 0x56130c75de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c7653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56130c76a760 con 0x56130c74ff40 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56130c76ae30 con 0x56130c754830 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56130c76b500 con 0x56130c753e40 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 0x56130c75de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 0x56130c75de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56592/0 (socket says 172.21.15.5:56592) 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 -- 172.21.15.5:0/2232223936 learned_addr learned my addr 172.21.15.5:0/2232223936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f6659c640 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c7653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 -- 172.21.15.5:0/2232223936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 msgr2=0x56130c7653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c7653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 -- 172.21.15.5:0/2232223936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 msgr2=0x56130c75e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.634+0000 7f8f65d9b640 1 -- 172.21.15.5:0/2232223936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130c76a590 con 0x56130c74ff40 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 0x56130c75de50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e176051a94c08b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f4ffff640 1 -- 172.21.15.5:0/2232223936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c02e070 con 0x56130c74ff40 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f4ffff640 1 -- 172.21.15.5:0/2232223936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c0044f0 con 0x56130c74ff40 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f4ffff640 1 -- 172.21.15.5:0/2232223936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00de20 con 0x56130c74ff40 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 -- 172.21.15.5:0/2232223936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 msgr2=0x56130c75de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 0x56130c75de50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c011bb0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 -- 172.21.15.5:0/2232223936 shutdown_connections 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c754830 0x56130c75e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c753e40 0x56130c7653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/2232223936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c74ff40 0x56130c75de50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 -- 172.21.15.5:0/2232223936 >> 172.21.15.5:0/2232223936 conn(0x56130c75c160 msgr2=0x56130c749800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 -- 172.21.15.5:0/2232223936 shutdown_connections 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.635+0000 7f8f6f9af740 1 -- 172.21.15.5:0/2232223936 wait complete. 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 -- start start 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 Processor -- start 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c74ff40 0x56130c760eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 0x56130c7613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c754830 0x56130c7643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56130c76c2a0 con 0x56130c753e40 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56130c76bbd0 con 0x56130c74ff40 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56130c76c970 con 0x56130c754830 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6f9af740 10 client.0 _ll_register_callbacks cb 0x7f8f5c00bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 0x56130c7613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 0x56130c7613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56598/0 (socket says 172.21.15.5:56598) 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 -- 172.21.15.5:0/1801982088 learned_addr learned my addr 172.21.15.5:0/1801982088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c754830 0x56130c7643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c74ff40 0x56130c760eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 -- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c754830 msgr2=0x56130c7643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c754830 0x56130c7643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 -- 172.21.15.5:0/1801982088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c74ff40 msgr2=0x56130c760eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c74ff40 0x56130c760eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 -- 172.21.15.5:0/1801982088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130c7649b0 con 0x56130c753e40 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c74ff40 0x56130c760eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f6659c640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 0x56130c7613f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800fd50 tx=0x7f8f5800a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b8da92df48bf6d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.636+0000 7f8f4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f4dffb640 1 -- 172.21.15.5:0/1801982088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5802c020 con 0x56130c753e40 2024-04-22T13:27:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f4dffb640 1 -- 172.21.15.5:0/1801982088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f8f58004be0 con 0x56130c753e40 2024-04-22T13:27:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f5c0044f0 con 0x56130c753e40 2024-04-22T13:27:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f4dffb640 1 -- 172.21.15.5:0/1801982088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f5801d040 con 0x56130c753e40 2024-04-22T13:27:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 10 client.15258 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56130c7d12f0 con 0x56130c753e40 2024-04-22T13:27:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f4dffb640 1 -- 172.21.15.5:0/1801982088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c0044f0 con 0x56130c753e40 2024-04-22T13:27:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f4dffb640 1 -- 172.21.15.5:0/1801982088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f58010a70 con 0x56130c753e40 2024-04-22T13:27:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 20 client.15258 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c7552d0 0x56130c7d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 4 client.15258 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "status"}) -- 0x56130c6c03a0 con 0x56130c7552d0 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.637+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c7552d0 0x56130c7d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f4dffb640 10 client.15258 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f65d9b640 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c7552d0 0x56130c7d48a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a191d9fc56c8135 server_cookie=83b3f4be28444b8f in_seq=0 out_seq=0 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f4dffb640 1 -- 172.21.15.5:0/1801982088 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56130c6c03a0 con 0x56130c7552d0 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f4dffb640 10 client.15258 handle_command_reply: tid=0 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 client.15258 shutdown 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 msgr2=0x56130c7613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 0x56130c7613f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800fd50 tx=0x7f8f5800a390 comp rx=0 tx=0).stop 2024-04-22T13:27:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 shutdown_connections 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56130c7552d0 0x56130c7d48a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56130c753e40 0x56130c7613f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56130c754830 0x56130c7643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 --2- 172.21.15.5:0/1801982088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56130c74ff40 0x56130c760eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f4dffb640 0 client.15258 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 >> 172.21.15.5:0/1801982088 conn(0x56130c75c160 msgr2=0x56130c75b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 shutdown_connections 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 1 -- 172.21.15.5:0/1801982088 wait complete. 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.638+0000 7f8f6f9af740 20 client.15258 trim_cache size 0 max 16384 2024-04-22T13:27:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24469, 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 103.702581936, 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 128.886649734 2024-04-22T13:27:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8f500a3bb0 msgr2=0x7f8f500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8f500a3bb0 0x7f8f500a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f600bb100 tx=0x7f8f600b96c0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 msgr2=0x7f8f6819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f6819f680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f6002c170 tx=0x7f8f60004270 comp rx=0 tx=0).stop 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 shutdown_connections 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f8f500a3bb0 0x7f8f500a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f68102990 0x7f8f6819f680 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f68101fd0 0x7f8f68198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 --2- 172.21.15.5:0/136987604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f68100e80 0x7f8f681980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 >> 172.21.15.5:0/136987604 conn(0x7f8f680fc3f0 msgr2=0x7f8f68105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 shutdown_connections 2024-04-22T13:27:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.639+0000 7f8f6cce0640 1 -- 172.21.15.5:0/136987604 wait complete. 2024-04-22T13:27:00.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 -- 172.21.15.5:0/775163435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102a20 msgr2=0x7f44d4104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 --2- 172.21.15.5:0/775163435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102a20 0x7f44d4104e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009ca0 tx=0x7f44c402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 -- 172.21.15.5:0/775163435 shutdown_connections 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 --2- 172.21.15.5:0/775163435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4107c80 0x7f44d410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 --2- 172.21.15.5:0/775163435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d4105350 0x7f44d4107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 --2- 172.21.15.5:0/775163435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4102a20 0x7f44d4104e10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 -- 172.21.15.5:0/775163435 >> 172.21.15.5:0/775163435 conn(0x7f44d40fc430 msgr2=0x7f44d40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 -- 172.21.15.5:0/775163435 shutdown_connections 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 -- 172.21.15.5:0/775163435 wait complete. 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 Processor -- start 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.753+0000 7f44d9350640 1 -- start start 2024-04-22T13:27:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d9350640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d4102a20 0x7f44d4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d9350640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4105350 0x7f44d41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d9350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 0x7f44d419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d9350640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410dae0 con 0x7f44d4107c80 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d9350640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d960 con 0x7f44d4105350 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d9350640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410dc60 con 0x7f44d4102a20 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 0x7f44d419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 0x7f44d419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56610/0 (socket says 172.21.15.5:56610) 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 -- 172.21.15.5:0/3935546399 learned_addr learned my addr 172.21.15.5:0/3935546399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d27fc640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4105350 0x7f44d41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d4102a20 0x7f44d4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 -- 172.21.15.5:0/3935546399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d4102a20 msgr2=0x7f44d4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d4102a20 0x7f44d4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 -- 172.21.15.5:0/3935546399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4105350 msgr2=0x7f44d41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4105350 0x7f44d41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 -- 172.21.15.5:0/3935546399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419fda0 con 0x7f44d4107c80 2024-04-22T13:27:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44d37fe640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 0x7f44d419f730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c970 tx=0x7f44cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.754+0000 7f44bbfff640 1 -- 172.21.15.5:0/3935546399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc007bb0 con 0x7f44d4107c80 2024-04-22T13:27:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.755+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d41a0090 con 0x7f44d4107c80 2024-04-22T13:27:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.755+0000 7f44bbfff640 1 -- 172.21.15.5:0/3935546399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44cc007d50 con 0x7f44d4107c80 2024-04-22T13:27:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.755+0000 7f44bbfff640 1 -- 172.21.15.5:0/3935546399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc018770 con 0x7f44d4107c80 2024-04-22T13:27:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.755+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41a05e0 con 0x7f44d4107c80 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44bbfff640 1 -- 172.21.15.5:0/3935546399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f44cc00f990 con 0x7f44d4107c80 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 Processor -- start 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 -- start start 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44bbfff640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f44bc0a3bb0 0x7f44bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3b830 0x562068b3bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b42eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562068a9fb10 con 0x562068b3c2d0 2024-04-22T13:27:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562068a3d240 con 0x562068b3b830 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44dc01c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562068a9bad0 con 0x562068b3ae40 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b42eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f44bc0a3bb0 0x7f44bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b42eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56626/0 (socket says 172.21.15.5:56626) 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.757+0000 7f44d37fe640 1 -- 172.21.15.5:0/3340744118 learned_addr learned my addr 172.21.15.5:0/3340744118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44bbfff640 1 -- 172.21.15.5:0/3935546399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f44cc0d0460 con 0x7f44d4107c80 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d27fc640 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3b830 0x562068b3bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d27fc640 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d37fe640 1 -- 172.21.15.5:0/3340744118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 msgr2=0x562068b3b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d37fe640 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b3b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d37fe640 1 -- 172.21.15.5:0/3340744118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3b830 msgr2=0x562068b3bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d37fe640 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3b830 0x562068b3bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d37fe640 1 -- 172.21.15.5:0/3340744118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620687b22d0 con 0x562068b3c2d0 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d27fc640 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3b830 0x562068b3bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d37fe640 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b42eb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0a4e30 tx=0x7f44cc00fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a77667fe3c70d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f44bc0a3bb0 0x7f44bc0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005e30 tx=0x7f44c4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44b97fa640 1 -- 172.21.15.5:0/3340744118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0b6050 con 0x562068b3c2d0 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44b97fa640 1 -- 172.21.15.5:0/3340744118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44cc09cdf0 con 0x562068b3c2d0 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.758+0000 7f44b97fa640 1 -- 172.21.15.5:0/3340744118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0b5e70 con 0x562068b3c2d0 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- 172.21.15.5:0/3340744118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 msgr2=0x562068b42eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b42eb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0a4e30 tx=0x7f44cc00fd30 comp rx=0 tx=0).stop 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- 172.21.15.5:0/3340744118 shutdown_connections 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b42eb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3b830 0x562068b3bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- 172.21.15.5:0/3340744118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- 172.21.15.5:0/3340744118 >> 172.21.15.5:0/3340744118 conn(0x562068b36f40 msgr2=0x562068b35490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- 172.21.15.5:0/3340744118 shutdown_connections 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- 172.21.15.5:0/3340744118 wait complete. 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- start start 2024-04-22T13:27:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 Processor -- start 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b44ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b45420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3c2d0 0x562068b4c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562068b434b0 con 0x562068b3b830 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562068a28190 con 0x562068b3c2d0 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562068b43630 con 0x562068b3ae40 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.759+0000 7f44dc01c740 10 client.0 _ll_register_callbacks cb 0x562068a238b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b45420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b45420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56636/0 (socket says 172.21.15.5:56636) 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 -- 172.21.15.5:0/2402862773 learned_addr learned my addr 172.21.15.5:0/2402862773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b44ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d37fe640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3c2d0 0x562068b4c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 -- 172.21.15.5:0/2402862773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 msgr2=0x562068b44ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b44ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 -- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3c2d0 msgr2=0x562068b4c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3c2d0 0x562068b4c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 -- 172.21.15.5:0/2402862773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206887c6b0 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44d27fc640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b45420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44c800bfc0 tx=0x7f44c80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a4cf9f03e2b6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44b6ff5640 1 -- 172.21.15.5:0/2402862773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8018030 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44b6ff5640 1 -- 172.21.15.5:0/2402862773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f44c8004030 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44cc09cdf0 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44b6ff5640 1 -- 172.21.15.5:0/2402862773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44c8004230 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44dc01c740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562068b4f8f0 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.760+0000 7f44b6ff5640 1 -- 172.21.15.5:0/2402862773 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44cc09cdf0 con 0x562068b3b830 2024-04-22T13:27:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44b6ff5640 1 -- 172.21.15.5:0/2402862773 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44c8003820 con 0x562068b3b830 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44dc01c740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44dc01c740 10 client.15270 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44dc01c740 20 client.15270 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44dc01c740 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068b53440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44dc01c740 4 client.15270 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562068aad530 con 0x562068b51fd0 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068b53440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44b6ff5640 10 client.15270 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.761+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068b53440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba4927474841bd71 server_cookie=d9a79cafafa6549 in_seq=0 out_seq=0 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44b6ff5640 1 -- 172.21.15.5:0/2402862773 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562068aad530 con 0x562068b51fd0 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44b6ff5640 10 client.15270 handle_command_reply: tid=0 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 client.15270 shutdown 2024-04-22T13:27:00.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 msgr2=0x562068b45420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b45420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44c800bfc0 tx=0x7f44c80128e0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 shutdown_connections 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068b53440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b45420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44b6ff5640 0 client.15270 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3c2d0 0x562068b4c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 --2- 172.21.15.5:0/2402862773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3ae40 0x562068b44ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 >> 172.21.15.5:0/2402862773 conn(0x562068b36f40 msgr2=0x562068b34a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 shutdown_connections 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.762+0000 7f44dc01c740 1 -- 172.21.15.5:0/2402862773 wait complete. 2024-04-22T13:27:00.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.763+0000 7f44dc01c740 20 client.15270 trim_cache size 0 max 16384 2024-04-22T13:27:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.780+0000 7f44dc01c740 1 Processor -- start 2024-04-22T13:27:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 -- start start 2024-04-22T13:27:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3b830 0x562068b3fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b4c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562068b51780 con 0x562068b3c2d0 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562068b51e50 con 0x562068b3ae40 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44dc01c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562068b52520 con 0x562068b3b830 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b4c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b4c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56652/0 (socket says 172.21.15.5:56652) 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 -- 172.21.15.5:0/1060596503 learned_addr learned my addr 172.21.15.5:0/1060596503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d27fc640 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3b830 0x562068b3fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 -- 172.21.15.5:0/1060596503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3b830 msgr2=0x562068b3fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3b830 0x562068b3fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 -- 172.21.15.5:0/1060596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 msgr2=0x562068b3f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b3f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 -- 172.21.15.5:0/1060596503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562068b30e70 con 0x562068b3c2d0 2024-04-22T13:27:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44d37fe640 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b4c420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f44cc087010 tx=0x7f44cc011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3272f929eabd3b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44b8ff9640 1 -- 172.21.15.5:0/1060596503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0a0aa0 con 0x562068b3c2d0 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44b8ff9640 1 -- 172.21.15.5:0/1060596503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44cc09c6f0 con 0x562068b3c2d0 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.781+0000 7f44b8ff9640 1 -- 172.21.15.5:0/1060596503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc07b120 con 0x562068b3c2d0 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 -- 172.21.15.5:0/1060596503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 msgr2=0x562068b4c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b4c420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f44cc087010 tx=0x7f44cc011040 comp rx=0 tx=0).stop 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 -- 172.21.15.5:0/1060596503 shutdown_connections 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3c2d0 0x562068b4c420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3b830 0x562068b3fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 --2- 172.21.15.5:0/1060596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 -- 172.21.15.5:0/1060596503 >> 172.21.15.5:0/1060596503 conn(0x562068b36f40 msgr2=0x562068b42e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 -- 172.21.15.5:0/1060596503 shutdown_connections 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 -- 172.21.15.5:0/1060596503 wait complete. 2024-04-22T13:27:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 -- start start 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.782+0000 7f44dc01c740 1 Processor -- start 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3c2d0 0x562068b4ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562068b532c0 con 0x562068b3b830 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562068b52bf0 con 0x562068b3ae40 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562068b53990 con 0x562068b3c2d0 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 10 client.0 _ll_register_callbacks cb 0x7f44c800b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b47d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56658/0 (socket says 172.21.15.5:56658) 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 -- 172.21.15.5:0/217807593 learned_addr learned my addr 172.21.15.5:0/217807593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d37fe640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3c2d0 0x562068b4ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 -- 172.21.15.5:0/217807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3c2d0 msgr2=0x562068b4ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3c2d0 0x562068b4ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 -- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 msgr2=0x562068b477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 -- 172.21.15.5:0/217807593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562068b4b310 con 0x562068b3b830 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44d27fc640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b47d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f44c8003ce0 tx=0x7f44c800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59a224d7e1eee15 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44b6ff5640 1 -- 172.21.15.5:0/217807593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c802c070 con 0x562068b3b830 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44b6ff5640 1 -- 172.21.15.5:0/217807593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f44c80035f0 con 0x562068b3b830 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44cc09c6f0 con 0x562068b3b830 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44b6ff5640 1 -- 172.21.15.5:0/217807593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44c8018040 con 0x562068b3b830 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.783+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562068b4c5b0 con 0x562068b3b830 2024-04-22T13:27:00.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44b6ff5640 1 -- 172.21.15.5:0/217807593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44cc09c6f0 con 0x562068b3b830 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44b6ff5640 1 -- 172.21.15.5:0/217807593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44c803b450 con 0x562068b3b830 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44dc01c740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44dc01c740 10 client.15276 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44dc01c740 20 client.15276 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44dc01c740 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068bbb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44dc01c740 4 client.15276 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "status"}) -- 0x562068aa73a0 con 0x562068b51fd0 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068bbb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44b6ff5640 10 client.15276 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.784+0000 7f44d2ffd640 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068bbb500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8223b69964c56442 server_cookie=7c85d80d7c412460 in_seq=0 out_seq=0 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44b6ff5640 1 -- 172.21.15.5:0/217807593 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562068aa73a0 con 0x562068b51fd0 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44b6ff5640 10 client.15276 handle_command_reply: tid=0 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 client.15276 shutdown 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 msgr2=0x562068b47d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b47d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f44c8003ce0 tx=0x7f44c800b4a0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 shutdown_connections 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562068b51fd0 0x562068bbb500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562068b3b830 0x562068b47d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44b6ff5640 0 client.15276 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562068b3c2d0 0x562068b4ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 --2- 172.21.15.5:0/217807593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562068b3ae40 0x562068b477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 >> 172.21.15.5:0/217807593 conn(0x562068b36f40 msgr2=0x562068b35350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 shutdown_connections 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 1 -- 172.21.15.5:0/217807593 wait complete. 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44dc01c740 20 client.15276 trim_cache size 0 max 16384 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:00.789 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24485, 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 104.85911384400001, 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 134.36678838700001 2024-04-22T13:27:00.790 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f44bc0a3bb0 msgr2=0x7f44bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f44bc0a3bb0 0x7f44bc0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005e30 tx=0x7f44c4000f40 comp rx=0 tx=0).stop 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 msgr2=0x7f44d419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 0x7f44d419f730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c970 tx=0x7f44cc00ce30 comp rx=0 tx=0).stop 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 shutdown_connections 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f44bc0a3bb0 0x7f44bc0a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4107c80 0x7f44d419f730 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d4105350 0x7f44d41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 --2- 172.21.15.5:0/3935546399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44d4102a20 0x7f44d4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.785+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 >> 172.21.15.5:0/3935546399 conn(0x7f44d40fc430 msgr2=0x7f44d41032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.786+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 shutdown_connections 2024-04-22T13:27:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.786+0000 7f44d9350640 1 -- 172.21.15.5:0/3935546399 wait complete. 2024-04-22T13:27:00.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T13:27:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/3480391115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb18100ea0 msgr2=0x7fdb181012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/3480391115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb18100ea0 0x7fdb181012c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009900 tx=0x7fdb0802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/3480391115 shutdown_connections 2024-04-22T13:27:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/3480391115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb181029b0 0x7fdb181092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/3480391115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb18101ff0 0x7fdb18102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/3480391115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb18100ea0 0x7fdb181012c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/3480391115 >> 172.21.15.5:0/3480391115 conn(0x7fdb180fc430 msgr2=0x7fdb180fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/3480391115 shutdown_connections 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.899+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/3480391115 wait complete. 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 Processor -- start 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 -- start start 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb18100ea0 0x7fdb181970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb18101ff0 0x7fdb18197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 0x7fdb1819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1810c9d0 con 0x7fdb181029b0 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1810c850 con 0x7fdb18101ff0 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1fcd2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1810cb50 con 0x7fdb18100ea0 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 0x7fdb1819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 0x7fdb1819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56690/0 (socket says 172.21.15.5:56690) 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 -- 172.21.15.5:0/1175725778 learned_addr learned my addr 172.21.15.5:0/1175725778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb18101ff0 0x7fdb18197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb18100ea0 0x7fdb181970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 -- 172.21.15.5:0/1175725778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb18100ea0 msgr2=0x7fdb181970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb18100ea0 0x7fdb181970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 -- 172.21.15.5:0/1175725778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb18101ff0 msgr2=0x7fdb18197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb18101ff0 0x7fdb18197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 -- 172.21.15.5:0/1175725778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1819ed00 con 0x7fdb181029b0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb18101ff0 0x7fdb18197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb1e26f640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 0x7fdb1819e690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000cad0 tx=0x7fdb1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.900+0000 7fdb0effd640 1 -- 172.21.15.5:0/1175725778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000ae20 con 0x7fdb181029b0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.901+0000 7fdb0effd640 1 -- 172.21.15.5:0/1175725778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb10005020 con 0x7fdb181029b0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.901+0000 7fdb0effd640 1 -- 172.21.15.5:0/1175725778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000f6b0 con 0x7fdb181029b0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.901+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1819eff0 con 0x7fdb181029b0 2024-04-22T13:27:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.901+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1819f4b0 con 0x7fdb181029b0 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.903+0000 7fdb229a6740 1 Processor -- start 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.903+0000 7fdb0effd640 1 -- 172.21.15.5:0/1175725778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fdb100051d0 con 0x7fdb181029b0 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.903+0000 7fdb229a6740 1 -- start start 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.903+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a1e40 0x5617290a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x5617290a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb229a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561729006b10 con 0x5617290a32d0 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb229a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561728fa4240 con 0x5617290a1e40 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb229a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561729002ad0 con 0x5617290a2830 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x5617290a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x5617290a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56692/0 (socket says 172.21.15.5:56692) 2024-04-22T13:27:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 -- 172.21.15.5:0/444750138 learned_addr learned my addr 172.21.15.5:0/444750138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a1e40 0x5617290a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 -- 172.21.15.5:0/444750138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 msgr2=0x5617290a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb0effd640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb040a3c30 0x7fdb040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 -- 172.21.15.5:0/444750138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a1e40 msgr2=0x5617290a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a1e40 0x5617290a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 -- 172.21.15.5:0/444750138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561728d192d0 con 0x5617290a32d0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1e26f640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb040a3c30 0x7fdb040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x5617290a9eb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400be90 tx=0x7fdb1400b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58aeddf11e4a6de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb0effd640 1 -- 172.21.15.5:0/1175725778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fdb100d0430 con 0x7fdb181029b0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb03fff640 1 -- 172.21.15.5:0/444750138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14019030 con 0x5617290a32d0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb03fff640 1 -- 172.21.15.5:0/444750138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb140054d0 con 0x5617290a32d0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.904+0000 7fdb03fff640 1 -- 172.21.15.5:0/444750138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400f7e0 con 0x5617290a32d0 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 -- 172.21.15.5:0/444750138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 msgr2=0x5617290a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb1e26f640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb040a3c30 0x7fdb040a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb10098b80 tx=0x7fdb10066d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x5617290a9eb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400be90 tx=0x7fdb1400b550 comp rx=0 tx=0).stop 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 -- 172.21.15.5:0/444750138 shutdown_connections 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x5617290a9eb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 --2- 172.21.15.5:0/444750138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a1e40 0x5617290a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 -- 172.21.15.5:0/444750138 >> 172.21.15.5:0/444750138 conn(0x56172909df40 msgr2=0x56172909c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 -- 172.21.15.5:0/444750138 shutdown_connections 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.905+0000 7fdb229a6740 1 -- 172.21.15.5:0/444750138 wait complete. 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 -- start start 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 Processor -- start 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x5617290ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a32d0 0x5617290b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617290aa4b0 con 0x5617290a1e40 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561728f8f190 con 0x5617290a2830 2024-04-22T13:27:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5617290aa630 con 0x5617290a32d0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb229a6740 10 client.0 _ll_register_callbacks cb 0x561728f8a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56706/0 (socket says 172.21.15.5:56706) 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb1d26d640 1 -- 172.21.15.5:0/2213064011 learned_addr learned my addr 172.21.15.5:0/2213064011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x5617290ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a32d0 0x5617290b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.906+0000 7fdb1d26d640 1 -- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a32d0 msgr2=0x5617290b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a32d0 0x5617290b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb1d26d640 1 -- 172.21.15.5:0/2213064011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 msgr2=0x5617290ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x5617290ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb1d26d640 1 -- 172.21.15.5:0/2213064011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561728de36b0 con 0x5617290a1e40 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x5617290ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290ae090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006010 tx=0x7fdb1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9ce288f946fa25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb017fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb017fa640 1 -- 172.21.15.5:0/2213064011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14019040 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb017fa640 1 -- 172.21.15.5:0/2213064011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fdb14013040 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb140054d0 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb017fa640 1 -- 172.21.15.5:0/2213064011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb14013240 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 10 client.15288 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5617290b8980 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb017fa640 1 -- 172.21.15.5:0/2213064011 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb140054d0 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb017fa640 1 -- 172.21.15.5:0/2213064011 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdb1401e580 con 0x5617290a1e40 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 10 client.15288 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 20 client.15288 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617290bad60 0x5617290bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 4 client.15288 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.907+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561729014530 con 0x5617290bad60 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.908+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617290bad60 0x5617290bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.908+0000 7fdb017fa640 10 client.15288 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.908+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617290bad60 0x5617290bc1d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3c41105eb68e79 server_cookie=a54f3ab9d3f69ce0 in_seq=0 out_seq=0 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb017fa640 1 -- 172.21.15.5:0/2213064011 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561729014530 con 0x5617290bad60 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb017fa640 10 client.15288 handle_command_reply: tid=0 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 client.15288 shutdown 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 msgr2=0x5617290ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290ae090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006010 tx=0x7fdb1400b330 comp rx=0 tx=0).stop 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 shutdown_connections 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617290bad60 0x5617290bc1d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a32d0 0x5617290b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x5617290ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 --2- 172.21.15.5:0/2213064011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290ae090 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb017fa640 0 client.15288 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 >> 172.21.15.5:0/2213064011 conn(0x56172909df40 msgr2=0x56172909ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 shutdown_connections 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 1 -- 172.21.15.5:0/2213064011 wait complete. 2024-04-22T13:27:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.909+0000 7fdb229a6740 20 client.15288 trim_cache size 0 max 16384 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.927+0000 7fdb229a6740 1 Processor -- start 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.927+0000 7fdb229a6740 1 -- start start 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.927+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a1e40 0x5617290bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.927+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x56172909c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.927+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x56172909c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.927+0000 7fdb229a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617290ba510 con 0x5617290a32d0 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb229a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617290babe0 con 0x5617290a2830 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb229a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5617290bb2b0 con 0x5617290a1e40 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x56172909c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x56172909c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56708/0 (socket says 172.21.15.5:56708) 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1724583121 learned_addr learned my addr 172.21.15.5:0/1724583121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x56172909c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a1e40 0x5617290bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1724583121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a1e40 msgr2=0x5617290bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a1e40 0x5617290bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1724583121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 msgr2=0x56172909c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x56172909c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1724583121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56172909dd40 con 0x5617290a32d0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x56172909c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x56172909c890 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdb140059f0 tx=0x7fdb14000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17a34d9f661ea51 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb037fe640 1 -- 172.21.15.5:0/1724583121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14019070 con 0x5617290a32d0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb037fe640 1 -- 172.21.15.5:0/1724583121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1401ebc0 con 0x5617290a32d0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.928+0000 7fdb037fe640 1 -- 172.21.15.5:0/1724583121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400f730 con 0x5617290a32d0 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 -- 172.21.15.5:0/1724583121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 msgr2=0x56172909c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x56172909c890 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdb140059f0 tx=0x7fdb14000bc0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 -- 172.21.15.5:0/1724583121 shutdown_connections 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a32d0 0x56172909c890 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a2830 0x56172909c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1724583121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a1e40 0x5617290bbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 -- 172.21.15.5:0/1724583121 >> 172.21.15.5:0/1724583121 conn(0x56172909df40 msgr2=0x561729097800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 -- 172.21.15.5:0/1724583121 shutdown_connections 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 -- 172.21.15.5:0/1724583121 wait complete. 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 -- start start 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 Processor -- start 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.929+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a32d0 0x5617290a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617290bc050 con 0x5617290a1e40 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5617290bb980 con 0x5617290a32d0 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5617290bc720 con 0x5617290a2830 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 10 client.0 _ll_register_callbacks cb 0x7fdb14013610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56722/0 (socket says 172.21.15.5:56722) 2024-04-22T13:27:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1038657168 learned_addr learned my addr 172.21.15.5:0/1038657168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a32d0 0x5617290a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 msgr2=0x5617290a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1038657168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a32d0 msgr2=0x5617290a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a32d0 0x5617290a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 -- 172.21.15.5:0/1038657168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617290a4810 con 0x5617290a1e40 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb1d26d640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290a59b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdb1401eac0 tx=0x7fdb1400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507d1683c6af6751 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb017fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb017fa640 1 -- 172.21.15.5:0/1038657168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14040060 con 0x5617290a1e40 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb017fa640 1 -- 172.21.15.5:0/1038657168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fdb1401e660 con 0x5617290a1e40 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1401ebc0 con 0x5617290a1e40 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb017fa640 1 -- 172.21.15.5:0/1038657168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb14019070 con 0x5617290a1e40 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.930+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5617290ae5e0 con 0x5617290a1e40 2024-04-22T13:27:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb017fa640 1 -- 172.21.15.5:0/1038657168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1401ebc0 con 0x5617290a1e40 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb017fa640 1 -- 172.21.15.5:0/1038657168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdb14036070 con 0x5617290a1e40 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb229a6740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb229a6740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb229a6740 20 client.15294 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617291270c0 0x561729123bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb229a6740 4 client.15294 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "status"}) -- 0x56172900e3a0 con 0x5617291270c0 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617291270c0 0x561729123bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb017fa640 10 client.15294 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb1da6e640 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617291270c0 0x561729123bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=873052dc4e60ddce server_cookie=a5ff3f2c191e7e2e in_seq=0 out_seq=0 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.931+0000 7fdb017fa640 1 -- 172.21.15.5:0/1038657168 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56172900e3a0 con 0x5617291270c0 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb017fa640 10 client.15294 handle_command_reply: tid=0 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 client.15294 shutdown 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 msgr2=0x5617290a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290a59b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdb1401eac0 tx=0x7fdb1400b2e0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 shutdown_connections 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5617291270c0 0x561729123bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5617290a32d0 0x5617290a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617290a2830 0x5617290a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 --2- 172.21.15.5:0/1038657168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617290a1e40 0x5617290a59b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb017fa640 0 client.15294 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 >> 172.21.15.5:0/1038657168 conn(0x56172909df40 msgr2=0x5617290a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 shutdown_connections 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 1 -- 172.21.15.5:0/1038657168 wait complete. 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb229a6740 20 client.15294 trim_cache size 0 max 16384 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24496, 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 106.012837417, 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 123.70989232700001 2024-04-22T13:27:00.936 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb040a3c30 msgr2=0x7fdb040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb040a3c30 0x7fdb040a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb10098b80 tx=0x7fdb10066d70 comp rx=0 tx=0).stop 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 msgr2=0x7fdb1819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 0x7fdb1819e690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000cad0 tx=0x7fdb1000cf90 comp rx=0 tx=0).stop 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 shutdown_connections 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fdb040a3c30 0x7fdb040a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb181029b0 0x7fdb1819e690 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb18101ff0 0x7fdb18197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 --2- 172.21.15.5:0/1175725778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb18100ea0 0x7fdb181970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.932+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 >> 172.21.15.5:0/1175725778 conn(0x7fdb180fc430 msgr2=0x7fdb18105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.933+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 shutdown_connections 2024-04-22T13:27:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:00.933+0000 7fdb1fcd2640 1 -- 172.21.15.5:0/1175725778 wait complete. 2024-04-22T13:27:00.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.046+0000 7f7077775640 1 -- 172.21.15.5:0/3887875446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 msgr2=0x7f707010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.046+0000 7f7077775640 1 --2- 172.21.15.5:0/3887875446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707010a0e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7068009ca0 tx=0x7f706802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- 172.21.15.5:0/3887875446 shutdown_connections 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 --2- 172.21.15.5:0/3887875446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707010a0e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 --2- 172.21.15.5:0/3887875446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70701053c0 0x7f70701077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 --2- 172.21.15.5:0/3887875446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7070102a90 0x7f7070104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- 172.21.15.5:0/3887875446 >> 172.21.15.5:0/3887875446 conn(0x7f70700fc420 msgr2=0x7f70700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- 172.21.15.5:0/3887875446 shutdown_connections 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- 172.21.15.5:0/3887875446 wait complete. 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 Processor -- start 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- start start 2024-04-22T13:27:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102a90 0x7f70701981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70701053c0 0x7f7070198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f707010daf0 con 0x7f7070107cf0 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f707010d970 con 0x7f7070102a90 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7077775640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f707010dc70 con 0x7f70701053c0 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.047+0000 7f7075d12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56732/0 (socket says 172.21.15.5:56732) 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 -- 172.21.15.5:0/125876086 learned_addr learned my addr 172.21.15.5:0/125876086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075511640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102a90 0x7f70701981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7074d10640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70701053c0 0x7f7070198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 -- 172.21.15.5:0/125876086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70701053c0 msgr2=0x7f7070198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70701053c0 0x7f7070198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 -- 172.21.15.5:0/125876086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102a90 msgr2=0x7f70701981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102a90 0x7f70701981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 -- 172.21.15.5:0/125876086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707019fde0 con 0x7f7070107cf0 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7075d12640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707019f770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f706802c700 tx=0x7f7068034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f70667fc640 1 -- 172.21.15.5:0/125876086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068039040 con 0x7f7070107cf0 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f70667fc640 1 -- 172.21.15.5:0/125876086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7068044070 con 0x7f7070107cf0 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70701a0070 con 0x7f7070107cf0 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f70667fc640 1 -- 172.21.15.5:0/125876086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680042a0 con 0x7f7070107cf0 2024-04-22T13:27:01.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.048+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70701a0560 con 0x7f7070107cf0 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.050+0000 7f70667fc640 1 -- 172.21.15.5:0/125876086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f7068003970 con 0x7f7070107cf0 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 Processor -- start 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 -- start start 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 0x563ee10c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c56e0 0x563ee10cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ee1024b10 con 0x563ee10c4250 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563ee0fc2240 con 0x563ee10c4c40 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7079ffd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ee1020ad0 con 0x563ee10c56e0 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f70667fc640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f705c0a3bb0 0x7f705c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7074d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 0x563ee10c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7074d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 0x563ee10c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56738/0 (socket says 172.21.15.5:56738) 2024-04-22T13:27:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7074d10640 1 -- 172.21.15.5:0/850548373 learned_addr learned my addr 172.21.15.5:0/850548373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7075d12640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f705c0a3bb0 0x7f705c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f70667fc640 1 -- 172.21.15.5:0/125876086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f70680f2720 con 0x7f7070107cf0 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7074d10640 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c56e0 0x563ee10cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7075511640 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.051+0000 7f7074d10640 1 -- 172.21.15.5:0/850548373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c56e0 msgr2=0x563ee10cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f7074d10640 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c56e0 0x563ee10cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f7074d10640 1 -- 172.21.15.5:0/850548373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 msgr2=0x563ee10c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f7074d10640 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f7074d10640 1 -- 172.21.15.5:0/850548373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee0d372d0 con 0x563ee10c4250 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f7074d10640 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 0x563ee10c4670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f706000bfd0 tx=0x7f706000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154f1cff24b53638 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f7075d12640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f705c0a3bb0 0x7f705c0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70680bae70 tx=0x7f7068089110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f705b7fe640 1 -- 172.21.15.5:0/850548373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060004030 con 0x563ee10c4250 2024-04-22T13:27:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f705b7fe640 1 -- 172.21.15.5:0/850548373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70600041d0 con 0x563ee10c4250 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.052+0000 7f705b7fe640 1 -- 172.21.15.5:0/850548373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706000f6e0 con 0x563ee10c4250 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 -- 172.21.15.5:0/850548373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 msgr2=0x563ee10c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 0x563ee10c4670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f706000bfd0 tx=0x7f706000b820 comp rx=0 tx=0).stop 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 -- 172.21.15.5:0/850548373 shutdown_connections 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c56e0 0x563ee10cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 --2- 172.21.15.5:0/850548373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4250 0x563ee10c4670 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 -- 172.21.15.5:0/850548373 >> 172.21.15.5:0/850548373 conn(0x563ee10bbf40 msgr2=0x563ee10ba490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 -- 172.21.15.5:0/850548373 shutdown_connections 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 -- 172.21.15.5:0/850548373 wait complete. 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 -- start start 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.053+0000 7f7079ffd740 1 Processor -- start 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4250 0x563ee10ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4c40 0x563ee10ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ee10cc8c0 con 0x563ee10c56e0 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563ee0fad190 con 0x563ee10c4250 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ee10cca40 con 0x563ee10c4c40 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 10 client.0 _ll_register_callbacks cb 0x563ee0fa88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56748/0 (socket says 172.21.15.5:56748) 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 -- 172.21.15.5:0/1813875906 learned_addr learned my addr 172.21.15.5:0/1813875906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4250 0x563ee10ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7075511640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4c40 0x563ee10ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 -- 172.21.15.5:0/1813875906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4c40 msgr2=0x563ee10ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4c40 0x563ee10ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 -- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4250 msgr2=0x563ee10ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4250 0x563ee10ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 -- 172.21.15.5:0/1813875906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee0e016b0 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7074d10640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d5870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7060005870 tx=0x7f7060005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd1f7cb708019bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7058ff9640 1 -- 172.21.15.5:0/1813875906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70600035f0 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7058ff9640 1 -- 172.21.15.5:0/1813875906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f7060003790 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.054+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70600041d0 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7058ff9640 1 -- 172.21.15.5:0/1813875906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7060003990 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 10 client.15306 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ee10d8be0 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7058ff9640 1 -- 172.21.15.5:0/1813875906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70600041d0 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7058ff9640 1 -- 172.21.15.5:0/1813875906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f706001c070 con 0x563ee10c56e0 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 10 client.15306 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 20 client.15306 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee10daf30 0x563ee10dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 4 client.15306 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee1032530 con 0x563ee10daf30 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.055+0000 7f7075511640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee10daf30 0x563ee10dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.056+0000 7f7075511640 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee10daf30 0x563ee10dc3a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2513fb23f17d743 server_cookie=cfabfc3689ca30c3 in_seq=0 out_seq=0 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.056+0000 7f7058ff9640 10 client.15306 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7058ff9640 1 -- 172.21.15.5:0/1813875906 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ee1032530 con 0x563ee10daf30 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7058ff9640 10 client.15306 handle_command_reply: tid=0 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 client.15306 shutdown 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 msgr2=0x563ee10d5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d5870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7060005870 tx=0x7f7060005c50 comp rx=0 tx=0).stop 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 shutdown_connections 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee10daf30 0x563ee10dc3a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d5870 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4c40 0x563ee10ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7058ff9640 0 client.15306 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1813875906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4250 0x563ee10ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 >> 172.21.15.5:0/1813875906 conn(0x563ee10bbf40 msgr2=0x563ee10b9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 shutdown_connections 2024-04-22T13:27:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 1 -- 172.21.15.5:0/1813875906 wait complete. 2024-04-22T13:27:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.057+0000 7f7079ffd740 20 client.15306 trim_cache size 0 max 16384 2024-04-22T13:27:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:01 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/687190096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:01 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1845933429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 Processor -- start 2024-04-22T13:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 -- start start 2024-04-22T13:27:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 0x563ee10c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c56e0 0x563ee10c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ee10da6e0 con 0x563ee10c4c40 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563ee10dadb0 con 0x563ee10c56e0 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7079ffd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ee10db480 con 0x563ee10c4250 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.075+0000 7f7075511640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 0x563ee10c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 0x563ee10c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56762/0 (socket says 172.21.15.5:56762) 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 -- 172.21.15.5:0/3890013517 learned_addr learned my addr 172.21.15.5:0/3890013517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7074d10640 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7074d10640 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c56e0 0x563ee10c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 -- 172.21.15.5:0/3890013517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 msgr2=0x563ee10d9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10d9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 -- 172.21.15.5:0/3890013517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c56e0 msgr2=0x563ee10c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c56e0 0x563ee10c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 -- 172.21.15.5:0/3890013517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee10bbd40 con 0x563ee10c4c40 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7074d10640 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c56e0 0x563ee10c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f7075511640 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 0x563ee10c8110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f706c005fd0 tx=0x7f706c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6477f1e66b764dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f705affd640 1 -- 172.21.15.5:0/3890013517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c01d430 con 0x563ee10c4c40 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f705affd640 1 -- 172.21.15.5:0/3890013517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f706c011440 con 0x563ee10c4c40 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.076+0000 7f705affd640 1 -- 172.21.15.5:0/3890013517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c010690 con 0x563ee10c4c40 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- 172.21.15.5:0/3890013517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 msgr2=0x563ee10c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 0x563ee10c8110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f706c005fd0 tx=0x7f706c0039c0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- 172.21.15.5:0/3890013517 shutdown_connections 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c4c40 0x563ee10c8110 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c56e0 0x563ee10c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- 172.21.15.5:0/3890013517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- 172.21.15.5:0/3890013517 >> 172.21.15.5:0/3890013517 conn(0x563ee10bbf40 msgr2=0x563ee10ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- 172.21.15.5:0/3890013517 shutdown_connections 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- 172.21.15.5:0/3890013517 wait complete. 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- start start 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 Processor -- start 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ee10dc220 con 0x563ee10c56e0 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.077+0000 7f7079ffd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563ee10dbb50 con 0x563ee10c4c40 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7079ffd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ee10dc8f0 con 0x563ee10c4250 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7079ffd740 10 client.0 _ll_register_callbacks cb 0x7f70600123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56778/0 (socket says 172.21.15.5:56778) 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 -- 172.21.15.5:0/1120427117 learned_addr learned my addr 172.21.15.5:0/1120427117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 -- 172.21.15.5:0/1120427117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 msgr2=0x563ee10caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 -- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 msgr2=0x563ee10cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 -- 172.21.15.5:0/1120427117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee10d3d40 con 0x563ee10c56e0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7074d10640 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d3780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7060005870 tx=0x7f70600056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e9b09a4cecd3af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7058ff9640 1 -- 172.21.15.5:0/1120427117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70600036b0 con 0x563ee10c56e0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7058ff9640 1 -- 172.21.15.5:0/1120427117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f7060003b30 con 0x563ee10c56e0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f706c011440 con 0x563ee10c56e0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7058ff9640 1 -- 172.21.15.5:0/1120427117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f706003e050 con 0x563ee10c56e0 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7079ffd740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.078+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ee10d4fe0 con 0x563ee10c56e0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7058ff9640 1 -- 172.21.15.5:0/1120427117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f706c011440 con 0x563ee10c56e0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7058ff9640 1 -- 172.21.15.5:0/1120427117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7060036020 con 0x563ee10c56e0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7079ffd740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7079ffd740 10 client.15312 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7079ffd740 20 client.15312 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee11457e0 0x563ee11444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7079ffd740 4 client.15312 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "status"}) -- 0x563ee102c3a0 con 0x563ee11457e0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7075511640 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee11457e0 0x563ee11444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7058ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.079+0000 7f7075511640 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee11457e0 0x563ee11444f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6951e4a7d9af8331 server_cookie=91b445ea5de59649 in_seq=0 out_seq=0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7058ff9640 1 -- 172.21.15.5:0/1120427117 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x563ee102c3a0 con 0x563ee11457e0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7058ff9640 10 client.15312 handle_command_reply: tid=0 2024-04-22T13:27:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 client.15312 shutdown 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 msgr2=0x563ee10d3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d3780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7060005870 tx=0x7f70600056d0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 shutdown_connections 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563ee11457e0 0x563ee11444f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ee10c56e0 0x563ee10d3780 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ee10c4c40 0x563ee10cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7058ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 --2- 172.21.15.5:0/1120427117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ee10c4250 0x563ee10caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 >> 172.21.15.5:0/1120427117 conn(0x563ee10bbf40 msgr2=0x563ee10c82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 shutdown_connections 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 1 -- 172.21.15.5:0/1120427117 wait complete. 2024-04-22T13:27:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7079ffd740 20 client.15312 trim_cache size 0 max 16384 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24512, 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 107.170344075, 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 126.48316054 2024-04-22T13:27:01.085 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f705c0a3bb0 msgr2=0x7f705c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f705c0a3bb0 0x7f705c0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70680bae70 tx=0x7f7068089110 comp rx=0 tx=0).stop 2024-04-22T13:27:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 msgr2=0x7f707019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707019f770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f706802c700 tx=0x7f7068034760 comp rx=0 tx=0).stop 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 shutdown_connections 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f705c0a3bb0 0x7f705c0a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7070107cf0 0x7f707019f770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70701053c0 0x7f7070198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 --2- 172.21.15.5:0/125876086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7070102a90 0x7f70701981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.080+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 >> 172.21.15.5:0/125876086 conn(0x7f70700fc420 msgr2=0x7f70701033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.081+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 shutdown_connections 2024-04-22T13:27:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.081+0000 7f7077775640 1 -- 172.21.15.5:0/125876086 wait complete. 2024-04-22T13:27:01.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T13:27:01.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 -- 172.21.15.5:0/1496290878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 msgr2=0x7f22a810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1496290878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a810a3f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009ca0 tx=0x7f22a002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 -- 172.21.15.5:0/1496290878 shutdown_connections 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1496290878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a810a3f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1496290878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22a81031a0 0x7f22a8103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1496290878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a8101fb0 0x7f22a81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 -- 172.21.15.5:0/1496290878 >> 172.21.15.5:0/1496290878 conn(0x7f22a80fd520 msgr2=0x7f22a80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 -- 172.21.15.5:0/1496290878 shutdown_connections 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.195+0000 7f22adf0e640 1 -- 172.21.15.5:0/1496290878 wait complete. 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 Processor -- start 2024-04-22T13:27:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 -- start start 2024-04-22T13:27:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22a8101fb0 0x7f22a8198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a81031a0 0x7f22a8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22a810dab0 con 0x7f22a8103b60 2024-04-22T13:27:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22a810d930 con 0x7f22a8101fb0 2024-04-22T13:27:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22adf0e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22a810dc30 con 0x7f22a81031a0 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56794/0 (socket says 172.21.15.5:56794) 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 -- 172.21.15.5:0/1362791676 learned_addr learned my addr 172.21.15.5:0/1362791676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a77fe640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22a8101fb0 0x7f22a8198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a81031a0 0x7f22a8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 -- 172.21.15.5:0/1362791676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a81031a0 msgr2=0x7f22a8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a81031a0 0x7f22a8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 -- 172.21.15.5:0/1362791676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22a8101fb0 msgr2=0x7f22a8198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22a8101fb0 0x7f22a8198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a7fff640 1 -- 172.21.15.5:0/1362791676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a819fd70 con 0x7f22a8103b60 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.196+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a81031a0 0x7f22a8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.197+0000 7f22a7fff640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a819f700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22a0032520 tx=0x7f22a00347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.197+0000 7f22a4ff9640 1 -- 172.21.15.5:0/1362791676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0039040 con 0x7f22a8103b60 2024-04-22T13:27:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.197+0000 7f22a4ff9640 1 -- 172.21.15.5:0/1362791676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a0044070 con 0x7f22a8103b60 2024-04-22T13:27:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.197+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a81a0000 con 0x7f22a8103b60 2024-04-22T13:27:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.197+0000 7f22a4ff9640 1 -- 172.21.15.5:0/1362791676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00042a0 con 0x7f22a8103b60 2024-04-22T13:27:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.197+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a81a04f0 con 0x7f22a8103b60 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a4ff9640 1 -- 172.21.15.5:0/1362791676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f22a0003970 con 0x7f22a8103b60 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 Processor -- start 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 -- start start 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e5f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5fc40 0x557666e600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e606e0 0x557666e672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557666dbfb10 con 0x557666e5f250 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557666d5d240 con 0x557666e606e0 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22b0bda740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557666dbbad0 con 0x557666e5fc40 2024-04-22T13:27:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e5f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e5f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56802/0 (socket says 172.21.15.5:56802) 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1262451686 learned_addr learned my addr 172.21.15.5:0/1262451686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a77fe640 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5fc40 0x557666e600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a4ff9640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22900a3bb0 0x7f22900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1262451686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5fc40 msgr2=0x557666e600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5fc40 0x557666e600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1262451686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e606e0 msgr2=0x557666e672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e606e0 0x557666e672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a7fff640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22900a3bb0 0x7f22900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1262451686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557666ad22d0 con 0x557666e5f250 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.200+0000 7f22a4ff9640 1 -- 172.21.15.5:0/1362791676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f22a00f2750 con 0x7f22a8103b60 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e5f670 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f229c0058c0 tx=0x7f229c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df1920dc593cf9a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f228dffb640 1 -- 172.21.15.5:0/1262451686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c011e60 con 0x557666e5f250 2024-04-22T13:27:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22a7fff640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22900a3bb0 0x7f22900a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22a00bb410 tx=0x7f22a0089090 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f228dffb640 1 -- 172.21.15.5:0/1262451686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c01aab0 con 0x557666e5f250 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f228dffb640 1 -- 172.21.15.5:0/1262451686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c012b40 con 0x557666e5f250 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 -- 172.21.15.5:0/1262451686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 msgr2=0x557666e5f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e5f670 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f229c0058c0 tx=0x7f229c011750 comp rx=0 tx=0).stop 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 -- 172.21.15.5:0/1262451686 shutdown_connections 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e606e0 0x557666e672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5fc40 0x557666e600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1262451686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e5f670 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.201+0000 7f22b0bda740 1 -- 172.21.15.5:0/1262451686 >> 172.21.15.5:0/1262451686 conn(0x557666e56f40 msgr2=0x557666e55490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 -- 172.21.15.5:0/1262451686 shutdown_connections 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 -- 172.21.15.5:0/1262451686 wait complete. 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 -- start start 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 Processor -- start 2024-04-22T13:27:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e69830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e606e0 0x557666e70870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557666e678c0 con 0x557666e5f250 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557666d48190 con 0x557666e5fc40 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557666e67a40 con 0x557666e606e0 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22b0bda740 10 client.0 _ll_register_callbacks cb 0x557666d438b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e692f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56806/0 (socket says 172.21.15.5:56806) 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.202+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1383053293 learned_addr learned my addr 172.21.15.5:0/1383053293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a77fe640 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e69830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1383053293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e606e0 msgr2=0x557666e70870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e606e0 0x557666e70870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 msgr2=0x557666e69830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e69830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1383053293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557666b9c6b0 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e692f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f229c004630 tx=0x7f229c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541b51e64acffd00 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f2286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f2286ffd640 1 -- 172.21.15.5:0/1383053293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c02b060 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f2286ffd640 1 -- 172.21.15.5:0/1383053293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f229c01a400 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c01aab0 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f2286ffd640 1 -- 172.21.15.5:0/1383053293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f229c020070 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22b0bda740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557666e73be0 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.203+0000 7f2286ffd640 1 -- 172.21.15.5:0/1383053293 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c01aab0 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f2286ffd640 1 -- 172.21.15.5:0/1383053293 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f229c026420 con 0x557666e5f250 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22b0bda740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22b0bda740 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:27:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22b0bda740 20 client.15324 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666e75dc0 0x557666e77230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22b0bda740 4 client.15324 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557666dcd530 con 0x557666e75dc0 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22a77fe640 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666e75dc0 0x557666e77230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f2286ffd640 10 client.15324 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.204+0000 7f22a77fe640 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666e75dc0 0x557666e77230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87268b35ca62f423 server_cookie=4b247276df1d4f13 in_seq=0 out_seq=0 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f2286ffd640 1 -- 172.21.15.5:0/1383053293 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557666dcd530 con 0x557666e75dc0 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f2286ffd640 10 client.15324 handle_command_reply: tid=0 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 client.15324 shutdown 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 msgr2=0x557666e692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e692f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f229c004630 tx=0x7f229c00b4f0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 shutdown_connections 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666e75dc0 0x557666e77230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e606e0 0x557666e70870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e69830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f2286ffd640 0 client.15324 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1383053293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e5f250 0x557666e692f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.205+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 >> 172.21.15.5:0/1383053293 conn(0x557666e56f40 msgr2=0x557666e54a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.206+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 shutdown_connections 2024-04-22T13:27:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.206+0000 7f22b0bda740 1 -- 172.21.15.5:0/1383053293 wait complete. 2024-04-22T13:27:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.206+0000 7f22b0bda740 20 client.15324 trim_cache size 0 max 16384 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 Processor -- start 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 -- start start 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557666e75570 con 0x557666e606e0 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557666e75c40 con 0x557666e5fc40 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22b0bda740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557666e76310 con 0x557666e5f250 2024-04-22T13:27:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e63cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56818/0 (socket says 172.21.15.5:56818) 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1152199630 learned_addr learned my addr 172.21.15.5:0/1152199630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1152199630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 msgr2=0x557666e672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1152199630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 msgr2=0x557666e70800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 -- 172.21.15.5:0/1152199630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557666e56d40 con 0x557666e606e0 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e63cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f229c00bbf0 tx=0x7f229c00baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f57593ca212d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.224+0000 7f228d7fa640 1 -- 172.21.15.5:0/1152199630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c036020 con 0x557666e606e0 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f228d7fa640 1 -- 172.21.15.5:0/1152199630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c026620 con 0x557666e606e0 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f228d7fa640 1 -- 172.21.15.5:0/1152199630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c02b070 con 0x557666e606e0 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 -- 172.21.15.5:0/1152199630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 msgr2=0x557666e63cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e63cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f229c00bbf0 tx=0x7f229c00baa0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 -- 172.21.15.5:0/1152199630 shutdown_connections 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e63cf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e70800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 --2- 172.21.15.5:0/1152199630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 -- 172.21.15.5:0/1152199630 >> 172.21.15.5:0/1152199630 conn(0x557666e56f40 msgr2=0x557666e67de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 -- 172.21.15.5:0/1152199630 shutdown_connections 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 -- 172.21.15.5:0/1152199630 wait complete. 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.225+0000 7f22b0bda740 1 -- start start 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 Processor -- start 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e6f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557666e770b0 con 0x557666e606e0 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557666e769e0 con 0x557666e5fc40 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557666e77780 con 0x557666e5f250 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22b0bda740 10 client.0 _ll_register_callbacks cb 0x7f229800a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e6f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e6f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56828/0 (socket says 172.21.15.5:56828) 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 -- 172.21.15.5:0/2941861084 learned_addr learned my addr 172.21.15.5:0/2941861084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a77fe640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 -- 172.21.15.5:0/2941861084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 msgr2=0x557666e6c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e6c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 -- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 msgr2=0x557666e6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 -- 172.21.15.5:0/2941861084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557666e6f870 con 0x557666e606e0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a77fe640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f22a6ffd640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e6f200 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b530 tx=0x7f229c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d926887f7d135e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.226+0000 7f2286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f2286ffd640 1 -- 172.21.15.5:0/2941861084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c036070 con 0x557666e606e0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f2286ffd640 1 -- 172.21.15.5:0/2941861084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f229c004030 con 0x557666e606e0 2024-04-22T13:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c026620 con 0x557666e606e0 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f2286ffd640 1 -- 172.21.15.5:0/2941861084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f229c02b070 con 0x557666e606e0 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 10 client.15330 fetch_fsmap learned FSMap version 26 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557666e70b50 con 0x557666e606e0 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f2286ffd640 1 -- 172.21.15.5:0/2941861084 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c026620 con 0x557666e606e0 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f2286ffd640 1 -- 172.21.15.5:0/2941861084 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f229c003680 con 0x557666e606e0 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 20 client.15330 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666ee0320 0x557666edf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 4 client.15330 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.227+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "status"}) -- 0x557666dc73a0 con 0x557666ee0320 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22a77fe640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666ee0320 0x557666edf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22a77fe640 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666ee0320 0x557666edf030 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41026a6a072849ec server_cookie=1c147dff1a6688d0 in_seq=0 out_seq=0 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f2286ffd640 10 client.15330 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f2286ffd640 1 -- 172.21.15.5:0/2941861084 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557666dc73a0 con 0x557666ee0320 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f2286ffd640 10 client.15330 handle_command_reply: tid=0 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 client.15330 shutdown 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 msgr2=0x557666e6f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e6f200 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b530 tx=0x7f229c011e60 comp rx=0 tx=0).stop 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 shutdown_connections 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557666ee0320 0x557666edf030 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557666e606e0 0x557666e6f200 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f2286ffd640 0 client.15330 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557666e5fc40 0x557666e6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 --2- 172.21.15.5:0/2941861084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557666e5f250 0x557666e6c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 >> 172.21.15.5:0/2941861084 conn(0x557666e56f40 msgr2=0x557666e632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.228+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 shutdown_connections 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22b0bda740 1 -- 172.21.15.5:0/2941861084 wait complete. 2024-04-22T13:27:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22b0bda740 20 client.15330 trim_cache size 0 max 16384 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24557, 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 110.339589135, 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 115.66155800999999 2024-04-22T13:27:01.232 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22900a3bb0 msgr2=0x7f22900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22900a3bb0 0x7f22900a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22a00bb410 tx=0x7f22a0089090 comp rx=0 tx=0).stop 2024-04-22T13:27:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 msgr2=0x7f22a819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a819f700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22a0032520 tx=0x7f22a00347b0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 shutdown_connections 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f22900a3bb0 0x7f22900a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a8103b60 0x7f22a819f700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22a81031a0 0x7f22a8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 --2- 172.21.15.5:0/1362791676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22a8101fb0 0x7f22a8198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 >> 172.21.15.5:0/1362791676 conn(0x7f22a80fd520 msgr2=0x7f22a8106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 shutdown_connections 2024-04-22T13:27:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.229+0000 7f22adf0e640 1 -- 172.21.15.5:0/1362791676 wait complete. 2024-04-22T13:27:01.237 INFO:teuthology.run_tasks:Running task sequential... 2024-04-22T13:27:01.247 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-22T13:27:01.247 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-22T13:27:01.247 INFO:teuthology.task.print:Enabling mgr modules 2024-04-22T13:27:01.247 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-22T13:27:01.248 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:27:01.248 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:27:01.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-22T13:27:01.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:01 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/687190096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:01.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:01 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1845933429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 -- 172.21.15.5:0/125343594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 msgr2=0x7f1fe410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 --2- 172.21.15.5:0/125343594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe410a0f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009ca0 tx=0x7f1fdc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 -- 172.21.15.5:0/125343594 shutdown_connections 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 --2- 172.21.15.5:0/125343594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe410a0f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 --2- 172.21.15.5:0/125343594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe4105370 0x7f1fe4107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 --2- 172.21.15.5:0/125343594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 0x7f1fe4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 -- 172.21.15.5:0/125343594 >> 172.21.15.5:0/125343594 conn(0x7f1fe40fc430 msgr2=0x7f1fe40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 -- 172.21.15.5:0/125343594 shutdown_connections 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.352+0000 7f1feb132640 1 -- 172.21.15.5:0/125343594 wait complete. 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 Processor -- start 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 -- start start 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 0x7f1fe4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe4105370 0x7f1fe41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410db00 con 0x7f1fe4107ca0 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410d980 con 0x7f1fe4102a40 2024-04-22T13:27:01.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1feb132640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410dc80 con 0x7f1fe4105370 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56840/0 (socket says 172.21.15.5:56840) 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 -- 172.21.15.5:0/4238806556 learned_addr learned my addr 172.21.15.5:0/4238806556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe3fff640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe4105370 0x7f1fe41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 -- 172.21.15.5:0/4238806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe4105370 msgr2=0x7f1fe41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe8ece640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 0x7f1fe4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe4105370 0x7f1fe41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 -- 172.21.15.5:0/4238806556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 msgr2=0x7f1fe4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 0x7f1fe4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.353+0000 7f1fe96cf640 1 -- 172.21.15.5:0/4238806556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe419fda0 con 0x7f1fe4107ca0 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1fe8ece640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 0x7f1fe4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1fe96cf640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe419f730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc02c700 tx=0x7f1fdc034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc039040 con 0x7f1fe4107ca0 2024-04-22T13:27:01.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc044070 con 0x7f1fe4107ca0 2024-04-22T13:27:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41a0030 con 0x7f1fe4107ca0 2024-04-22T13:27:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0042a0 con 0x7f1fe4107ca0 2024-04-22T13:27:01.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.354+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a0520 con 0x7f1fe4107ca0 2024-04-22T13:27:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.355+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40fdf20 con 0x7f1fe4107ca0 2024-04-22T13:27:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.356+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1fdc003970 con 0x7f1fe4107ca0 2024-04-22T13:27:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.357+0000 7f1fe1ffb640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1fcc0a3ba0 0x7f1fcc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.357+0000 7f1fe8ece640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1fcc0a3ba0 0x7f1fcc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.357+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f1fdc0f28e0 con 0x7f1fe4107ca0 2024-04-22T13:27:01.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.357+0000 7f1fe8ece640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1fcc0a3ba0 0x7f1fcc0a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005d00 tx=0x7f1fd4005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:01.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.361+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fdc0bf270 con 0x7f1fe4107ca0 2024-04-22T13:27:01.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:01 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/687190096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:01.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:01 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1845933429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:01.598+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1fe4062b50 con 0x7f1fe4107ca0 2024-04-22T13:27:02.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.027+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/4238806556 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f1fdc0beb70 con 0x7f1fe4107ca0 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1fcc0a3ba0 msgr2=0x7f1fcc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1fcc0a3ba0 0x7f1fcc0a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005d00 tx=0x7f1fd4005ed0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 msgr2=0x7f1fe419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe419f730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc02c700 tx=0x7f1fdc034760 comp rx=0 tx=0).stop 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 shutdown_connections 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f1fcc0a3ba0 0x7f1fcc0a6060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fe4107ca0 0x7f1fe419f730 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe4105370 0x7f1fe41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 --2- 172.21.15.5:0/4238806556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe4102a40 0x7f1fe4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.030+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 >> 172.21.15.5:0/4238806556 conn(0x7f1fe40fc430 msgr2=0x7f1fe41032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.031+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 shutdown_connections 2024-04-22T13:27:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.031+0000 7f1feb132640 1 -- 172.21.15.5:0/4238806556 wait complete. 2024-04-22T13:27:02.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 -- 172.21.15.5:0/1509297066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78102a00 msgr2=0x7fed78109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 --2- 172.21.15.5:0/1509297066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78102a00 0x7fed78109320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed70009ca0 tx=0x7fed7002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 -- 172.21.15.5:0/1509297066 shutdown_connections 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 --2- 172.21.15.5:0/1509297066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78102a00 0x7fed78109320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 --2- 172.21.15.5:0/1509297066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 0x7fed781024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 --2- 172.21.15.5:0/1509297066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed78100e60 0x7fed78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 -- 172.21.15.5:0/1509297066 >> 172.21.15.5:0/1509297066 conn(0x7fed780fc420 msgr2=0x7fed780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 -- 172.21.15.5:0/1509297066 shutdown_connections 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.160+0000 7fed7de4e640 1 -- 172.21.15.5:0/1509297066 wait complete. 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 Processor -- start 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 -- start start 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 0x7fed7806a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 0x7fed7819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed78102a00 0x7fed7819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed7810c9f0 con 0x7fed78100e60 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed7810c870 con 0x7fed78102a00 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed7de4e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed7810cb70 con 0x7fed78102040 2024-04-22T13:27:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 0x7fed7806a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 0x7fed7806a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56882/0 (socket says 172.21.15.5:56882) 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed777fe640 1 -- 172.21.15.5:0/2811735561 learned_addr learned my addr 172.21.15.5:0/2811735561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed76ffd640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 0x7fed7819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.161+0000 7fed77fff640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed78102a00 0x7fed7819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed777fe640 1 -- 172.21.15.5:0/2811735561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 msgr2=0x7fed7819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed777fe640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 0x7fed7819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed777fe640 1 -- 172.21.15.5:0/2811735561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed78102a00 msgr2=0x7fed7819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed777fe640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed78102a00 0x7fed7819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed777fe640 1 -- 172.21.15.5:0/2811735561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7819fd80 con 0x7fed78100e60 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed76ffd640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 0x7fed7819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed777fe640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 0x7fed7806a2b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fed6800e9c0 tx=0x7fed6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6800cd80 con 0x7fed78100e60 2024-04-22T13:27:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed781a0070 con 0x7fed78100e60 2024-04-22T13:27:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed680035f0 con 0x7fed78100e60 2024-04-22T13:27:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68010690 con 0x7fed78100e60 2024-04-22T13:27:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.162+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed781a05c0 con 0x7fed78100e60 2024-04-22T13:27:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.163+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78108280 con 0x7fed78100e60 2024-04-22T13:27:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.164+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7fed680037a0 con 0x7fed78100e60 2024-04-22T13:27:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.165+0000 7fed74ff9640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed600a3bf0 0x7fed600a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.165+0000 7fed76ffd640 1 -- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed600a3bf0 msgr2=0x7fed600a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.165+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fed680d08d0 con 0x7fed78100e60 2024-04-22T13:27:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.165+0000 7fed76ffd640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed600a3bf0 0x7fed600a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:27:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.169+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed6809d2c0 con 0x7fed78100e60 2024-04-22T13:27:02.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-mon[28006]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:27:02.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4238806556' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T13:27:02.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: ignoring --setuser ceph since I am not root 2024-04-22T13:27:02.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: ignoring --setgroup ceph since I am not root 2024-04-22T13:27:02.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:02.222+0000 7f3d4f845200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:27:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.357+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fed7802d5c0 con 0x7fed78100e60 2024-04-22T13:27:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.361+0000 7fed74ff9640 1 -- 172.21.15.5:0/2811735561 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7fed6809cbc0 con 0x7fed78100e60 2024-04-22T13:27:02.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed600a3bf0 msgr2=0x7fed600a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed600a3bf0 0x7fed600a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 msgr2=0x7fed7806a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 0x7fed7806a2b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fed6800e9c0 tx=0x7fed6800ee80 comp rx=0 tx=0).stop 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 shutdown_connections 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fed600a3bf0 0x7fed600a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed78102a00 0x7fed7819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed78102040 0x7fed7819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 --2- 172.21.15.5:0/2811735561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed78100e60 0x7fed7806a2b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 >> 172.21.15.5:0/2811735561 conn(0x7fed780fc420 msgr2=0x7fed78105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 shutdown_connections 2024-04-22T13:27:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.363+0000 7fed7de4e640 1 -- 172.21.15.5:0/2811735561 wait complete. 2024-04-22T13:27:02.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-mon[31859]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4238806556' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: ignoring --setuser ceph since I am not root 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: ignoring --setgroup ceph since I am not root 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-mgr[33720]: -- 172.21.15.107:0/2040999419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56423ceba5a0 con 0x56423ce8e800 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:02.208+0000 7f35973f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:27:02.387 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:02.390+0000 7f35973f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:27:02.406 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-mon[31379]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:27:02.406 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4238806556' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T13:27:02.406 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: ignoring --setuser ceph since I am not root 2024-04-22T13:27:02.406 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: ignoring --setgroup ceph since I am not root 2024-04-22T13:27:02.407 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-mgr[32759]: -- 172.21.15.49:0/4159098010 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569293cc5a0 con 0x5569293a0800 2024-04-22T13:27:02.407 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:02.219+0000 7fd04cd31200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:27:02.407 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:02.410+0000 7fd04cd31200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 -- 172.21.15.5:0/729350362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570105400 msgr2=0x7fc5701077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 --2- 172.21.15.5:0/729350362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570105400 0x7fc5701077f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc560009900 tx=0x7fc56002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 -- 172.21.15.5:0/729350362 shutdown_connections 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 --2- 172.21.15.5:0/729350362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570105400 0x7fc5701077f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 --2- 172.21.15.5:0/729350362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc570107d30 0x7fc57010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 --2- 172.21.15.5:0/729350362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570102ad0 0x7fc570104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 -- 172.21.15.5:0/729350362 >> 172.21.15.5:0/729350362 conn(0x7fc5700fc420 msgr2=0x7fc5700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 -- 172.21.15.5:0/729350362 shutdown_connections 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 -- 172.21.15.5:0/729350362 wait complete. 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 Processor -- start 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.480+0000 7fc57652c640 1 -- start start 2024-04-22T13:27:02.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc57652c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc570102ad0 0x7fc570198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc57652c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570105400 0x7fc570198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc57652c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 0x7fc57019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc57652c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc57010db10 con 0x7fc570107d30 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc57652c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc57010d990 con 0x7fc570102ad0 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc57652c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc57010dc90 con 0x7fc570105400 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 0x7fc57019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 0x7fc57019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56910/0 (socket says 172.21.15.5:56910) 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 -- 172.21.15.5:0/1109108297 learned_addr learned my addr 172.21.15.5:0/1109108297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc56ffff640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc570102ad0 0x7fc570198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc56f7fe640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570105400 0x7fc570198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 -- 172.21.15.5:0/1109108297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570105400 msgr2=0x7fc570198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570105400 0x7fc570198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 -- 172.21.15.5:0/1109108297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc570102ad0 msgr2=0x7fc570198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc570102ad0 0x7fc570198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 -- 172.21.15.5:0/1109108297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57019fe70 con 0x7fc570107d30 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc574ac9640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 0x7fc57019f800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc56800eb10 tx=0x7fc56800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.481+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568004b20 con 0x7fc570107d30 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.482+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc568004cc0 con 0x7fc570107d30 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.482+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5701a0160 con 0x7fc570107d30 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.482+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568010670 con 0x7fc570107d30 2024-04-22T13:27:02.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.482+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5701a0680 con 0x7fc570107d30 2024-04-22T13:27:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.483+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5700fdf10 con 0x7fc570107d30 2024-04-22T13:27:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.484+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7fc568005020 con 0x7fc570107d30 2024-04-22T13:27:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.484+0000 7fc56d7fa640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 0x7fc55c0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.484+0000 7fc56ffff640 1 -- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 msgr2=0x7fc55c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.485+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fc5680d0980 con 0x7fc570107d30 2024-04-22T13:27:02.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.485+0000 7fc56ffff640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 0x7fc55c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:27:02.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.489+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc56809d370 con 0x7fc570107d30 2024-04-22T13:27:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.681+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc57002d5c0 con 0x7fc570107d30 2024-04-22T13:27:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.684+0000 7fc56d7fa640 1 -- 172.21.15.5:0/1109108297 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7fc568087020 con 0x7fc570107d30 2024-04-22T13:27:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.686+0000 7fc56ffff640 1 -- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 msgr2=0x7fc55c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.686+0000 7fc56ffff640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 0x7fc55c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:27:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 msgr2=0x7fc55c0a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:27:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 0x7fc55c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 msgr2=0x7fc57019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 0x7fc57019f800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc56800eb10 tx=0x7fc56800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 shutdown_connections 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fc55c0a3c70 0x7fc55c0a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc570107d30 0x7fc57019f800 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570105400 0x7fc570198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 --2- 172.21.15.5:0/1109108297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc570102ad0 0x7fc570198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 >> 172.21.15.5:0/1109108297 conn(0x7fc5700fc420 msgr2=0x7fc570104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 shutdown_connections 2024-04-22T13:27:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:02.687+0000 7fc57652c640 1 -- 172.21.15.5:0/1109108297 wait complete. 2024-04-22T13:27:02.694 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:02.419+0000 7f3d4f845200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:27:02.694 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:02.531+0000 7f3d4f845200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:27:02.695 INFO:teuthology.run_tasks:Running task kclient... 2024-04-22T13:27:02.718 INFO:tasks.kclient:Mounting kernel clients... 2024-04-22T13:27:02.719 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-22T13:27:02.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:02.719 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-22T13:27:02.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:02.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-22T13:27:02.770 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:02.516+0000 7fd04cd31200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:27:02.784 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:02.493+0000 7f35973f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:27:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-04-22T13:27:02.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:27:02.803 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-22T13:27:02.803 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T13:27:02.803 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:02.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T13:27:02.953 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:02.861+0000 7f3d4f845200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:27:02.954 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:02.956+0000 7f3d4f845200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:27:03.103 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:27:03.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:03 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4238806556' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T13:27:03.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:03 smithi005 ceph-mon[28006]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T13:27:03.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:03 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2811735561' entity='client.admin' 2024-04-22T13:27:03.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:03 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1109108297' entity='client.admin' 2024-04-22T13:27:03.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:03 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4238806556' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T13:27:03.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:03 smithi049 ceph-mon[31379]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T13:27:03.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:03 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2811735561' entity='client.admin' 2024-04-22T13:27:03.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:03 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1109108297' entity='client.admin' 2024-04-22T13:27:03.271 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:02.836+0000 7fd04cd31200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:27:03.271 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:02.926+0000 7fd04cd31200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:27:03.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:03 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4238806556' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T13:27:03.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:03 smithi107 ceph-mon[31859]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T13:27:03.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:03 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2811735561' entity='client.admin' 2024-04-22T13:27:03.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:03 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1109108297' entity='client.admin' 2024-04-22T13:27:03.284 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:02.798+0000 7f35973f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:27:03.284 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:02.882+0000 7f35973f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:27:04.400 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:04.063+0000 7f35973f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:27:04.400 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:04.234+0000 7f35973f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:27:04.400 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:04.403+0000 7f35973f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:27:04.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:04.249+0000 7f3d4f845200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:27:04.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:04.435+0000 7f3d4f845200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:27:04.522 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:04.169+0000 7fd04cd31200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:27:04.522 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:04.345+0000 7fd04cd31200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:27:04.522 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:04.522+0000 7fd04cd31200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:27:04.780 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:04.618+0000 7f3d4f845200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:27:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 -- 172.21.15.5:0/3282291596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0108ad0 msgr2=0x7f3da0108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 --2- 172.21.15.5:0/3282291596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0108ad0 0x7f3da0108eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009990 tx=0x7f3d9c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:27:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 -- 172.21.15.5:0/3282291596 shutdown_connections 2024-04-22T13:27:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 --2- 172.21.15.5:0/3282291596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da01019d0 0x7f3da010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 --2- 172.21.15.5:0/3282291596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0108ad0 0x7f3da0108eb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 --2- 172.21.15.5:0/3282291596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da0101030 0x7f3da0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 -- 172.21.15.5:0/3282291596 >> 172.21.15.5:0/3282291596 conn(0x7f3da0077fb0 msgr2=0x7f3da00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 -- 172.21.15.5:0/3282291596 shutdown_connections 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.782+0000 7f3da7962700 1 -- 172.21.15.5:0/3282291596 wait complete. 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 Processor -- start 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 -- start start 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 0x7f3da019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da01019d0 0x7f3da019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da0108ad0 0x7f3da019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0112be0 con 0x7f3da0101030 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0112a60 con 0x7f3da01019d0 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da7962700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0112d60 con 0x7f3da0108ad0 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da56fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 0x7f3da019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da56fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 0x7f3da019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56930/0 (socket says 172.21.15.5:56930) 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da56fe700 1 -- 172.21.15.5:0/1431971887 learned_addr learned my addr 172.21.15.5:0/1431971887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da5eff700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da0108ad0 0x7f3da019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.783+0000 7f3da4efd700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da01019d0 0x7f3da019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da56fe700 1 -- 172.21.15.5:0/1431971887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da0108ad0 msgr2=0x7f3da019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da56fe700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da0108ad0 0x7f3da019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da56fe700 1 -- 172.21.15.5:0/1431971887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da01019d0 msgr2=0x7f3da019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da56fe700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da01019d0 0x7f3da019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da56fe700 1 -- 172.21.15.5:0/1431971887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da01a0300 con 0x7f3da0101030 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da5eff700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da0108ad0 0x7f3da019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da56fe700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 0x7f3da019b2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000c00 tx=0x7f3d9c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c027070 con 0x7f3da0101030 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da01a0590 con 0x7f3da0101030 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d9c022470 con 0x7f3da0101030 2024-04-22T13:27:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da01a7e70 con 0x7f3da0101030 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.784+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c017650 con 0x7f3da0101030 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.786+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da0104350 con 0x7f3da0101030 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.786+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f3d9c02a070 con 0x7f3da0101030 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.787+0000 7f3d967fc700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 0x7f3d8c0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.787+0000 7f3da4efd700 1 -- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 msgr2=0x7f3d8c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.787+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f3d9c0d10d0 con 0x7f3da0101030 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.787+0000 7f3da4efd700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 0x7f3d8c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:27:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.791+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d9c013070 con 0x7f3da0101030 2024-04-22T13:27:04.980 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:04.689+0000 7f35973f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:27:04.980 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:04.780+0000 7f35973f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:27:04.980 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:04.983+0000 7f35973f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:27:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.988+0000 7f3da4efd700 1 -- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 msgr2=0x7f3d8c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:04.988+0000 7f3da4efd700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 0x7f3d8c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:27:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.047+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3da01a8040 con 0x7f3da0101030 2024-04-22T13:27:05.045 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:04.931+0000 7f3d4f845200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:27:05.045 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:05.029+0000 7f3d4f845200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:27:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.048+0000 7f3d967fc700 1 -- 172.21.15.5:0/1431971887 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f3d9c09daf0 con 0x7f3da0101030 2024-04-22T13:27:05.045 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 msgr2=0x7f3d8c0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 0x7f3d8c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 msgr2=0x7f3da019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 0x7f3da019b2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000c00 tx=0x7f3d9c00bfd0 comp rx=0 tx=0).stop 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 shutdown_connections 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3da0108ad0 0x7f3da019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da0101030 0x7f3da019b2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f3d8c0a3c90 0x7f3d8c0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 --2- 172.21.15.5:0/1431971887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3da01019d0 0x7f3da019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 >> 172.21.15.5:0/1431971887 conn(0x7f3da0077fb0 msgr2=0x7f3da00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 shutdown_connections 2024-04-22T13:27:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05.050+0000 7f3da7962700 1 -- 172.21.15.5:0/1431971887 wait complete. 2024-04-22T13:27:05.096 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:04.822+0000 7fd04cd31200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:27:05.096 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:04.918+0000 7fd04cd31200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:27:05.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:05 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1431971887' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T13:27:05.284 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:05.178+0000 7f35973f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:27:05.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:05 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1431971887' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T13:27:05.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:05.245+0000 7f3d4f845200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:27:05.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:05 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1431971887' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T13:27:05.521 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:05.130+0000 7fd04cd31200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:27:05.521 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:05.331+0000 7fd04cd31200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:27:05.735 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:05.452+0000 7f3d4f845200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T13:27:05.764 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T13:27:05.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:05.765 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-22T13:27:05.781 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41952sec preferred_lft 41952sec 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T13:27:05.782 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-22T13:27:05.783 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-22T13:27:05.783 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T13:27:05.783 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:27:05.783 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-22T13:27:05.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-04-22T13:27:05.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-04-22T13:27:05.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-04-22T13:27:05.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T13:27:05.784 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-22T13:27:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:27:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:05.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:05.970 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T13:27:05.995 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-22T13:27:05.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:05.998 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-22T13:27:06.034 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:05.770+0000 7f35973f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:27:06.069 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-22T13:27:06.069 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T13:27:06.069 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T13:27:06.069 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T13:27:06.069 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T13:27:06.069 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:27:06.070 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-22T13:27:06.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T13:27:06.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T13:27:06.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T13:27:06.070 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-22T13:27:06.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:27:06.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:06.103+0000 7f3d4f845200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:27:06.270 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:05.957+0000 7fd04cd31200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:27:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:06.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:06.351 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-22T13:27:06.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:06.367 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-04-22T13:27:06.423 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:27:06.423 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-22T13:27:06.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T13:27:06.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T13:27:06.424 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-22T13:27:06.481 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:06.188+0000 7f35973f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:27:06.481 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:06.283+0000 7f35973f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:27:06.481 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:06.484+0000 7f35973f3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:27:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:27:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:06.544 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T13:27:06.545 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-22T13:27:06.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:27:06.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:06.675 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:27:06.676 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-22T13:27:06.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-22T13:27:06.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T13:27:06.676 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-22T13:27:06.704 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:06 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:06.396+0000 7fd04cd31200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:27:06.704 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:06 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:06.497+0000 7fd04cd31200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:27:06.704 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:06 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:06.707+0000 7fd04cd31200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:27:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:27:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:06.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T13:27:06.755 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:06.778 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T13:27:06.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T13:27:06.779 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:06.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:06.573+0000 7f3d4f845200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:27:06.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:06.685+0000 7f3d4f845200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:27:06.823 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-22T13:27:06.823 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-22T13:27:06.823 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/9f193bc0-536a-473d-81e1-fda6e7626db4 2024-04-22T13:27:06.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:06.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/9f193bc0-536a-473d-81e1-fda6e7626db4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-22T13:27:06.914 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-22T13:27:06.914 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-22T13:27:07.014 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-22T13:27:07.014 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-22T13:27:07.014 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.49:3300,172.21.15.107:3300" 2024-04-22T13:27:07.014 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@73e95e94-00aa-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/9f193bc0-536a-473d-81e1-fda6e7626db4 2024-04-22T13:27:07.014 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.5:3300/172.21.15.49:3300/172.21.15.107:3300" will pass to kernel 2024-04-22T13:27:07.016 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-22T13:27:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-22T13:27:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-22T13:27:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-22T13:27:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-22T13:27:07.017 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-22T13:27:07.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:07.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:07.048 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T13:27:07.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T13:27:07.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:07.149 INFO:teuthology.orchestra.run.smithi005.stdout:client15375 2024-04-22T13:27:07.149 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:27:07.149 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T13:27:07.166 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T13:27:07.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T13:27:07.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:07.263 INFO:teuthology.orchestra.run.smithi005.stdout:client15375 2024-04-22T13:27:07.263 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T13:27:07.284 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.006+0000 7f35973f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:27:07.284 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.099+0000 7f35973f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:27:07.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:06.906+0000 7f3d4f845200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:27:07.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T13:27:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:07.321 INFO:teuthology.orchestra.run.smithi005.stdout:client15375 2024-04-22T13:27:07.321 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/73e95e94-00aa-11ef-bc93-c7b262605968.client15375/status') 2024-04-22T13:27:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:27:07.381 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.15375 (3)192.168.0.1:0/3372385335 2024-04-22T13:27:07.381 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-04-22T13:27:07.381 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-04-22T13:27:07.381 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-04-22T13:27:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:70 bytes copied, 2.461e-05 s, 2.8 MB/s 2024-04-22T13:27:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:27:07.389 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T13:27:07.398 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:27:07.398 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:27:07.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 -- 172.21.15.5:0/4169538527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 msgr2=0x7fa5ac10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 --2- 172.21.15.5:0/4169538527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac10a120 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009ca0 tx=0x7fa5a402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 -- 172.21.15.5:0/4169538527 shutdown_connections 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 --2- 172.21.15.5:0/4169538527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac10a120 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 --2- 172.21.15.5:0/4169538527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac105400 0x7fa5ac1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 --2- 172.21.15.5:0/4169538527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac102ad0 0x7fa5ac104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 -- 172.21.15.5:0/4169538527 >> 172.21.15.5:0/4169538527 conn(0x7fa5ac0fc420 msgr2=0x7fa5ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 -- 172.21.15.5:0/4169538527 shutdown_connections 2024-04-22T13:27:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 -- 172.21.15.5:0/4169538527 wait complete. 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.507+0000 7fa5b4359640 1 Processor -- start 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 -- start start 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac102ad0 0x7fa5ac102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac105400 0x7fa5ac100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10db10 con 0x7fa5ac107d30 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10d990 con 0x7fa5ac105400 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b4359640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10dc90 con 0x7fa5ac102ad0 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b28f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac102ad0 0x7fa5ac102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b28f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56956/0 (socket says 172.21.15.5:56956) 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b28f6640 1 -- 172.21.15.5:0/2214432282 learned_addr learned my addr 172.21.15.5:0/2214432282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b18f4640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac105400 0x7fa5ac100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 -- 172.21.15.5:0/2214432282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac105400 msgr2=0x7fa5ac100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac105400 0x7fa5ac100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 -- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 msgr2=0x7fa5ac100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 -- 172.21.15.5:0/2214432282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac101850 con 0x7fa5ac102ad0 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b28f6640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.508+0000 7fa5b20f5640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac102ad0 0x7fa5ac102440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9f0 tx=0x7fa5a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.509+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800ccc0 con 0x7fa5ac102ad0 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.509+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1a22c0 con 0x7fa5ac102ad0 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.509+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a80108e0 con 0x7fa5ac102ad0 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.509+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8018880 con 0x7fa5ac102ad0 2024-04-22T13:27:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.509+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1a2810 con 0x7fa5ac102ad0 2024-04-22T13:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.510+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac0fdf10 con 0x7fa5ac102ad0 2024-04-22T13:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.511+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7fa5a8010430 con 0x7fa5ac102ad0 2024-04-22T13:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.511+0000 7fa5a37fe640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 0x7fa5980a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.511+0000 7fa5b18f4640 1 -- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 msgr2=0x7fa5980a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.512+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fa5a80d0760 con 0x7fa5ac102ad0 2024-04-22T13:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.512+0000 7fa5b18f4640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 0x7fa5980a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:27:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.515+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5a809cb00 con 0x7fa5ac102ad0 2024-04-22T13:27:07.520 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:07.253+0000 7fd04cd31200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:27:07.520 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:07.350+0000 7fd04cd31200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:27:07.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:07.477+0000 7f3d4f845200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:27:07.679 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.339+0000 7f35973f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:27:07.679 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.472+0000 7f35973f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:27:07.679 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.568+0000 7f35973f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:27:07.679 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.682+0000 7f35973f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:27:07.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.713+0000 7fa5b18f4640 1 -- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 msgr2=0x7fa5980a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:07.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.713+0000 7fa5b18f4640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 0x7fa5980a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:27:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.745+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fa5ac02d5c0 con 0x7fa5ac102ad0 2024-04-22T13:27:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.801+0000 7fa5a37fe640 1 -- 172.21.15.5:0/2214432282 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fa5a80a0e00 con 0x7fa5ac102ad0 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.803+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 msgr2=0x7fa5980a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.803+0000 7fa5b4359640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 0x7fa5980a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.803+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac102ad0 msgr2=0x7fa5ac102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.803+0000 7fa5b4359640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac102ad0 0x7fa5ac102440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9f0 tx=0x7fa5a800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 shutdown_connections 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7fa5980a3bf0 0x7fa5980a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac107d30 0x7fa5ac100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac105400 0x7fa5ac100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 --2- 172.21.15.5:0/2214432282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac102ad0 0x7fa5ac102440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 >> 172.21.15.5:0/2214432282 conn(0x7fa5ac0fc420 msgr2=0x7fa5ac1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:07.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 shutdown_connections 2024-04-22T13:27:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.804+0000 7fa5b4359640 1 -- 172.21.15.5:0/2214432282 wait complete. 2024-04-22T13:27:07.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-22T13:27:07.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:07 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2214432282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:27:07.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:07 smithi005 ceph-mon[28006]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:27:07.835 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:07.582+0000 7f3d4f845200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 -- 172.21.15.5:0/947375102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 msgr2=0x7f1788104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 --2- 172.21.15.5:0/947375102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 0x7f1788104e20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 -- 172.21.15.5:0/947375102 shutdown_connections 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 --2- 172.21.15.5:0/947375102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1788107c90 0x7f178810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 --2- 172.21.15.5:0/947375102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1788105360 0x7f1788107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 --2- 172.21.15.5:0/947375102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 0x7f1788104e20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.925+0000 7f178f6d9640 1 -- 172.21.15.5:0/947375102 >> 172.21.15.5:0/947375102 conn(0x7f17880fc420 msgr2=0x7f17880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.926+0000 7f178f6d9640 1 -- 172.21.15.5:0/947375102 shutdown_connections 2024-04-22T13:27:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.926+0000 7f178f6d9640 1 -- 172.21.15.5:0/947375102 wait complete. 2024-04-22T13:27:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.926+0000 7f178f6d9640 1 Processor -- start 2024-04-22T13:27:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 -- start start 2024-04-22T13:27:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 0x7f1788198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1788105360 0x7f1788198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1788107c90 0x7f178819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f178810daf0 con 0x7f1788102a30 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f178810d970 con 0x7f1788105360 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178f6d9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f178810dc70 con 0x7f1788107c90 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178d475640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 0x7f1788198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178d475640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 0x7f1788198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56964/0 (socket says 172.21.15.5:56964) 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1788105360 0x7f1788198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178d475640 1 -- 172.21.15.5:0/3072312835 learned_addr learned my addr 172.21.15.5:0/3072312835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178dc76640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1788107c90 0x7f178819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1788107c90 msgr2=0x7f178819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1788107c90 0x7f178819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 msgr2=0x7f1788198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788102a30 0x7f1788198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 -- 172.21.15.5:0/3072312835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819fe50 con 0x7f1788105360 2024-04-22T13:27:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.927+0000 7f178cc74640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1788105360 0x7f1788198770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f177800e9c0 tx=0x7f177800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.928+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177800ccc0 con 0x7f1788105360 2024-04-22T13:27:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.928+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17881a0140 con 0x7f1788105360 2024-04-22T13:27:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.928+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1778005020 con 0x7f1788105360 2024-04-22T13:27:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.928+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778004a00 con 0x7f1788105360 2024-04-22T13:27:07.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.928+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a0640 con 0x7f1788105360 2024-04-22T13:27:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.929+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17880fdf10 con 0x7f1788105360 2024-04-22T13:27:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.930+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f17780051d0 con 0x7f1788105360 2024-04-22T13:27:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.931+0000 7f177e7fc640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 0x7f17740a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.931+0000 7f178d475640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 msgr2=0x7f17740a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.931+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f17780d03e0 con 0x7f1788105360 2024-04-22T13:27:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.931+0000 7f178d475640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 0x7f17740a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:27:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:07.940+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f177809cdd0 con 0x7f1788105360 2024-04-22T13:27:07.965 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2214432282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:27:07.965 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-mon[31379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:27:07.966 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:07.604+0000 7fd04cd31200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:27:07.966 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:07.742+0000 7fd04cd31200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:27:07.966 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:07.843+0000 7fd04cd31200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:27:07.966 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:07.968+0000 7fd04cd31200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:27:07.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2214432282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:27:07.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-mon[31859]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:27:08.111 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:07.861+0000 7f3d4f845200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:27:08.127 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:08.010+0000 7f3d4f845200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:27:08.128 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:08.114+0000 7f3d4f845200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:27:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:08.132+0000 7f178d475640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 msgr2=0x7f17740a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:08.132+0000 7f178d475640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 0x7f17740a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:27:08.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:08.157+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 --> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1788062430 con 0x7f17740a3bf0 2024-04-22T13:27:08.284 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:07.984+0000 7f35973f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:27:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:08.533+0000 7f178d475640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 msgr2=0x7f17740a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:08.533+0000 7f178d475640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 0x7f17740a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T13:27:08.554 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:08.239+0000 7f3d4f845200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:27:08.771 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:08.281+0000 7fd04cd31200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:08.556+0000 7f3d4f845200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: Standby manager daemon z restarted 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: Standby manager daemon z started 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:27:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:27:08.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:08 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:27:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:08.819+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 20) ==== 138662+0+0 (secure 0 0 0) 0x7f1778098d40 con 0x7f1788105360 2024-04-22T13:27:09.034 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:08.584+0000 7f35973f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:27:09.034 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T13:27:08.797+0000 7f35973f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: Standby manager daemon z restarted 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: Standby manager daemon z started 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:27:09.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:08 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:08.916+0000 7fd04cd31200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y[32755]: 2024-04-22T13:27:09.141+0000 7fd04cd31200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: Standby manager daemon z restarted 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: Standby manager daemon z started 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:27:09.138 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:08 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.107:0/653121439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:27:09.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:09.334+0000 7f178d475640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 msgr2=0x7f17740a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2674774949 2024-04-22T13:27:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:09.334+0000 7f178d475640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 0x7f17740a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T13:27:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:09.452+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f17780cdf40 con 0x7f1788105360 2024-04-22T13:27:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:09.452+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 msgr2=0x7f17740a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:27:09.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:09.452+0000 7f177e7fc640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/2674774949,v1:172.21.15.5:6801/2674774949] conn(0x7f17740a3bf0 0x7f17740a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:09.558 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.209+0000 7f3d4f845200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:27:09.558 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.439+0000 7f3d4f845200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: Standby manager daemon y restarted 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: Standby manager daemon y started 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:27:10.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: Active manager daemon x restarted 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: Activating manager daemon x 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: mgrmap e21: x(active, starting, since 0.0122074s), standbys: z, y 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:27:10.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:27:10.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: Manager daemon x is now available 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:27:10.024 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:09 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:27:10.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-22T13:27:10.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: Standby manager daemon y restarted 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: Standby manager daemon y started 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: Active manager daemon x restarted 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: Activating manager daemon x 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: mgrmap e21: x(active, starting, since 0.0122074s), standbys: z, y 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:27:10.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:27:10.036 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: Manager daemon x is now available 2024-04-22T13:27:10.037 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:10.038 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:27:10.038 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:09 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:27:10.058 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.688+0000 7f3caf179700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.058 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.688+0000 7f3caf179700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.058 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.688+0000 7f3caf179700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.688+0000 7f3caf179700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.688+0000 7f3caf179700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.689+0000 7f3cb217f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.689+0000 7f3cb217f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.689+0000 7f3cb217f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.689+0000 7f3cb217f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:09.689+0000 7f3cb217f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-22T13:27:10.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: Standby manager daemon y restarted 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: Standby manager daemon y started 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.? 172.21.15.49:0/1856178004' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: Active manager daemon x restarted 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: Activating manager daemon x 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: mgrmap e21: x(active, starting, since 0.0122074s), standbys: z, y 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:27:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:27:10.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:27:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: Manager daemon x is now available 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:27:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:09 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:27:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.460+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mon.2 v2:172.21.15.49:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f177805ea60 con 0x7f1788105360 2024-04-22T13:27:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.461+0000 7f177e7fc640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1774035370 0x7f1774035750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.461+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f177809b3c0 con 0x7f1774035370 2024-04-22T13:27:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.463+0000 7f178d475640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1774035370 0x7f1774035750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.465+0000 7f178d475640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1774035370 0x7f1774035750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1784000c00 tx=0x7f178402c7e0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.490+0000 7f177e7fc640 1 -- 172.21.15.5:0/3072312835 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f177809b3c0 con 0x7f1774035370 2024-04-22T13:27:10.486 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1774035370 msgr2=0x7f1774035750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1774035370 0x7f1774035750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1784000c00 tx=0x7f178402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1788105360 msgr2=0x7f1788198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 --2- 172.21.15.5:0/3072312835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1788105360 0x7f1788198770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f177800e9c0 tx=0x7f177800ee80 comp rx=0 tx=0).stop 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178dc76640 1 -- 172.21.15.5:0/3072312835 reap_dead start 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 shutdown_connections 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 >> 172.21.15.5:0/3072312835 conn(0x7f17880fc420 msgr2=0x7f17881048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 shutdown_connections 2024-04-22T13:27:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.493+0000 7f178f6d9640 1 -- 172.21.15.5:0/3072312835 wait complete. 2024-04-22T13:27:10.501 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-22T13:27:10.513 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-22T13:27:10.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 -- 172.21.15.5:0/1121753502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4102990 msgr2=0x7f3de41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/1121753502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4102990 0x7f3de41092b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009ca0 tx=0x7f3dd402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 -- 172.21.15.5:0/1121753502 shutdown_connections 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/1121753502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4102990 0x7f3de41092b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/1121753502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 0x7f3de4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/1121753502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4100e80 0x7f3de41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 -- 172.21.15.5:0/1121753502 >> 172.21.15.5:0/1121753502 conn(0x7f3de40fc3f0 msgr2=0x7f3de40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 -- 172.21.15.5:0/1121753502 shutdown_connections 2024-04-22T13:27:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 -- 172.21.15.5:0/1121753502 wait complete. 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 Processor -- start 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.649+0000 7f3de8dd5640 1 -- start start 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 0x7f3de4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 0x7f3de41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4102990 0x7f3de419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410c9d0 con 0x7f3de4100e80 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410c850 con 0x7f3de4102990 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410cb50 con 0x7f3de4101fd0 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 0x7f3de4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 0x7f3de4197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50728/0 (socket says 172.21.15.5:50728) 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 -- 172.21.15.5:0/4274476669 learned_addr learned my addr 172.21.15.5:0/4274476669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de1d9b640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 0x7f3de41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de2d9d640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4102990 0x7f3de419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 -- 172.21.15.5:0/4274476669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 msgr2=0x7f3de41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 0x7f3de41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 -- 172.21.15.5:0/4274476669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4102990 msgr2=0x7f3de419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4102990 0x7f3de419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 -- 172.21.15.5:0/4274476669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de419eca0 con 0x7f3de4100e80 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de1d9b640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 0x7f3de41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de259c640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 0x7f3de4197080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00ea60 tx=0x7f3ddc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00cde0 con 0x7f3de4100e80 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc0035f0 con 0x7f3de4100e80 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de419ef90 con 0x7f3de4100e80 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc010690 con 0x7f3de4100e80 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.650+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de419f3f0 con 0x7f3de4100e80 2024-04-22T13:27:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.651+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de4108130 con 0x7f3de4100e80 2024-04-22T13:27:10.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.652+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f3ddc0037a0 con 0x7f3de4100e80 2024-04-22T13:27:10.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.653+0000 7f3ddb7fe640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3dd00a39b0 0x7f3dd00a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.653+0000 7f3de1d9b640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3dd00a39b0 0x7f3dd00a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.653+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f3ddc0d0380 con 0x7f3de4100e80 2024-04-22T13:27:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.654+0000 7f3de1d9b640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3dd00a39b0 0x7f3dd00a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3de4198590 tx=0x7f3dcc005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.658+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ddc09cf60 con 0x7f3de4100e80 2024-04-22T13:27:10.733 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:10.474+0000 7f3cac173700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.734 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:10.474+0000 7f3cac173700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.734 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:10.474+0000 7f3cac173700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.734 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:10.474+0000 7f3cac173700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.734 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:27:10.474+0000 7f3cac173700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:27:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.849+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3de4062c50 con 0x7f3de4100e80 2024-04-22T13:27:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.851+0000 7f3ddb7fe640 1 -- 172.21.15.5:0/4274476669 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7f3ddc09c860 con 0x7f3de4100e80 2024-04-22T13:27:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:10.848 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.853+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3dd00a39b0 msgr2=0x7f3dd00a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.853+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3dd00a39b0 0x7f3dd00a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3de4198590 tx=0x7f3dcc005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.853+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 msgr2=0x7f3de4197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.853+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 0x7f3de4197080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00ea60 tx=0x7f3ddc00ef20 comp rx=0 tx=0).stop 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 shutdown_connections 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3dd00a39b0 0x7f3dd00a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de4102990 0x7f3de419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3de4101fd0 0x7f3de41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 --2- 172.21.15.5:0/4274476669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4100e80 0x7f3de4197080 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 >> 172.21.15.5:0/4274476669 conn(0x7f3de40fc3f0 msgr2=0x7f3de4105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 shutdown_connections 2024-04-22T13:27:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.854+0000 7f3de8dd5640 1 -- 172.21.15.5:0/4274476669 wait complete. 2024-04-22T13:27:10.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:27:10.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:27:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 -- 172.21.15.5:0/1374803413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb34103b60 msgr2=0x7feb3410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 --2- 172.21.15.5:0/1374803413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb34103b60 0x7feb3410a3f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009ca0 tx=0x7feb2c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 -- 172.21.15.5:0/1374803413 shutdown_connections 2024-04-22T13:27:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 --2- 172.21.15.5:0/1374803413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb34103b60 0x7feb3410a3f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 --2- 172.21.15.5:0/1374803413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb341031a0 0x7feb34103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 --2- 172.21.15.5:0/1374803413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34101fb0 0x7feb341023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 -- 172.21.15.5:0/1374803413 >> 172.21.15.5:0/1374803413 conn(0x7feb340fd520 msgr2=0x7feb340ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 -- 172.21.15.5:0/1374803413 shutdown_connections 2024-04-22T13:27:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.975+0000 7feb3bd15640 1 -- 172.21.15.5:0/1374803413 wait complete. 2024-04-22T13:27:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 Processor -- start 2024-04-22T13:27:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 -- start start 2024-04-22T13:27:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34101fb0 0x7feb341981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 0x7feb341986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb34103b60 0x7feb3419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410daf0 con 0x7feb341031a0 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410d970 con 0x7feb34103b60 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3bd15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410dc70 con 0x7feb34101fb0 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 0x7feb341986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 0x7feb341986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50744/0 (socket says 172.21.15.5:50744) 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 -- 172.21.15.5:0/2697715822 learned_addr learned my addr 172.21.15.5:0/2697715822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb39ab1640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34101fb0 0x7feb341981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3a2b2640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb34103b60 0x7feb3419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 -- 172.21.15.5:0/2697715822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34101fb0 msgr2=0x7feb341981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34101fb0 0x7feb341981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 -- 172.21.15.5:0/2697715822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb34103b60 msgr2=0x7feb3419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb34103b60 0x7feb3419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb392b0640 1 -- 172.21.15.5:0/2697715822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3419fdc0 con 0x7feb341031a0 2024-04-22T13:27:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.976+0000 7feb3a2b2640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb34103b60 0x7feb3419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.977+0000 7feb392b0640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 0x7feb341986e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e9f0 tx=0x7feb2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.977+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2400cd80 con 0x7feb341031a0 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.977+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb240108e0 con 0x7feb341031a0 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.977+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341a00b0 con 0x7feb341031a0 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.977+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24018980 con 0x7feb341031a0 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.977+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a0650 con 0x7feb341031a0 2024-04-22T13:27:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.978+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb341092d0 con 0x7feb341031a0 2024-04-22T13:27:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.979+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7feb24010430 con 0x7feb341031a0 2024-04-22T13:27:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.980+0000 7feb2affd640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7feb200a3a30 0x7feb200a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.980+0000 7feb39ab1640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7feb200a3a30 0x7feb200a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.980+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7feb240d0420 con 0x7feb341031a0 2024-04-22T13:27:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.980+0000 7feb39ab1640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7feb200a3a30 0x7feb200a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb34103000 tx=0x7feb30005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:10.985+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb2409d000 con 0x7feb341031a0 2024-04-22T13:27:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:10 smithi049 ceph-mon[31379]: [22/Apr/2024:13:27:09] ENGINE Bus STARTING 2024-04-22T13:27:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:10 smithi049 ceph-mon[31379]: [22/Apr/2024:13:27:10] ENGINE Serving on http://172.21.15.5:8765 2024-04-22T13:27:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:10 smithi049 ceph-mon[31379]: mgrmap e22: x(active, since 1.02006s), standbys: z, y 2024-04-22T13:27:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:10 smithi107 ceph-mon[31859]: [22/Apr/2024:13:27:09] ENGINE Bus STARTING 2024-04-22T13:27:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:10 smithi107 ceph-mon[31859]: [22/Apr/2024:13:27:10] ENGINE Serving on http://172.21.15.5:8765 2024-04-22T13:27:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:10 smithi107 ceph-mon[31859]: mgrmap e22: x(active, since 1.02006s), standbys: z, y 2024-04-22T13:27:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-mon[28006]: [22/Apr/2024:13:27:09] ENGINE Bus STARTING 2024-04-22T13:27:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-mon[28006]: [22/Apr/2024:13:27:10] ENGINE Serving on http://172.21.15.5:8765 2024-04-22T13:27:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:10 smithi005 ceph-mon[28006]: mgrmap e22: x(active, since 1.02006s), standbys: z, y 2024-04-22T13:27:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.172+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb34062b50 con 0x7feb341031a0 2024-04-22T13:27:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.173+0000 7feb2affd640 1 -- 172.21.15.5:0/2697715822 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7feb2409c900 con 0x7feb341031a0 2024-04-22T13:27:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:27:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7feb200a3a30 msgr2=0x7feb200a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7feb200a3a30 0x7feb200a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb34103000 tx=0x7feb30005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 msgr2=0x7feb341986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 0x7feb341986e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e9f0 tx=0x7feb2400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 shutdown_connections 2024-04-22T13:27:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb341031a0 0x7feb341986e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7feb200a3a30 0x7feb200a5ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb34103b60 0x7feb3419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 --2- 172.21.15.5:0/2697715822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34101fb0 0x7feb341981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 >> 172.21.15.5:0/2697715822 conn(0x7feb340fd520 msgr2=0x7feb341064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 shutdown_connections 2024-04-22T13:27:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.175+0000 7feb3bd15640 1 -- 172.21.15.5:0/2697715822 wait complete. 2024-04-22T13:27:11.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:27:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 -- 172.21.15.5:0/1178657699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 msgr2=0x7f037410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1178657699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 0x7f037410a0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f036c009ca0 tx=0x7f036c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 -- 172.21.15.5:0/1178657699 shutdown_connections 2024-04-22T13:27:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1178657699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 0x7f037410a0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1178657699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0374105340 0x7f0374107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1178657699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f0374104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 -- 172.21.15.5:0/1178657699 >> 172.21.15.5:0/1178657699 conn(0x7f03740fc420 msgr2=0x7f03740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 -- 172.21.15.5:0/1178657699 shutdown_connections 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.300+0000 7f037b4b3640 1 -- 172.21.15.5:0/1178657699 wait complete. 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 Processor -- start 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 -- start start 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f037419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0374105340 0x7f037419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 0x7f03741a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0374101c70 con 0x7f0374102a10 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0374101af0 con 0x7f0374107c70 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037b4b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0374101df0 con 0x7f0374105340 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f037419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f037419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50754/0 (socket says 172.21.15.5:50754) 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 -- 172.21.15.5:0/1883170425 learned_addr learned my addr 172.21.15.5:0/1883170425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f0378a4e640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0374105340 0x7f037419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f0379a50640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 0x7f03741a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 -- 172.21.15.5:0/1883170425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0374105340 msgr2=0x7f037419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0374105340 0x7f037419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 -- 172.21.15.5:0/1883170425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 msgr2=0x7f03741a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 0x7f03741a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.301+0000 7f037924f640 1 -- 172.21.15.5:0/1883170425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03741a63b0 con 0x7f0374102a10 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.302+0000 7f037924f640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f037419e790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9f0 tx=0x7f036400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.302+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036400cd80 con 0x7f0374102a10 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.302+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03640108e0 con 0x7f0374102a10 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.302+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03741a66a0 con 0x7f0374102a10 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.302+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364018980 con 0x7f0374102a10 2024-04-22T13:27:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.302+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03741a6bf0 con 0x7f0374102a10 2024-04-22T13:27:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.303+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03740fdf10 con 0x7f0374102a10 2024-04-22T13:27:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.304+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0364010430 con 0x7f0374102a10 2024-04-22T13:27:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.305+0000 7f036a7fc640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03600a39b0 0x7f03600a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.305+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f03640d0400 con 0x7f0374102a10 2024-04-22T13:27:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.305+0000 7f0378a4e640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03600a39b0 0x7f03600a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.305+0000 7f0378a4e640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03600a39b0 0x7f03600a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f037419fca0 tx=0x7f0370005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.309+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f036409c950 con 0x7f0374102a10 2024-04-22T13:27:11.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.552+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0374062b50 con 0x7f0374102a10 2024-04-22T13:27:11.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.553+0000 7f036a7fc640 1 -- 172.21.15.5:0/1883170425 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f03640a0c50 con 0x7f0374102a10 2024-04-22T13:27:11.549 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03600a39b0 msgr2=0x7f03600a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03600a39b0 0x7f03600a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f037419fca0 tx=0x7f0370005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 msgr2=0x7f037419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f037419e790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9f0 tx=0x7f036400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 shutdown_connections 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03600a39b0 0x7f03600a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0374107c70 0x7f03741a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0374105340 0x7f037419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 --2- 172.21.15.5:0/1883170425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0374102a10 0x7f037419e790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 >> 172.21.15.5:0/1883170425 conn(0x7f03740fc420 msgr2=0x7f0374103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 shutdown_connections 2024-04-22T13:27:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.556+0000 7f037b4b3640 1 -- 172.21.15.5:0/1883170425 wait complete. 2024-04-22T13:27:11.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:27:11.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 -- 172.21.15.5:0/4153353342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 msgr2=0x7f281c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 --2- 172.21.15.5:0/4153353342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c102460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 -- 172.21.15.5:0/4153353342 shutdown_connections 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 --2- 172.21.15.5:0/4153353342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c102460 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 --2- 172.21.15.5:0/4153353342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 0x7f281c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 --2- 172.21.15.5:0/4153353342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 0x7f281c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 -- 172.21.15.5:0/4153353342 >> 172.21.15.5:0/4153353342 conn(0x7f281c0fc420 msgr2=0x7f281c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 -- 172.21.15.5:0/4153353342 shutdown_connections 2024-04-22T13:27:11.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.695+0000 7f2820b0e640 1 -- 172.21.15.5:0/4153353342 wait complete. 2024-04-22T13:27:11.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 Processor -- start 2024-04-22T13:27:11.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 -- start start 2024-04-22T13:27:11.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 0x7f281c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 0x7f281c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10c9d0 con 0x7f281c101fe0 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10c850 con 0x7f281c1029a0 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.696+0000 7f2820b0e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10cb50 con 0x7f281c100e90 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f281ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 0x7f281c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f281a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 0x7f281c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50784/0 (socket says 172.21.15.5:50784) 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 -- 172.21.15.5:0/2468117438 learned_addr learned my addr 172.21.15.5:0/2468117438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 -- 172.21.15.5:0/2468117438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 msgr2=0x7f281c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 0x7f281c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 -- 172.21.15.5:0/2468117438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 msgr2=0x7f281c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 0x7f281c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 -- 172.21.15.5:0/2468117438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c19fe10 con 0x7f281c101fe0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f281a59c640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 0x7f281c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f281ad9d640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 0x7f281c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2819d9b640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c1987d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c043020 con 0x7f281c101fe0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c1a00a0 con 0x7f281c101fe0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c004350 con 0x7f281c101fe0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c003b80 con 0x7f281c101fe0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.697+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c1a0640 con 0x7f281c101fe0 2024-04-22T13:27:11.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.699+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c1a0960 con 0x7f281c101fe0 2024-04-22T13:27:11.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.700+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f280c046020 con 0x7f281c101fe0 2024-04-22T13:27:11.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.700+0000 7f280b7fe640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f28040a39b0 0x7f28040a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:11.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.700+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f280c0f2650 con 0x7f281c101fe0 2024-04-22T13:27:11.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.700+0000 7f281a59c640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f28040a39b0 0x7f28040a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:11.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.701+0000 7f281a59c640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f28040a39b0 0x7f28040a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2810009fc0 tx=0x7f2810005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:11.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.704+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f280c04d020 con 0x7f281c101fe0 2024-04-22T13:27:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.835+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7f280c0bb1a0 con 0x7f281c101fe0 2024-04-22T13:27:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.900+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f281c1a0b30 con 0x7f281c101fe0 2024-04-22T13:27:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.901+0000 7f280b7fe640 1 -- 172.21.15.5:0/2468117438 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f280c0bf230 con 0x7f281c101fe0 2024-04-22T13:27:11.897 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f28040a39b0 msgr2=0x7f28040a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f28040a39b0 0x7f28040a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2810009fc0 tx=0x7f2810005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 msgr2=0x7f281c1987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c1987d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c005dc0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 shutdown_connections 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101fe0 0x7f281c1987d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f28040a39b0 0x7f28040a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f281c1029a0 0x7f281c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 --2- 172.21.15.5:0/2468117438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f281c100e90 0x7f281c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 >> 172.21.15.5:0/2468117438 conn(0x7f281c0fc420 msgr2=0x7f281c069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 shutdown_connections 2024-04-22T13:27:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:11.903+0000 7f2820b0e640 1 -- 172.21.15.5:0/2468117438 wait complete. 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: [22/Apr/2024:13:27:10] ENGINE Serving on https://172.21.15.5:7150 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: [22/Apr/2024:13:27:10] ENGINE Bus STARTED 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4274476669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2697715822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:11.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:11.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1883170425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:11.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:11.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:11.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:11.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:11 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:11.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 -- 172.21.15.5:0/3351309713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c102a40 msgr2=0x7fac2c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 --2- 172.21.15.5:0/3351309713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c102a40 0x7fac2c104e30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fac14009900 tx=0x7fac1402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 -- 172.21.15.5:0/3351309713 shutdown_connections 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 --2- 172.21.15.5:0/3351309713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac2c107ca0 0x7fac2c10a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 --2- 172.21.15.5:0/3351309713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c105370 0x7fac2c107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 --2- 172.21.15.5:0/3351309713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c102a40 0x7fac2c104e30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 -- 172.21.15.5:0/3351309713 >> 172.21.15.5:0/3351309713 conn(0x7fac2c0fc3f0 msgr2=0x7fac2c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 -- 172.21.15.5:0/3351309713 shutdown_connections 2024-04-22T13:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 -- 172.21.15.5:0/3351309713 wait complete. 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 Processor -- start 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.019+0000 7fac32352640 1 -- start start 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac32352640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac2c102a40 0x7fac2c1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac32352640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 0x7fac2c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac32352640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c107ca0 0x7fac2c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac32352640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c10dad0 con 0x7fac2c105370 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac32352640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c10d950 con 0x7fac2c107ca0 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac32352640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c10dc50 con 0x7fac2c102a40 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 0x7fac2c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 0x7fac2c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50804/0 (socket says 172.21.15.5:50804) 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 -- 172.21.15.5:0/809653266 learned_addr learned my addr 172.21.15.5:0/809653266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2bfff640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac2c102a40 0x7fac2c1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac308ef640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c107ca0 0x7fac2c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 -- 172.21.15.5:0/809653266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac2c102a40 msgr2=0x7fac2c1023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac2c102a40 0x7fac2c1023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 -- 172.21.15.5:0/809653266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c107ca0 msgr2=0x7fac2c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c107ca0 0x7fac2c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 -- 172.21.15.5:0/809653266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1017c0 con 0x7fac2c105370 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.020+0000 7fac2b7fe640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 0x7fac2c1009f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e9c0 tx=0x7fac1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.021+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00cd80 con 0x7fac2c105370 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.021+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054d0 con 0x7fac2c105370 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.021+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a22f0 con 0x7fac2c105370 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.021+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c010430 con 0x7fac2c105370 2024-04-22T13:27:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.021+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1a2840 con 0x7fac2c105370 2024-04-22T13:27:12.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.022+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c0fdee0 con 0x7fac2c105370 2024-04-22T13:27:12.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.023+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7fac1c004fb0 con 0x7fac2c105370 2024-04-22T13:27:12.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.024+0000 7fac297fa640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac180a3db0 0x7fac180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.024+0000 7fac2bfff640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac180a3db0 0x7fac180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.024+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fac1c0d0f60 con 0x7fac2c105370 2024-04-22T13:27:12.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.025+0000 7fac2bfff640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac180a3db0 0x7fac180a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac14009230 tx=0x7fac14005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.028+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac1c053ff0 con 0x7fac2c105370 2024-04-22T13:27:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.221+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac2c02d810 con 0x7fac2c105370 2024-04-22T13:27:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.223+0000 7fac297fa640 1 -- 172.21.15.5:0/809653266 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7fac1c09d4c0 con 0x7fac2c105370 2024-04-22T13:27:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:12.219 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac180a3db0 msgr2=0x7fac180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac180a3db0 0x7fac180a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac14009230 tx=0x7fac14005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 msgr2=0x7fac2c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 0x7fac2c1009f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e9c0 tx=0x7fac1c00ee80 comp rx=0 tx=0).stop 2024-04-22T13:27:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 shutdown_connections 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac2c105370 0x7fac2c1009f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac180a3db0 0x7fac180a6270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c107ca0 0x7fac2c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 --2- 172.21.15.5:0/809653266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac2c102a40 0x7fac2c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 >> 172.21.15.5:0/809653266 conn(0x7fac2c0fc3f0 msgr2=0x7fac2c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 shutdown_connections 2024-04-22T13:27:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.225+0000 7fac32352640 1 -- 172.21.15.5:0/809653266 wait complete. 2024-04-22T13:27:12.233 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:27:12.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: [22/Apr/2024:13:27:10] ENGINE Serving on https://172.21.15.5:7150 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: [22/Apr/2024:13:27:10] ENGINE Bus STARTED 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4274476669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2697715822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1883170425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:11 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: [22/Apr/2024:13:27:10] ENGINE Serving on https://172.21.15.5:7150 2024-04-22T13:27:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: [22/Apr/2024:13:27:10] ENGINE Bus STARTED 2024-04-22T13:27:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4274476669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2697715822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1883170425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:11 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 -- 172.21.15.5:0/4006221165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c075f30 msgr2=0x7f853c074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 --2- 172.21.15.5:0/4006221165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c075f30 0x7f853c074380 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f852c009900 tx=0x7f852c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 -- 172.21.15.5:0/4006221165 shutdown_connections 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 --2- 172.21.15.5:0/4006221165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c109e30 0x7f853c10c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 --2- 172.21.15.5:0/4006221165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c074950 0x7f853c074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 --2- 172.21.15.5:0/4006221165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c075f30 0x7f853c074380 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 -- 172.21.15.5:0/4006221165 >> 172.21.15.5:0/4006221165 conn(0x7f853c0fd510 msgr2=0x7f853c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 -- 172.21.15.5:0/4006221165 shutdown_connections 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.346+0000 7f8541611640 1 -- 172.21.15.5:0/4006221165 wait complete. 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 Processor -- start 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 -- start start 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c074950 0x7f853c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c075f30 0x7f853c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 0x7f853c1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f853c10fce0 con 0x7f853c109e30 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f853c10fb60 con 0x7f853c075f30 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f8541611640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f853c10fe60 con 0x7f853c074950 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 0x7f853c1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.347+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 0x7f853c1a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50820/0 (socket says 172.21.15.5:50820) 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 -- 172.21.15.5:0/2452351350 learned_addr learned my addr 172.21.15.5:0/2452351350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853a7fc640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c075f30 0x7f853c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853affd640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c074950 0x7f853c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 -- 172.21.15.5:0/2452351350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c074950 msgr2=0x7f853c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c074950 0x7f853c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 -- 172.21.15.5:0/2452351350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c075f30 msgr2=0x7f853c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c075f30 0x7f853c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 -- 172.21.15.5:0/2452351350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c1a2020 con 0x7f853c109e30 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853a7fc640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c075f30 0x7f853c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853affd640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c074950 0x7f853c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f853b7fe640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 0x7f853c1a19b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f853400eaa0 tx=0x7f853400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400cdb0 con 0x7f853c109e30 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8534005020 con 0x7f853c109e30 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1a2310 con 0x7f853c109e30 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340116d0 con 0x7f853c109e30 2024-04-22T13:27:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.348+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c1a2810 con 0x7f853c109e30 2024-04-22T13:27:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.350+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c108bf0 con 0x7f853c109e30 2024-04-22T13:27:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.351+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7f85340051d0 con 0x7f853c109e30 2024-04-22T13:27:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.351+0000 7f8523fff640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85240a3db0 0x7f85240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.351+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f85340d0aa0 con 0x7f853c109e30 2024-04-22T13:27:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.351+0000 7f853affd640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85240a3db0 0x7f85240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.352+0000 7f853affd640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85240a3db0 0x7f85240a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f852c009230 tx=0x7f852c005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.356+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8534016030 con 0x7f853c109e30 2024-04-22T13:27:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.543+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f853c062b50 con 0x7f853c109e30 2024-04-22T13:27:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.544+0000 7f8523fff640 1 -- 172.21.15.5:0/2452351350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f853409c9b0 con 0x7f853c109e30 2024-04-22T13:27:12.540 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:27:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85240a3db0 msgr2=0x7f85240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85240a3db0 0x7f85240a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f852c009230 tx=0x7f852c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 msgr2=0x7f853c1a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 0x7f853c1a19b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f853400eaa0 tx=0x7f853400ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 shutdown_connections 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f853c109e30 0x7f853c1a19b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85240a3db0 0x7f85240a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f853c075f30 0x7f853c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 --2- 172.21.15.5:0/2452351350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f853c074950 0x7f853c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 >> 172.21.15.5:0/2452351350 conn(0x7f853c0fd510 msgr2=0x7f853c105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 shutdown_connections 2024-04-22T13:27:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.546+0000 7f8541611640 1 -- 172.21.15.5:0/2452351350 wait complete. 2024-04-22T13:27:12.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:27:12.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3540735922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c101fe0 msgr2=0x7f689c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3540735922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c101fe0 0x7f689c102460 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6888009900 tx=0x7f688802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3540735922 shutdown_connections 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3540735922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c101fe0 0x7f689c102460 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3540735922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f689c1029a0 0x7f689c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3540735922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c100e90 0x7f689c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3540735922 >> 172.21.15.5:0/3540735922 conn(0x7f689c0fc420 msgr2=0x7f689c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3540735922 shutdown_connections 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.666+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3540735922 wait complete. 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 Processor -- start 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 -- start start 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f689c100e90 0x7f689c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c101fe0 0x7f689c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 0x7f689c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10c9d0 con 0x7f689c1029a0 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10c850 con 0x7f689c101fe0 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a2d7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10cb50 con 0x7f689c100e90 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a131b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 0x7f689c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a131b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 0x7f689c19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50830/0 (socket says 172.21.15.5:50830) 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a131b640 1 -- 172.21.15.5:0/3471849925 learned_addr learned my addr 172.21.15.5:0/3471849925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f689bfff640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c101fe0 0x7f689c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.667+0000 7f68a0b1a640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f689c100e90 0x7f689c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a131b640 1 -- 172.21.15.5:0/3471849925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f689c100e90 msgr2=0x7f689c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a131b640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f689c100e90 0x7f689c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a131b640 1 -- 172.21.15.5:0/3471849925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c101fe0 msgr2=0x7f689c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a131b640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c101fe0 0x7f689c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a131b640 1 -- 172.21.15.5:0/3471849925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c19fe30 con 0x7f689c1029a0 2024-04-22T13:27:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f689bfff640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c101fe0 0x7f689c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a131b640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 0x7f689c19f7c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f689400eaa0 tx=0x7f689400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689400cdb0 con 0x7f689c1029a0 2024-04-22T13:27:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1a0120 con 0x7f689c1029a0 2024-04-22T13:27:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6894005020 con 0x7f689c1029a0 2024-04-22T13:27:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940116d0 con 0x7f689c1029a0 2024-04-22T13:27:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.668+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a0640 con 0x7f689c1029a0 2024-04-22T13:27:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.669+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c1a0900 con 0x7f689c1029a0 2024-04-22T13:27:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.670+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7f68940051d0 con 0x7f689c1029a0 2024-04-22T13:27:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.671+0000 7f6899ffb640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f688c0a3db0 0x7f688c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.671+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f68940d0b50 con 0x7f689c1029a0 2024-04-22T13:27:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.671+0000 7f68a0b1a640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f688c0a3db0 0x7f688c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.672+0000 7f68a0b1a640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f688c0a3db0 0x7f688c0a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6890005fd0 tx=0x7f6890005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.680+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6894016030 con 0x7f689c1029a0 2024-04-22T13:27:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.872+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f689c1a0ad0 con 0x7f689c1029a0 2024-04-22T13:27:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.872+0000 7f6899ffb640 1 -- 172.21.15.5:0/3471849925 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f689409d0b0 con 0x7f689c1029a0 2024-04-22T13:27:12.869 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f688c0a3db0 msgr2=0x7f688c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f688c0a3db0 0x7f688c0a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6890005fd0 tx=0x7f6890005850 comp rx=0 tx=0).stop 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 msgr2=0x7f689c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 0x7f689c19f7c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f689400eaa0 tx=0x7f689400ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 shutdown_connections 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f689c1029a0 0x7f689c19f7c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f688c0a3db0 0x7f688c0a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689c101fe0 0x7f689c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 --2- 172.21.15.5:0/3471849925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f689c100e90 0x7f689c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 >> 172.21.15.5:0/3471849925 conn(0x7f689c0fc420 msgr2=0x7f689c069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 shutdown_connections 2024-04-22T13:27:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.876+0000 7f68a2d7e640 1 -- 172.21.15.5:0/3471849925 wait complete. 2024-04-22T13:27:12.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:27:12.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 -- 172.21.15.5:0/1972947508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c103b60 msgr2=0x7ff43c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 --2- 172.21.15.5:0/1972947508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c103b60 0x7ff43c10a3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff434009900 tx=0x7ff43402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:12.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 -- 172.21.15.5:0/1972947508 shutdown_connections 2024-04-22T13:27:12.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 --2- 172.21.15.5:0/1972947508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c103b60 0x7ff43c10a3f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 --2- 172.21.15.5:0/1972947508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c1031a0 0x7ff43c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 --2- 172.21.15.5:0/1972947508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c101fb0 0x7ff43c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 -- 172.21.15.5:0/1972947508 >> 172.21.15.5:0/1972947508 conn(0x7ff43c0fd520 msgr2=0x7ff43c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:12.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 -- 172.21.15.5:0/1972947508 shutdown_connections 2024-04-22T13:27:12.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.998+0000 7ff442a81640 1 -- 172.21.15.5:0/1972947508 wait complete. 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 Processor -- start 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 -- start start 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 0x7ff43c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c1031a0 0x7ff43c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c103b60 0x7ff43c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10daf0 con 0x7ff43c101fb0 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10d970 con 0x7ff43c103b60 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff442a81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10dc70 con 0x7ff43c1031a0 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44081d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 0x7ff43c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44081d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 0x7ff43c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50848/0 (socket says 172.21.15.5:50848) 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44081d640 1 -- 172.21.15.5:0/2029409103 learned_addr learned my addr 172.21.15.5:0/2029409103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44101e640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c103b60 0x7ff43c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff43bfff640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c1031a0 0x7ff43c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:12.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44081d640 1 -- 172.21.15.5:0/2029409103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c1031a0 msgr2=0x7ff43c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44081d640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c1031a0 0x7ff43c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:12.999+0000 7ff44081d640 1 -- 172.21.15.5:0/2029409103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c103b60 msgr2=0x7ff43c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff44081d640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c103b60 0x7ff43c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff44081d640 1 -- 172.21.15.5:0/2029409103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c19fdc0 con 0x7ff43c101fb0 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff44101e640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c103b60 0x7ff43c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff44081d640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 0x7ff43c1981a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00eb70 tx=0x7ff42c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003b70 con 0x7ff43c101fb0 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c1a00b0 con 0x7ff43c101fb0 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42c00ee70 con 0x7ff43c101fb0 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c010650 con 0x7ff43c101fb0 2024-04-22T13:27:12.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.000+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a0600 con 0x7ff43c101fb0 2024-04-22T13:27:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.002+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c109320 con 0x7ff43c101fb0 2024-04-22T13:27:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.002+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7ff42c004020 con 0x7ff43c101fb0 2024-04-22T13:27:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.003+0000 7ff439ffb640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4240a3db0 0x7ff4240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.003+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff42c0d1080 con 0x7ff43c101fb0 2024-04-22T13:27:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.003+0000 7ff43bfff640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4240a3db0 0x7ff4240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.004+0000 7ff43bfff640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4240a3db0 0x7ff4240a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff43c1996b0 tx=0x7ff430005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.008+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff42c09d5e0 con 0x7ff43c101fb0 2024-04-22T13:27:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.220+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff43c02d800 con 0x7ff43c101fb0 2024-04-22T13:27:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.221+0000 7ff439ffb640 1 -- 172.21.15.5:0/2029409103 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7ff42c09cee0 con 0x7ff43c101fb0 2024-04-22T13:27:13.217 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.222+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4240a3db0 msgr2=0x7ff4240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.222+0000 7ff442a81640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4240a3db0 0x7ff4240a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff43c1996b0 tx=0x7ff430005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 msgr2=0x7ff43c1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 0x7ff43c1981a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00eb70 tx=0x7ff42c00c6a0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 shutdown_connections 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4240a3db0 0x7ff4240a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff43c103b60 0x7ff43c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c1031a0 0x7ff43c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 --2- 172.21.15.5:0/2029409103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff43c101fb0 0x7ff43c1981a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 >> 172.21.15.5:0/2029409103 conn(0x7ff43c0fd520 msgr2=0x7ff43c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 shutdown_connections 2024-04-22T13:27:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.223+0000 7ff442a81640 1 -- 172.21.15.5:0/2029409103 wait complete. 2024-04-22T13:27:13.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:13.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T13:27:13.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2468117438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:13.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/809653266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2452351350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:12 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.231 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-22T13:27:13.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2468117438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/809653266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2452351350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:12 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2468117438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/809653266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2452351350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:12 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 -- 172.21.15.5:0/4181547982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc101fd0 msgr2=0x7fbcdc102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 --2- 172.21.15.5:0/4181547982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc101fd0 0x7fbcdc102450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009900 tx=0x7fbcc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 -- 172.21.15.5:0/4181547982 shutdown_connections 2024-04-22T13:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 --2- 172.21.15.5:0/4181547982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc101fd0 0x7fbcdc102450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 --2- 172.21.15.5:0/4181547982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc102990 0x7fbcdc1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 --2- 172.21.15.5:0/4181547982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc100e80 0x7fbcdc1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 -- 172.21.15.5:0/4181547982 >> 172.21.15.5:0/4181547982 conn(0x7fbcdc0fc3f0 msgr2=0x7fbcdc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 -- 172.21.15.5:0/4181547982 shutdown_connections 2024-04-22T13:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.339+0000 7fbce29fb640 1 -- 172.21.15.5:0/4181547982 wait complete. 2024-04-22T13:27:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 Processor -- start 2024-04-22T13:27:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 -- start start 2024-04-22T13:27:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 0x7fbcdc197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc101fd0 0x7fbcdc1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc102990 0x7fbcdc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10c9d0 con 0x7fbcdc100e80 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10c850 con 0x7fbcdc102990 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce29fb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10cb50 con 0x7fbcdc101fd0 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbcdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 0x7fbcdc197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbcdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 0x7fbcdc197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50874/0 (socket says 172.21.15.5:50874) 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbcdbfff640 1 -- 172.21.15.5:0/2032131645 learned_addr learned my addr 172.21.15.5:0/2032131645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbcdb7fe640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc101fd0 0x7fbcdc1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.340+0000 7fbce0f98640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc102990 0x7fbcdc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcdbfff640 1 -- 172.21.15.5:0/2032131645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc101fd0 msgr2=0x7fbcdc1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcdbfff640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc101fd0 0x7fbcdc1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcdbfff640 1 -- 172.21.15.5:0/2032131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc102990 msgr2=0x7fbcdc19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcdbfff640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc102990 0x7fbcdc19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcdbfff640 1 -- 172.21.15.5:0/2032131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc19eca0 con 0x7fbcdc100e80 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcdbfff640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 0x7fbcdc197080 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c920 tx=0x7fbccc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc007c70 con 0x7fbcdc100e80 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbccc007e10 con 0x7fbcdc100e80 2024-04-22T13:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc0692b0 con 0x7fbcdc100e80 2024-04-22T13:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc018820 con 0x7fbcdc100e80 2024-04-22T13:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.341+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc0697d0 con 0x7fbcdc100e80 2024-04-22T13:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.342+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc068e00 con 0x7fbcdc100e80 2024-04-22T13:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.344+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7fbccc00f450 con 0x7fbcdc100e80 2024-04-22T13:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.344+0000 7fbcd97fa640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcc80a3db0 0x7fbcc80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.344+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbccc0d09f0 con 0x7fbcdc100e80 2024-04-22T13:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.344+0000 7fbcdb7fe640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcc80a3db0 0x7fbcc80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.345+0000 7fbcdb7fe640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcc80a3db0 0x7fbcc80a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009230 tx=0x7fbcc402c740 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:13.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.349+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbccc014030 con 0x7fbcdc100e80 2024-04-22T13:27:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.587+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcdc107ba0 con 0x7fbcdc100e80 2024-04-22T13:27:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.588+0000 7fbcd97fa640 1 -- 172.21.15.5:0/2032131645 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7fbccc09cf50 con 0x7fbcdc100e80 2024-04-22T13:27:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:13.584 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcc80a3db0 msgr2=0x7fbcc80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcc80a3db0 0x7fbcc80a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009230 tx=0x7fbcc402c740 comp rx=0 tx=0).stop 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 msgr2=0x7fbcdc197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 0x7fbcdc197080 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c920 tx=0x7fbccc00cde0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 shutdown_connections 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcc80a3db0 0x7fbcc80a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc102990 0x7fbcdc19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc101fd0 0x7fbcdc1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 --2- 172.21.15.5:0/2032131645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcdc100e80 0x7fbcdc197080 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 >> 172.21.15.5:0/2032131645 conn(0x7fbcdc0fc3f0 msgr2=0x7fbcdc105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 shutdown_connections 2024-04-22T13:27:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.590+0000 7fbce29fb640 1 -- 172.21.15.5:0/2032131645 wait complete. 2024-04-22T13:27:13.597 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-22T13:27:13.597 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-22T13:27:13.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:27:13.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2615485541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f01010a0 msgr2=0x7ff1f0101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2615485541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f01010a0 0x7ff1f0101500 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009900 tx=0x7ff1e402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:13.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2615485541 shutdown_connections 2024-04-22T13:27:13.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2615485541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f01010a0 0x7ff1f0101500 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2615485541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1f0101a40 0x7ff1f010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2615485541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f0105dd0 0x7ff1f01061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.705+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2615485541 >> 172.21.15.5:0/2615485541 conn(0x7ff1f0077c40 msgr2=0x7ff1f0078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2615485541 shutdown_connections 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2615485541 wait complete. 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 Processor -- start 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 -- start start 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f01010a0 0x7ff1f01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 0x7ff1f0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1f0105dd0 0x7ff1f019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0111180 con 0x7ff1f0101a40 2024-04-22T13:27:13.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0111000 con 0x7ff1f01010a0 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.706+0000 7ff1f5e21640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0111300 con 0x7ff1f0105dd0 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 0x7ff1f0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 0x7ff1f0197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50882/0 (socket says 172.21.15.5:50882) 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 -- 172.21.15.5:0/2894169228 learned_addr learned my addr 172.21.15.5:0/2894169228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1ef7fe640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f01010a0 0x7ff1f01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1effff640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1f0105dd0 0x7ff1f019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 -- 172.21.15.5:0/2894169228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1f0105dd0 msgr2=0x7ff1f019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1f0105dd0 0x7ff1f019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 -- 172.21.15.5:0/2894169228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f01010a0 msgr2=0x7ff1f01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f01010a0 0x7ff1f01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 -- 172.21.15.5:0/2894169228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f019c680 con 0x7ff1f0101a40 2024-04-22T13:27:13.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1eeffd640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 0x7ff1f0197c20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005940 tx=0x7ff1e402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4043020 con 0x7ff1f0101a40 2024-04-22T13:27:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e402ca70 con 0x7ff1f0101a40 2024-04-22T13:27:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f019c910 con 0x7ff1f0101a40 2024-04-22T13:27:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40039f0 con 0x7ff1f0101a40 2024-04-22T13:27:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.707+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01a4200 con 0x7ff1f0101a40 2024-04-22T13:27:13.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.708+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0102540 con 0x7ff1f0101a40 2024-04-22T13:27:13.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.709+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7ff1e4046020 con 0x7ff1f0101a40 2024-04-22T13:27:13.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.710+0000 7ff1ecff9640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff1d80a3db0 0x7ff1d80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:13.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.710+0000 7ff1ef7fe640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff1d80a3db0 0x7ff1d80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:13.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.710+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff1e40f28b0 con 0x7ff1f0101a40 2024-04-22T13:27:13.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.711+0000 7ff1ef7fe640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff1d80a3db0 0x7ff1d80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009bd0 tx=0x7ff1e0005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:13.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.715+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1e404d020 con 0x7ff1f0101a40 2024-04-22T13:27:13.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-22T13:27:13.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-22T13:27:13.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-04-22T13:27:13.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-22T13:27:13.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:13.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:13.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.876+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7ff1e40bae30 con 0x7ff1f0101a40 2024-04-22T13:27:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.922+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1f0062390 con 0x7ff1d80a3db0 2024-04-22T13:27:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.929+0000 7ff1ecff9640 1 -- 172.21.15.5:0/2894169228 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257789 (secure 0 0 0) 0x7ff1f0062390 con 0x7ff1d80a3db0 2024-04-22T13:27:13.928 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.932+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff1d80a3db0 msgr2=0x7ff1d80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff1d80a3db0 0x7ff1d80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009bd0 tx=0x7ff1e0005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 msgr2=0x7ff1f0197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 0x7ff1f0197c20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005940 tx=0x7ff1e402c5f0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 shutdown_connections 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1f0101a40 0x7ff1f0197c20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff1d80a3db0 0x7ff1d80a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1f0105dd0 0x7ff1f019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 --2- 172.21.15.5:0/2894169228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1f01010a0 0x7ff1f01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 >> 172.21.15.5:0/2894169228 conn(0x7ff1f0077c40 msgr2=0x7ff1f00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 shutdown_connections 2024-04-22T13:27:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:13.933+0000 7ff1f5e21640 1 -- 172.21.15.5:0/2894169228 wait complete. 2024-04-22T13:27:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T13:27:13.460500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187956,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1411,"num_read_kb":870,"num_write":549,"num_write_kb":4397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":933,"ondisk_log_size":933,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":413824,"kb_used_data":14536,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528000,"statfs":{"total":1151940427776,"available":1151516672000,"internally_reserved":0,"allocated":14884864,"data_stored":8299033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988589"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563085+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:27:09.563085+0000","last_peered":"2024-04-22T13:27:09.563085+0000","last_clean":"2024-04-22T13:27:09.563085+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:27:09.563085+0000","last_undegraded":"2024-04-22T13:27:09.563085+0000","last_fullsized":"2024-04-22T13:27:09.563085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564651+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:27:09.564651+0000","last_peered":"2024-04-22T13:27:09.564651+0000","last_clean":"2024-04-22T13:27:09.564651+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:27:09.564651+0000","last_undegraded":"2024-04-22T13:27:09.564651+0000","last_fullsized":"2024-04-22T13:27:09.564651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564621+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:27:09.564621+0000","last_peered":"2024-04-22T13:27:09.564621+0000","last_clean":"2024-04-22T13:27:09.564621+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:27:09.564621+0000","last_undegraded":"2024-04-22T13:27:09.564621+0000","last_fullsized":"2024-04-22T13:27:09.564621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563725+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:27:09.563725+0000","last_peered":"2024-04-22T13:27:09.563725+0000","last_clean":"2024-04-22T13:27:09.563725+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:27:09.563725+0000","last_undegraded":"2024-04-22T13:27:09.563725+0000","last_fullsized":"2024-04-22T13:27:09.563725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448296+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:27:09.448296+0000","last_peered":"2024-04-22T13:27:09.448296+0000","last_clean":"2024-04-22T13:27:09.448296+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:27:09.448296+0000","last_undegraded":"2024-04-22T13:27:09.448296+0000","last_fullsized":"2024-04-22T13:27:09.448296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564063+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:27:09.564063+0000","last_peered":"2024-04-22T13:27:09.564063+0000","last_clean":"2024-04-22T13:27:09.564063+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:27:09.564063+0000","last_undegraded":"2024-04-22T13:27:09.564063+0000","last_fullsized":"2024-04-22T13:27:09.564063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540227+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:27:09.540227+0000","last_peered":"2024-04-22T13:27:09.540227+0000","last_clean":"2024-04-22T13:27:09.540227+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:27:09.540227+0000","last_undegraded":"2024-04-22T13:27:09.540227+0000","last_fullsized":"2024-04-22T13:27:09.540227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563416+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:27:09.563416+0000","last_peered":"2024-04-22T13:27:09.563416+0000","last_clean":"2024-04-22T13:27:09.563416+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:27:09.563416+0000","last_undegraded":"2024-04-22T13:27:09.563416+0000","last_fullsized":"2024-04-22T13:27:09.563416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448852+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:27:09.448852+0000","last_peered":"2024-04-22T13:27:09.448852+0000","last_clean":"2024-04-22T13:27:09.448852+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:27:09.448852+0000","last_undegraded":"2024-04-22T13:27:09.448852+0000","last_fullsized":"2024-04-22T13:27:09.448852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564466+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:27:09.564466+0000","last_peered":"2024-04-22T13:27:09.564466+0000","last_clean":"2024-04-22T13:27:09.564466+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:27:09.564466+0000","last_undegraded":"2024-04-22T13:27:09.564466+0000","last_fullsized":"2024-04-22T13:27:09.564466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563562+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:27:09.563562+0000","last_peered":"2024-04-22T13:27:09.563562+0000","last_clean":"2024-04-22T13:27:09.563562+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:27:09.563562+0000","last_undegraded":"2024-04-22T13:27:09.563562+0000","last_fullsized":"2024-04-22T13:27:09.563562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563593+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:27:09.563593+0000","last_peered":"2024-04-22T13:27:09.563593+0000","last_clean":"2024-04-22T13:27:09.563593+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:27:09.563593+0000","last_undegraded":"2024-04-22T13:27:09.563593+0000","last_fullsized":"2024-04-22T13:27:09.563593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563776+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:27:09.563776+0000","last_peered":"2024-04-22T13:27:09.563776+0000","last_clean":"2024-04-22T13:27:09.563776+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:27:09.563776+0000","last_undegraded":"2024-04-22T13:27:09.563776+0000","last_fullsized":"2024-04-22T13:27:09.563776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562794+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:27:09.562794+0000","last_peered":"2024-04-22T13:27:09.562794+0000","last_clean":"2024-04-22T13:27:09.562794+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:27:09.562794+0000","last_undegraded":"2024-04-22T13:27:09.562794+0000","last_fullsized":"2024-04-22T13:27:09.562794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562713+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:27:09.562713+0000","last_peered":"2024-04-22T13:27:09.562713+0000","last_clean":"2024-04-22T13:27:09.562713+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:27:09.562713+0000","last_undegraded":"2024-04-22T13:27:09.562713+0000","last_fullsized":"2024-04-22T13:27:09.562713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563250+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.563250+0000","last_peered":"2024-04-22T13:27:09.563250+0000","last_clean":"2024-04-22T13:27:09.563250+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:27:09.563250+0000","last_undegraded":"2024-04-22T13:27:09.563250+0000","last_fullsized":"2024-04-22T13:27:09.563250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446872+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:27:09.446872+0000","last_peered":"2024-04-22T13:27:09.446872+0000","last_clean":"2024-04-22T13:27:09.446872+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:27:09.446872+0000","last_undegraded":"2024-04-22T13:27:09.446872+0000","last_fullsized":"2024-04-22T13:27:09.446872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563565+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:27:09.563565+0000","last_peered":"2024-04-22T13:27:09.563565+0000","last_clean":"2024-04-22T13:27:09.563565+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:27:09.563565+0000","last_undegraded":"2024-04-22T13:27:09.563565+0000","last_fullsized":"2024-04-22T13:27:09.563565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.695974+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:27:09.695974+0000","last_peered":"2024-04-22T13:27:09.695974+0000","last_clean":"2024-04-22T13:27:09.695974+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:27:09.695974+0000","last_undegraded":"2024-04-22T13:27:09.695974+0000","last_fullsized":"2024-04-22T13:27:09.695974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541055+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:27:09.541055+0000","last_peered":"2024-04-22T13:27:09.541055+0000","last_clean":"2024-04-22T13:27:09.541055+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:27:09.541055+0000","last_undegraded":"2024-04-22T13:27:09.541055+0000","last_fullsized":"2024-04-22T13:27:09.541055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541272+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:27:09.541272+0000","last_peered":"2024-04-22T13:27:09.541272+0000","last_clean":"2024-04-22T13:27:09.541272+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:27:09.541272+0000","last_undegraded":"2024-04-22T13:27:09.541272+0000","last_fullsized":"2024-04-22T13:27:09.541272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564793+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:27:09.564793+0000","last_peered":"2024-04-22T13:27:09.564793+0000","last_clean":"2024-04-22T13:27:09.564793+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:27:09.564793+0000","last_undegraded":"2024-04-22T13:27:09.564793+0000","last_fullsized":"2024-04-22T13:27:09.564793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562983+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:27:09.562983+0000","last_peered":"2024-04-22T13:27:09.562983+0000","last_clean":"2024-04-22T13:27:09.562983+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:27:09.562983+0000","last_undegraded":"2024-04-22T13:27:09.562983+0000","last_fullsized":"2024-04-22T13:27:09.562983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562953+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:27:09.562953+0000","last_peered":"2024-04-22T13:27:09.562953+0000","last_clean":"2024-04-22T13:27:09.562953+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:27:09.562953+0000","last_undegraded":"2024-04-22T13:27:09.562953+0000","last_fullsized":"2024-04-22T13:27:09.562953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.451042+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:27:09.451042+0000","last_peered":"2024-04-22T13:27:09.451042+0000","last_clean":"2024-04-22T13:27:09.451042+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:27:09.451042+0000","last_undegraded":"2024-04-22T13:27:09.451042+0000","last_fullsized":"2024-04-22T13:27:09.451042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547806+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:27:09.547806+0000","last_peered":"2024-04-22T13:27:09.547806+0000","last_clean":"2024-04-22T13:27:09.547806+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:27:09.547806+0000","last_undegraded":"2024-04-22T13:27:09.547806+0000","last_fullsized":"2024-04-22T13:27:09.547806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547716+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:27:09.547716+0000","last_peered":"2024-04-22T13:27:09.547716+0000","last_clean":"2024-04-22T13:27:09.547716+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:27:09.547716+0000","last_undegraded":"2024-04-22T13:27:09.547716+0000","last_fullsized":"2024-04-22T13:27:09.547716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450920+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:27:09.450920+0000","last_peered":"2024-04-22T13:27:09.450920+0000","last_clean":"2024-04-22T13:27:09.450920+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:27:09.450920+0000","last_undegraded":"2024-04-22T13:27:09.450920+0000","last_fullsized":"2024-04-22T13:27:09.450920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448243+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:27:09.448243+0000","last_peered":"2024-04-22T13:27:09.448243+0000","last_clean":"2024-04-22T13:27:09.448243+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:27:09.448243+0000","last_undegraded":"2024-04-22T13:27:09.448243+0000","last_fullsized":"2024-04-22T13:27:09.448243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547784+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:27:09.547784+0000","last_peered":"2024-04-22T13:27:09.547784+0000","last_clean":"2024-04-22T13:27:09.547784+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:27:09.547784+0000","last_undegraded":"2024-04-22T13:27:09.547784+0000","last_fullsized":"2024-04-22T13:27:09.547784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562955+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:27:09.562955+0000","last_peered":"2024-04-22T13:27:09.562955+0000","last_clean":"2024-04-22T13:27:09.562955+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:27:09.562955+0000","last_undegraded":"2024-04-22T13:27:09.562955+0000","last_fullsized":"2024-04-22T13:27:09.562955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540293+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:27:09.540293+0000","last_peered":"2024-04-22T13:27:09.540293+0000","last_clean":"2024-04-22T13:27:09.540293+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:27:09.540293+0000","last_undegraded":"2024-04-22T13:27:09.540293+0000","last_fullsized":"2024-04-22T13:27:09.540293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547473+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:27:09.547473+0000","last_peered":"2024-04-22T13:27:09.547473+0000","last_clean":"2024-04-22T13:27:09.547473+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:27:09.547473+0000","last_undegraded":"2024-04-22T13:27:09.547473+0000","last_fullsized":"2024-04-22T13:27:09.547473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446526+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:27:09.446526+0000","last_peered":"2024-04-22T13:27:09.446526+0000","last_clean":"2024-04-22T13:27:09.446526+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:27:09.446526+0000","last_undegraded":"2024-04-22T13:27:09.446526+0000","last_fullsized":"2024-04-22T13:27:09.446526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":255,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399316+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:27:10.399316+0000","last_peered":"2024-04-22T13:27:10.399316+0000","last_clean":"2024-04-22T13:27:10.399316+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:27:10.399316+0000","last_undegraded":"2024-04-22T13:27:10.399316+0000","last_fullsized":"2024-04-22T13:27:10.399316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563427+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:27:09.563427+0000","last_peered":"2024-04-22T13:27:09.563427+0000","last_clean":"2024-04-22T13:27:09.563427+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:27:09.563427+0000","last_undegraded":"2024-04-22T13:27:09.563427+0000","last_fullsized":"2024-04-22T13:27:09.563427+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564441+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:27:09.564441+0000","last_peered":"2024-04-22T13:27:09.564441+0000","last_clean":"2024-04-22T13:27:09.564441+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:27:09.564441+0000","last_undegraded":"2024-04-22T13:27:09.564441+0000","last_fullsized":"2024-04-22T13:27:09.564441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564869+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:27:09.564869+0000","last_peered":"2024-04-22T13:27:09.564869+0000","last_clean":"2024-04-22T13:27:09.564869+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:27:09.564869+0000","last_undegraded":"2024-04-22T13:27:09.564869+0000","last_fullsized":"2024-04-22T13:27:09.564869+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547887+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:27:09.547887+0000","last_peered":"2024-04-22T13:27:09.547887+0000","last_clean":"2024-04-22T13:27:09.547887+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:27:09.547887+0000","last_undegraded":"2024-04-22T13:27:09.547887+0000","last_fullsized":"2024-04-22T13:27:09.547887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562984+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:27:09.562984+0000","last_peered":"2024-04-22T13:27:09.562984+0000","last_clean":"2024-04-22T13:27:09.562984+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:27:09.562984+0000","last_undegraded":"2024-04-22T13:27:09.562984+0000","last_fullsized":"2024-04-22T13:27:09.562984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":293,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.404428+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:27:10.404428+0000","last_peered":"2024-04-22T13:27:10.404428+0000","last_clean":"2024-04-22T13:27:10.404428+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:27:10.404428+0000","last_undegraded":"2024-04-22T13:27:10.404428+0000","last_fullsized":"2024-04-22T13:27:10.404428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":237,"num_read_kb":182,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450855+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:27:09.450855+0000","last_peered":"2024-04-22T13:27:09.450855+0000","last_clean":"2024-04-22T13:27:09.450855+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:27:09.450855+0000","last_undegraded":"2024-04-22T13:27:09.450855+0000","last_fullsized":"2024-04-22T13:27:09.450855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563392+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:27:09.563392+0000","last_peered":"2024-04-22T13:27:09.563392+0000","last_clean":"2024-04-22T13:27:09.563392+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:27:09.563392+0000","last_undegraded":"2024-04-22T13:27:09.563392+0000","last_fullsized":"2024-04-22T13:27:09.563392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562910+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:27:09.562910+0000","last_peered":"2024-04-22T13:27:09.562910+0000","last_clean":"2024-04-22T13:27:09.562910+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:27:09.562910+0000","last_undegraded":"2024-04-22T13:27:09.562910+0000","last_fullsized":"2024-04-22T13:27:09.562910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547599+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:27:09.547599+0000","last_peered":"2024-04-22T13:27:09.547599+0000","last_clean":"2024-04-22T13:27:09.547599+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:27:09.547599+0000","last_undegraded":"2024-04-22T13:27:09.547599+0000","last_fullsized":"2024-04-22T13:27:09.547599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564267+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:27:09.564267+0000","last_peered":"2024-04-22T13:27:09.564267+0000","last_clean":"2024-04-22T13:27:09.564267+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:27:09.564267+0000","last_undegraded":"2024-04-22T13:27:09.564267+0000","last_fullsized":"2024-04-22T13:27:09.564267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563351+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:27:09.563351+0000","last_peered":"2024-04-22T13:27:09.563351+0000","last_clean":"2024-04-22T13:27:09.563351+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:27:09.563351+0000","last_undegraded":"2024-04-22T13:27:09.563351+0000","last_fullsized":"2024-04-22T13:27:09.563351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563985+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:27:09.563985+0000","last_peered":"2024-04-22T13:27:09.563985+0000","last_clean":"2024-04-22T13:27:09.563985+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:27:09.563985+0000","last_undegraded":"2024-04-22T13:27:09.563985+0000","last_fullsized":"2024-04-22T13:27:09.563985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450657+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:27:09.450657+0000","last_peered":"2024-04-22T13:27:09.450657+0000","last_clean":"2024-04-22T13:27:09.450657+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:27:09.450657+0000","last_undegraded":"2024-04-22T13:27:09.450657+0000","last_fullsized":"2024-04-22T13:27:09.450657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547522+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:27:09.547522+0000","last_peered":"2024-04-22T13:27:09.547522+0000","last_clean":"2024-04-22T13:27:09.547522+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:27:09.547522+0000","last_undegraded":"2024-04-22T13:27:09.547522+0000","last_fullsized":"2024-04-22T13:27:09.547522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450885+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:27:09.450885+0000","last_peered":"2024-04-22T13:27:09.450885+0000","last_clean":"2024-04-22T13:27:09.450885+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:27:09.450885+0000","last_undegraded":"2024-04-22T13:27:09.450885+0000","last_fullsized":"2024-04-22T13:27:09.450885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563035+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:27:09.563035+0000","last_peered":"2024-04-22T13:27:09.563035+0000","last_clean":"2024-04-22T13:27:09.563035+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:27:09.563035+0000","last_undegraded":"2024-04-22T13:27:09.563035+0000","last_fullsized":"2024-04-22T13:27:09.563035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446714+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:27:09.446714+0000","last_peered":"2024-04-22T13:27:09.446714+0000","last_clean":"2024-04-22T13:27:09.446714+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:27:09.446714+0000","last_undegraded":"2024-04-22T13:27:09.446714+0000","last_fullsized":"2024-04-22T13:27:09.446714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448687+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:27:09.448687+0000","last_peered":"2024-04-22T13:27:09.448687+0000","last_clean":"2024-04-22T13:27:09.448687+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:27:09.448687+0000","last_undegraded":"2024-04-22T13:27:09.448687+0000","last_fullsized":"2024-04-22T13:27:09.448687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399166+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:27:10.399166+0000","last_peered":"2024-04-22T13:27:10.399166+0000","last_clean":"2024-04-22T13:27:10.399166+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:27:10.399166+0000","last_undegraded":"2024-04-22T13:27:10.399166+0000","last_fullsized":"2024-04-22T13:27:10.399166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448498+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:27:09.448498+0000","last_peered":"2024-04-22T13:27:09.448498+0000","last_clean":"2024-04-22T13:27:09.448498+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:27:09.448498+0000","last_undegraded":"2024-04-22T13:27:09.448498+0000","last_fullsized":"2024-04-22T13:27:09.448498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446492+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:27:09.446492+0000","last_peered":"2024-04-22T13:27:09.446492+0000","last_clean":"2024-04-22T13:27:09.446492+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:27:09.446492+0000","last_undegraded":"2024-04-22T13:27:09.446492+0000","last_fullsized":"2024-04-22T13:27:09.446492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":141,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563174+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:27:09.563174+0000","last_peered":"2024-04-22T13:27:09.563174+0000","last_clean":"2024-04-22T13:27:09.563174+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:27:09.563174+0000","last_undegraded":"2024-04-22T13:27:09.563174+0000","last_fullsized":"2024-04-22T13:27:09.563174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540568+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:27:09.540568+0000","last_peered":"2024-04-22T13:27:09.540568+0000","last_clean":"2024-04-22T13:27:09.540568+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:27:09.540568+0000","last_undegraded":"2024-04-22T13:27:09.540568+0000","last_fullsized":"2024-04-22T13:27:09.540568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563433+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:27:09.563433+0000","last_peered":"2024-04-22T13:27:09.563433+0000","last_clean":"2024-04-22T13:27:09.563433+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:27:09.563433+0000","last_undegraded":"2024-04-22T13:27:09.563433+0000","last_fullsized":"2024-04-22T13:27:09.563433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564057+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:27:09.564057+0000","last_peered":"2024-04-22T13:27:09.564057+0000","last_clean":"2024-04-22T13:27:09.564057+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:27:09.564057+0000","last_undegraded":"2024-04-22T13:27:09.564057+0000","last_fullsized":"2024-04-22T13:27:09.564057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562773+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:27:09.562773+0000","last_peered":"2024-04-22T13:27:09.562773+0000","last_clean":"2024-04-22T13:27:09.562773+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:27:09.562773+0000","last_undegraded":"2024-04-22T13:27:09.562773+0000","last_fullsized":"2024-04-22T13:27:09.562773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564209+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:27:09.564209+0000","last_peered":"2024-04-22T13:27:09.564209+0000","last_clean":"2024-04-22T13:27:09.564209+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:27:09.564209+0000","last_undegraded":"2024-04-22T13:27:09.564209+0000","last_fullsized":"2024-04-22T13:27:09.564209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563221+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:27:09.563221+0000","last_peered":"2024-04-22T13:27:09.563221+0000","last_clean":"2024-04-22T13:27:09.563221+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:27:09.563221+0000","last_undegraded":"2024-04-22T13:27:09.563221+0000","last_fullsized":"2024-04-22T13:27:09.563221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.657844+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:27:09.657844+0000","last_peered":"2024-04-22T13:27:09.657844+0000","last_clean":"2024-04-22T13:27:09.657844+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:27:09.657844+0000","last_undegraded":"2024-04-22T13:27:09.657844+0000","last_fullsized":"2024-04-22T13:27:09.657844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541266+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:27:09.541266+0000","last_peered":"2024-04-22T13:27:09.541266+0000","last_clean":"2024-04-22T13:27:09.541266+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:27:09.541266+0000","last_undegraded":"2024-04-22T13:27:09.541266+0000","last_fullsized":"2024-04-22T13:27:09.541266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563070+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:27:09.563070+0000","last_peered":"2024-04-22T13:27:09.563070+0000","last_clean":"2024-04-22T13:27:09.563070+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:27:09.563070+0000","last_undegraded":"2024-04-22T13:27:09.563070+0000","last_fullsized":"2024-04-22T13:27:09.563070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564799+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:27:09.564799+0000","last_peered":"2024-04-22T13:27:09.564799+0000","last_clean":"2024-04-22T13:27:09.564799+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:27:09.564799+0000","last_undegraded":"2024-04-22T13:27:09.564799+0000","last_fullsized":"2024-04-22T13:27:09.564799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562808+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:27:09.562808+0000","last_peered":"2024-04-22T13:27:09.562808+0000","last_clean":"2024-04-22T13:27:09.562808+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:27:09.562808+0000","last_undegraded":"2024-04-22T13:27:09.562808+0000","last_fullsized":"2024-04-22T13:27:09.562808+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450691+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:27:09.450691+0000","last_peered":"2024-04-22T13:27:09.450691+0000","last_clean":"2024-04-22T13:27:09.450691+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:27:09.450691+0000","last_undegraded":"2024-04-22T13:27:09.450691+0000","last_fullsized":"2024-04-22T13:27:09.450691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540308+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:27:09.540308+0000","last_peered":"2024-04-22T13:27:09.540308+0000","last_clean":"2024-04-22T13:27:09.540308+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:27:09.540308+0000","last_undegraded":"2024-04-22T13:27:09.540308+0000","last_fullsized":"2024-04-22T13:27:09.540308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562679+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:27:09.562679+0000","last_peered":"2024-04-22T13:27:09.562679+0000","last_clean":"2024-04-22T13:27:09.562679+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:27:09.562679+0000","last_undegraded":"2024-04-22T13:27:09.562679+0000","last_fullsized":"2024-04-22T13:27:09.562679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562789+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:27:09.562789+0000","last_peered":"2024-04-22T13:27:09.562789+0000","last_clean":"2024-04-22T13:27:09.562789+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:27:09.562789+0000","last_undegraded":"2024-04-22T13:27:09.562789+0000","last_fullsized":"2024-04-22T13:27:09.562789+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540822+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:27:09.540822+0000","last_peered":"2024-04-22T13:27:09.540822+0000","last_clean":"2024-04-22T13:27:09.540822+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:27:09.540822+0000","last_undegraded":"2024-04-22T13:27:09.540822+0000","last_fullsized":"2024-04-22T13:27:09.540822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450783+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:27:09.450783+0000","last_peered":"2024-04-22T13:27:09.450783+0000","last_clean":"2024-04-22T13:27:09.450783+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:27:09.450783+0000","last_undegraded":"2024-04-22T13:27:09.450783+0000","last_fullsized":"2024-04-22T13:27:09.450783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564270+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:27:09.564270+0000","last_peered":"2024-04-22T13:27:09.564270+0000","last_clean":"2024-04-22T13:27:09.564270+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:27:09.564270+0000","last_undegraded":"2024-04-22T13:27:09.564270+0000","last_fullsized":"2024-04-22T13:27:09.564270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":146,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399643+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:27:10.399643+0000","last_peered":"2024-04-22T13:27:10.399643+0000","last_clean":"2024-04-22T13:27:10.399643+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:27:10.399643+0000","last_undegraded":"2024-04-22T13:27:10.399643+0000","last_fullsized":"2024-04-22T13:27:10.399643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564284+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.564284+0000","last_peered":"2024-04-22T13:27:09.564284+0000","last_clean":"2024-04-22T13:27:09.564284+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:27:09.564284+0000","last_undegraded":"2024-04-22T13:27:09.564284+0000","last_fullsized":"2024-04-22T13:27:09.564284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.651858+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:27:09.651858+0000","last_peered":"2024-04-22T13:27:09.651858+0000","last_clean":"2024-04-22T13:27:09.651858+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:27:09.651858+0000","last_undegraded":"2024-04-22T13:27:09.651858+0000","last_fullsized":"2024-04-22T13:27:09.651858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564302+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:27:09.564302+0000","last_peered":"2024-04-22T13:27:09.564302+0000","last_clean":"2024-04-22T13:27:09.564302+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:27:09.564302+0000","last_undegraded":"2024-04-22T13:27:09.564302+0000","last_fullsized":"2024-04-22T13:27:09.564302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563457+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:27:09.563457+0000","last_peered":"2024-04-22T13:27:09.563457+0000","last_clean":"2024-04-22T13:27:09.563457+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:27:09.563457+0000","last_undegraded":"2024-04-22T13:27:09.563457+0000","last_fullsized":"2024-04-22T13:27:09.563457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564533+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:27:09.564533+0000","last_peered":"2024-04-22T13:27:09.564533+0000","last_clean":"2024-04-22T13:27:09.564533+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:27:09.564533+0000","last_undegraded":"2024-04-22T13:27:09.564533+0000","last_fullsized":"2024-04-22T13:27:09.564533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562753+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:27:09.562753+0000","last_peered":"2024-04-22T13:27:09.562753+0000","last_clean":"2024-04-22T13:27:09.562753+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:27:09.562753+0000","last_undegraded":"2024-04-22T13:27:09.562753+0000","last_fullsized":"2024-04-22T13:27:09.562753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563291+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:27:09.563291+0000","last_peered":"2024-04-22T13:27:09.563291+0000","last_clean":"2024-04-22T13:27:09.563291+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:27:09.563291+0000","last_undegraded":"2024-04-22T13:27:09.563291+0000","last_fullsized":"2024-04-22T13:27:09.563291+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563267+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:27:09.563267+0000","last_peered":"2024-04-22T13:27:09.563267+0000","last_clean":"2024-04-22T13:27:09.563267+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:27:09.563267+0000","last_undegraded":"2024-04-22T13:27:09.563267+0000","last_fullsized":"2024-04-22T13:27:09.563267+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564293+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:27:09.564293+0000","last_peered":"2024-04-22T13:27:09.564293+0000","last_clean":"2024-04-22T13:27:09.564293+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:27:09.564293+0000","last_undegraded":"2024-04-22T13:27:09.564293+0000","last_fullsized":"2024-04-22T13:27:09.564293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563391+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:27:09.563391+0000","last_peered":"2024-04-22T13:27:09.563391+0000","last_clean":"2024-04-22T13:27:09.563391+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:27:09.563391+0000","last_undegraded":"2024-04-22T13:27:09.563391+0000","last_fullsized":"2024-04-22T13:27:09.563391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563022+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:27:09.563022+0000","last_peered":"2024-04-22T13:27:09.563022+0000","last_clean":"2024-04-22T13:27:09.563022+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:27:09.563022+0000","last_undegraded":"2024-04-22T13:27:09.563022+0000","last_fullsized":"2024-04-22T13:27:09.563022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563649+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:27:09.563649+0000","last_peered":"2024-04-22T13:27:09.563649+0000","last_clean":"2024-04-22T13:27:09.563649+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:27:09.563649+0000","last_undegraded":"2024-04-22T13:27:09.563649+0000","last_fullsized":"2024-04-22T13:27:09.563649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541195+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:27:09.541195+0000","last_peered":"2024-04-22T13:27:09.541195+0000","last_clean":"2024-04-22T13:27:09.541195+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:27:09.541195+0000","last_undegraded":"2024-04-22T13:27:09.541195+0000","last_fullsized":"2024-04-22T13:27:09.541195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562758+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:27:09.562758+0000","last_peered":"2024-04-22T13:27:09.562758+0000","last_clean":"2024-04-22T13:27:09.562758+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:27:09.562758+0000","last_undegraded":"2024-04-22T13:27:09.562758+0000","last_fullsized":"2024-04-22T13:27:09.562758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540273+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:27:09.540273+0000","last_peered":"2024-04-22T13:27:09.540273+0000","last_clean":"2024-04-22T13:27:09.540273+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:27:09.540273+0000","last_undegraded":"2024-04-22T13:27:09.540273+0000","last_fullsized":"2024-04-22T13:27:09.540273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562941+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:27:09.562941+0000","last_peered":"2024-04-22T13:27:09.562941+0000","last_clean":"2024-04-22T13:27:09.562941+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:27:09.562941+0000","last_undegraded":"2024-04-22T13:27:09.562941+0000","last_fullsized":"2024-04-22T13:27:09.562941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563605+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:27:09.563605+0000","last_peered":"2024-04-22T13:27:09.563605+0000","last_clean":"2024-04-22T13:27:09.563605+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:27:09.563605+0000","last_undegraded":"2024-04-22T13:27:09.563605+0000","last_fullsized":"2024-04-22T13:27:09.563605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562834+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:27:09.562834+0000","last_peered":"2024-04-22T13:27:09.562834+0000","last_clean":"2024-04-22T13:27:09.562834+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:27:09.562834+0000","last_undegraded":"2024-04-22T13:27:09.562834+0000","last_fullsized":"2024-04-22T13:27:09.562834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597354,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1275,"num_read_kb":751,"num_write":366,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252899,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":485126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514536,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1134854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1036288,"data_stored":484357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939700,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1425408,"data_stored":877951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41092,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704060,"statfs":{"total":95995035648,"available":95952957440,"internally_reserved":0,"allocated":1630208,"data_stored":1077785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":549872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":486112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":550932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40492,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704660,"statfs":{"total":95995035648,"available":95953571840,"internally_reserved":0,"allocated":1019904,"data_stored":474972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37252,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707900,"statfs":{"total":95995035648,"available":95956889600,"internally_reserved":0,"allocated":1777664,"data_stored":1208672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":197830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:27:13.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 -- 172.21.15.5:0/3685110456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d8108060 msgr2=0x7fc9d8108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/3685110456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d8108060 0x7fc9d8108440 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009930 tx=0x7fc9c802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 -- 172.21.15.5:0/3685110456 shutdown_connections 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/3685110456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806b000 0x7fc9d810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/3685110456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d806a640 0x7fc9d806aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/3685110456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d8108060 0x7fc9d8108440 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 -- 172.21.15.5:0/3685110456 >> 172.21.15.5:0/3685110456 conn(0x7fc9d8077c40 msgr2=0x7fc9d8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 -- 172.21.15.5:0/3685110456 shutdown_connections 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.092+0000 7fc9dceb3640 1 -- 172.21.15.5:0/3685110456 wait complete. 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 Processor -- start 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 -- start start 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806a640 0x7fc9d8101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 0x7fc9d81016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d8108060 0x7fc9d8101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d81121f0 con 0x7fc9d806b000 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112070 con 0x7fc9d806a640 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9dceb3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112370 con 0x7fc9d8108060 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.093+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 0x7fc9d81016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 0x7fc9d81016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50906/0 (socket says 172.21.15.5:50906) 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9cffff640 1 -- 172.21.15.5:0/11200751 learned_addr learned my addr 172.21.15.5:0/11200751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806a640 0x7fc9d8101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d6d9d640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d8108060 0x7fc9d8101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 -- 172.21.15.5:0/11200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d8108060 msgr2=0x7fc9d8101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d8108060 0x7fc9d8101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 -- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 msgr2=0x7fc9d81016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 0x7fc9d81016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 -- 172.21.15.5:0/11200751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d81a6b90 con 0x7fc9d806a640 2024-04-22T13:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9cffff640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 0x7fc9d81016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d6d9d640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d8108060 0x7fc9d8101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9d659c640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806a640 0x7fc9d8101170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8036040 tx=0x7fc9c802c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8043020 con 0x7fc9d806a640 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c802ca70 con 0x7fc9d806a640 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d81a6e20 con 0x7fc9d806a640 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8003a10 con 0x7fc9d806a640 2024-04-22T13:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.094+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d81a72e0 con 0x7fc9d806a640 2024-04-22T13:27:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.095+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d8103650 con 0x7fc9d806a640 2024-04-22T13:27:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.096+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fc9c8046020 con 0x7fc9d806a640 2024-04-22T13:27:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.097+0000 7fc9cf7fe640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc9c40a3e40 0x7fc9c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.097+0000 7fc9cffff640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc9c40a3e40 0x7fc9c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.097+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc9c80f2920 con 0x7fc9d806a640 2024-04-22T13:27:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.097+0000 7fc9cffff640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc9c40a3e40 0x7fc9c40a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8102c00 tx=0x7fc9c0005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:14.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.102+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9c804d020 con 0x7fc9d806a640 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Unable to set osd_memory_target on smithi005 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3471849925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2029409103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:14.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2032131645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:14.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.286+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9d8062390 con 0x7fc9c40a3e40 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Unable to set osd_memory_target on smithi005 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3471849925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2029409103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:14.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2032131645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:14.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.293+0000 7fc9cf7fe640 1 -- 172.21.15.5:0/11200751 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257789 (secure 0 0 0) 0x7fc9d8062390 con 0x7fc9c40a3e40 2024-04-22T13:27:14.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc9c40a3e40 msgr2=0x7fc9c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc9c40a3e40 0x7fc9c40a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8102c00 tx=0x7fc9c0005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806a640 msgr2=0x7fc9d8101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806a640 0x7fc9d8101170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8036040 tx=0x7fc9c802c640 comp rx=0 tx=0).stop 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 shutdown_connections 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9d806b000 0x7fc9d81016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc9c40a3e40 0x7fc9c40a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d8108060 0x7fc9d8101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 --2- 172.21.15.5:0/11200751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9d806a640 0x7fc9d8101170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 >> 172.21.15.5:0/11200751 conn(0x7fc9d8077c40 msgr2=0x7fc9d80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 shutdown_connections 2024-04-22T13:27:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.297+0000 7fc9dceb3640 1 -- 172.21.15.5:0/11200751 wait complete. 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Unable to set osd_memory_target on smithi005 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:14.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:14.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-22T13:27:14.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Updating smithi049:/etc/ceph/ceph.conf 2024-04-22T13:27:14.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T13:27:14.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3471849925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:14.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2029409103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:14.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2032131645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:14.305 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T13:27:13.460500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187956,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1411,"num_read_kb":870,"num_write":549,"num_write_kb":4397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":933,"ondisk_log_size":933,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":413824,"kb_used_data":14536,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528000,"statfs":{"total":1151940427776,"available":1151516672000,"internally_reserved":0,"allocated":14884864,"data_stored":8299033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988589"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563085+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:27:09.563085+0000","last_peered":"2024-04-22T13:27:09.563085+0000","last_clean":"2024-04-22T13:27:09.563085+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:27:09.563085+0000","last_undegraded":"2024-04-22T13:27:09.563085+0000","last_fullsized":"2024-04-22T13:27:09.563085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564651+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:27:09.564651+0000","last_peered":"2024-04-22T13:27:09.564651+0000","last_clean":"2024-04-22T13:27:09.564651+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:27:09.564651+0000","last_undegraded":"2024-04-22T13:27:09.564651+0000","last_fullsized":"2024-04-22T13:27:09.564651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564621+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:27:09.564621+0000","last_peered":"2024-04-22T13:27:09.564621+0000","last_clean":"2024-04-22T13:27:09.564621+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:27:09.564621+0000","last_undegraded":"2024-04-22T13:27:09.564621+0000","last_fullsized":"2024-04-22T13:27:09.564621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563725+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:27:09.563725+0000","last_peered":"2024-04-22T13:27:09.563725+0000","last_clean":"2024-04-22T13:27:09.563725+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:27:09.563725+0000","last_undegraded":"2024-04-22T13:27:09.563725+0000","last_fullsized":"2024-04-22T13:27:09.563725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448296+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:27:09.448296+0000","last_peered":"2024-04-22T13:27:09.448296+0000","last_clean":"2024-04-22T13:27:09.448296+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:27:09.448296+0000","last_undegraded":"2024-04-22T13:27:09.448296+0000","last_fullsized":"2024-04-22T13:27:09.448296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564063+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:27:09.564063+0000","last_peered":"2024-04-22T13:27:09.564063+0000","last_clean":"2024-04-22T13:27:09.564063+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:27:09.564063+0000","last_undegraded":"2024-04-22T13:27:09.564063+0000","last_fullsized":"2024-04-22T13:27:09.564063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540227+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:27:09.540227+0000","last_peered":"2024-04-22T13:27:09.540227+0000","last_clean":"2024-04-22T13:27:09.540227+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:27:09.540227+0000","last_undegraded":"2024-04-22T13:27:09.540227+0000","last_fullsized":"2024-04-22T13:27:09.540227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563416+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:27:09.563416+0000","last_peered":"2024-04-22T13:27:09.563416+0000","last_clean":"2024-04-22T13:27:09.563416+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:27:09.563416+0000","last_undegraded":"2024-04-22T13:27:09.563416+0000","last_fullsized":"2024-04-22T13:27:09.563416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448852+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:27:09.448852+0000","last_peered":"2024-04-22T13:27:09.448852+0000","last_clean":"2024-04-22T13:27:09.448852+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:27:09.448852+0000","last_undegraded":"2024-04-22T13:27:09.448852+0000","last_fullsized":"2024-04-22T13:27:09.448852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564466+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:27:09.564466+0000","last_peered":"2024-04-22T13:27:09.564466+0000","last_clean":"2024-04-22T13:27:09.564466+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:27:09.564466+0000","last_undegraded":"2024-04-22T13:27:09.564466+0000","last_fullsized":"2024-04-22T13:27:09.564466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563562+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:27:09.563562+0000","last_peered":"2024-04-22T13:27:09.563562+0000","last_clean":"2024-04-22T13:27:09.563562+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:27:09.563562+0000","last_undegraded":"2024-04-22T13:27:09.563562+0000","last_fullsized":"2024-04-22T13:27:09.563562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563593+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:27:09.563593+0000","last_peered":"2024-04-22T13:27:09.563593+0000","last_clean":"2024-04-22T13:27:09.563593+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:27:09.563593+0000","last_undegraded":"2024-04-22T13:27:09.563593+0000","last_fullsized":"2024-04-22T13:27:09.563593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563776+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:27:09.563776+0000","last_peered":"2024-04-22T13:27:09.563776+0000","last_clean":"2024-04-22T13:27:09.563776+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:27:09.563776+0000","last_undegraded":"2024-04-22T13:27:09.563776+0000","last_fullsized":"2024-04-22T13:27:09.563776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562794+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:27:09.562794+0000","last_peered":"2024-04-22T13:27:09.562794+0000","last_clean":"2024-04-22T13:27:09.562794+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:27:09.562794+0000","last_undegraded":"2024-04-22T13:27:09.562794+0000","last_fullsized":"2024-04-22T13:27:09.562794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562713+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:27:09.562713+0000","last_peered":"2024-04-22T13:27:09.562713+0000","last_clean":"2024-04-22T13:27:09.562713+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:27:09.562713+0000","last_undegraded":"2024-04-22T13:27:09.562713+0000","last_fullsized":"2024-04-22T13:27:09.562713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563250+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.563250+0000","last_peered":"2024-04-22T13:27:09.563250+0000","last_clean":"2024-04-22T13:27:09.563250+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:27:09.563250+0000","last_undegraded":"2024-04-22T13:27:09.563250+0000","last_fullsized":"2024-04-22T13:27:09.563250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446872+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:27:09.446872+0000","last_peered":"2024-04-22T13:27:09.446872+0000","last_clean":"2024-04-22T13:27:09.446872+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:27:09.446872+0000","last_undegraded":"2024-04-22T13:27:09.446872+0000","last_fullsized":"2024-04-22T13:27:09.446872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563565+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:27:09.563565+0000","last_peered":"2024-04-22T13:27:09.563565+0000","last_clean":"2024-04-22T13:27:09.563565+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:27:09.563565+0000","last_undegraded":"2024-04-22T13:27:09.563565+0000","last_fullsized":"2024-04-22T13:27:09.563565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.695974+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:27:09.695974+0000","last_peered":"2024-04-22T13:27:09.695974+0000","last_clean":"2024-04-22T13:27:09.695974+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:27:09.695974+0000","last_undegraded":"2024-04-22T13:27:09.695974+0000","last_fullsized":"2024-04-22T13:27:09.695974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541055+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:27:09.541055+0000","last_peered":"2024-04-22T13:27:09.541055+0000","last_clean":"2024-04-22T13:27:09.541055+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:27:09.541055+0000","last_undegraded":"2024-04-22T13:27:09.541055+0000","last_fullsized":"2024-04-22T13:27:09.541055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541272+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:27:09.541272+0000","last_peered":"2024-04-22T13:27:09.541272+0000","last_clean":"2024-04-22T13:27:09.541272+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:27:09.541272+0000","last_undegraded":"2024-04-22T13:27:09.541272+0000","last_fullsized":"2024-04-22T13:27:09.541272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564793+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:27:09.564793+0000","last_peered":"2024-04-22T13:27:09.564793+0000","last_clean":"2024-04-22T13:27:09.564793+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:27:09.564793+0000","last_undegraded":"2024-04-22T13:27:09.564793+0000","last_fullsized":"2024-04-22T13:27:09.564793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562983+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:27:09.562983+0000","last_peered":"2024-04-22T13:27:09.562983+0000","last_clean":"2024-04-22T13:27:09.562983+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:27:09.562983+0000","last_undegraded":"2024-04-22T13:27:09.562983+0000","last_fullsized":"2024-04-22T13:27:09.562983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562953+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:27:09.562953+0000","last_peered":"2024-04-22T13:27:09.562953+0000","last_clean":"2024-04-22T13:27:09.562953+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:27:09.562953+0000","last_undegraded":"2024-04-22T13:27:09.562953+0000","last_fullsized":"2024-04-22T13:27:09.562953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.451042+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:27:09.451042+0000","last_peered":"2024-04-22T13:27:09.451042+0000","last_clean":"2024-04-22T13:27:09.451042+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:27:09.451042+0000","last_undegraded":"2024-04-22T13:27:09.451042+0000","last_fullsized":"2024-04-22T13:27:09.451042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547806+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:27:09.547806+0000","last_peered":"2024-04-22T13:27:09.547806+0000","last_clean":"2024-04-22T13:27:09.547806+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:27:09.547806+0000","last_undegraded":"2024-04-22T13:27:09.547806+0000","last_fullsized":"2024-04-22T13:27:09.547806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547716+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:27:09.547716+0000","last_peered":"2024-04-22T13:27:09.547716+0000","last_clean":"2024-04-22T13:27:09.547716+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:27:09.547716+0000","last_undegraded":"2024-04-22T13:27:09.547716+0000","last_fullsized":"2024-04-22T13:27:09.547716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450920+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:27:09.450920+0000","last_peered":"2024-04-22T13:27:09.450920+0000","last_clean":"2024-04-22T13:27:09.450920+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:27:09.450920+0000","last_undegraded":"2024-04-22T13:27:09.450920+0000","last_fullsized":"2024-04-22T13:27:09.450920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448243+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:27:09.448243+0000","last_peered":"2024-04-22T13:27:09.448243+0000","last_clean":"2024-04-22T13:27:09.448243+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:27:09.448243+0000","last_undegraded":"2024-04-22T13:27:09.448243+0000","last_fullsized":"2024-04-22T13:27:09.448243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547784+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:27:09.547784+0000","last_peered":"2024-04-22T13:27:09.547784+0000","last_clean":"2024-04-22T13:27:09.547784+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:27:09.547784+0000","last_undegraded":"2024-04-22T13:27:09.547784+0000","last_fullsized":"2024-04-22T13:27:09.547784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562955+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:27:09.562955+0000","last_peered":"2024-04-22T13:27:09.562955+0000","last_clean":"2024-04-22T13:27:09.562955+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:27:09.562955+0000","last_undegraded":"2024-04-22T13:27:09.562955+0000","last_fullsized":"2024-04-22T13:27:09.562955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540293+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:27:09.540293+0000","last_peered":"2024-04-22T13:27:09.540293+0000","last_clean":"2024-04-22T13:27:09.540293+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:27:09.540293+0000","last_undegraded":"2024-04-22T13:27:09.540293+0000","last_fullsized":"2024-04-22T13:27:09.540293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547473+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:27:09.547473+0000","last_peered":"2024-04-22T13:27:09.547473+0000","last_clean":"2024-04-22T13:27:09.547473+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:27:09.547473+0000","last_undegraded":"2024-04-22T13:27:09.547473+0000","last_fullsized":"2024-04-22T13:27:09.547473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446526+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:27:09.446526+0000","last_peered":"2024-04-22T13:27:09.446526+0000","last_clean":"2024-04-22T13:27:09.446526+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:27:09.446526+0000","last_undegraded":"2024-04-22T13:27:09.446526+0000","last_fullsized":"2024-04-22T13:27:09.446526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":255,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399316+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:27:10.399316+0000","last_peered":"2024-04-22T13:27:10.399316+0000","last_clean":"2024-04-22T13:27:10.399316+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:27:10.399316+0000","last_undegraded":"2024-04-22T13:27:10.399316+0000","last_fullsized":"2024-04-22T13:27:10.399316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563427+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:27:09.563427+0000","last_peered":"2024-04-22T13:27:09.563427+0000","last_clean":"2024-04-22T13:27:09.563427+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:27:09.563427+0000","last_undegraded":"2024-04-22T13:27:09.563427+0000","last_fullsized":"2024-04-22T13:27:09.563427+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564441+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:27:09.564441+0000","last_peered":"2024-04-22T13:27:09.564441+0000","last_clean":"2024-04-22T13:27:09.564441+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:27:09.564441+0000","last_undegraded":"2024-04-22T13:27:09.564441+0000","last_fullsized":"2024-04-22T13:27:09.564441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564869+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:27:09.564869+0000","last_peered":"2024-04-22T13:27:09.564869+0000","last_clean":"2024-04-22T13:27:09.564869+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:27:09.564869+0000","last_undegraded":"2024-04-22T13:27:09.564869+0000","last_fullsized":"2024-04-22T13:27:09.564869+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547887+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:27:09.547887+0000","last_peered":"2024-04-22T13:27:09.547887+0000","last_clean":"2024-04-22T13:27:09.547887+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:27:09.547887+0000","last_undegraded":"2024-04-22T13:27:09.547887+0000","last_fullsized":"2024-04-22T13:27:09.547887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562984+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:27:09.562984+0000","last_peered":"2024-04-22T13:27:09.562984+0000","last_clean":"2024-04-22T13:27:09.562984+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:27:09.562984+0000","last_undegraded":"2024-04-22T13:27:09.562984+0000","last_fullsized":"2024-04-22T13:27:09.562984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":293,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.404428+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:27:10.404428+0000","last_peered":"2024-04-22T13:27:10.404428+0000","last_clean":"2024-04-22T13:27:10.404428+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:27:10.404428+0000","last_undegraded":"2024-04-22T13:27:10.404428+0000","last_fullsized":"2024-04-22T13:27:10.404428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":237,"num_read_kb":182,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450855+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:27:09.450855+0000","last_peered":"2024-04-22T13:27:09.450855+0000","last_clean":"2024-04-22T13:27:09.450855+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:27:09.450855+0000","last_undegraded":"2024-04-22T13:27:09.450855+0000","last_fullsized":"2024-04-22T13:27:09.450855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563392+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:27:09.563392+0000","last_peered":"2024-04-22T13:27:09.563392+0000","last_clean":"2024-04-22T13:27:09.563392+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:27:09.563392+0000","last_undegraded":"2024-04-22T13:27:09.563392+0000","last_fullsized":"2024-04-22T13:27:09.563392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562910+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:27:09.562910+0000","last_peered":"2024-04-22T13:27:09.562910+0000","last_clean":"2024-04-22T13:27:09.562910+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:27:09.562910+0000","last_undegraded":"2024-04-22T13:27:09.562910+0000","last_fullsized":"2024-04-22T13:27:09.562910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547599+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:27:09.547599+0000","last_peered":"2024-04-22T13:27:09.547599+0000","last_clean":"2024-04-22T13:27:09.547599+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:27:09.547599+0000","last_undegraded":"2024-04-22T13:27:09.547599+0000","last_fullsized":"2024-04-22T13:27:09.547599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564267+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:27:09.564267+0000","last_peered":"2024-04-22T13:27:09.564267+0000","last_clean":"2024-04-22T13:27:09.564267+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:27:09.564267+0000","last_undegraded":"2024-04-22T13:27:09.564267+0000","last_fullsized":"2024-04-22T13:27:09.564267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563351+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:27:09.563351+0000","last_peered":"2024-04-22T13:27:09.563351+0000","last_clean":"2024-04-22T13:27:09.563351+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:27:09.563351+0000","last_undegraded":"2024-04-22T13:27:09.563351+0000","last_fullsized":"2024-04-22T13:27:09.563351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563985+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:27:09.563985+0000","last_peered":"2024-04-22T13:27:09.563985+0000","last_clean":"2024-04-22T13:27:09.563985+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:27:09.563985+0000","last_undegraded":"2024-04-22T13:27:09.563985+0000","last_fullsized":"2024-04-22T13:27:09.563985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450657+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:27:09.450657+0000","last_peered":"2024-04-22T13:27:09.450657+0000","last_clean":"2024-04-22T13:27:09.450657+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:27:09.450657+0000","last_undegraded":"2024-04-22T13:27:09.450657+0000","last_fullsized":"2024-04-22T13:27:09.450657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547522+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:27:09.547522+0000","last_peered":"2024-04-22T13:27:09.547522+0000","last_clean":"2024-04-22T13:27:09.547522+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:27:09.547522+0000","last_undegraded":"2024-04-22T13:27:09.547522+0000","last_fullsized":"2024-04-22T13:27:09.547522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450885+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:27:09.450885+0000","last_peered":"2024-04-22T13:27:09.450885+0000","last_clean":"2024-04-22T13:27:09.450885+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:27:09.450885+0000","last_undegraded":"2024-04-22T13:27:09.450885+0000","last_fullsized":"2024-04-22T13:27:09.450885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563035+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:27:09.563035+0000","last_peered":"2024-04-22T13:27:09.563035+0000","last_clean":"2024-04-22T13:27:09.563035+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:27:09.563035+0000","last_undegraded":"2024-04-22T13:27:09.563035+0000","last_fullsized":"2024-04-22T13:27:09.563035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446714+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:27:09.446714+0000","last_peered":"2024-04-22T13:27:09.446714+0000","last_clean":"2024-04-22T13:27:09.446714+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:27:09.446714+0000","last_undegraded":"2024-04-22T13:27:09.446714+0000","last_fullsized":"2024-04-22T13:27:09.446714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448687+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:27:09.448687+0000","last_peered":"2024-04-22T13:27:09.448687+0000","last_clean":"2024-04-22T13:27:09.448687+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:27:09.448687+0000","last_undegraded":"2024-04-22T13:27:09.448687+0000","last_fullsized":"2024-04-22T13:27:09.448687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399166+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:27:10.399166+0000","last_peered":"2024-04-22T13:27:10.399166+0000","last_clean":"2024-04-22T13:27:10.399166+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:27:10.399166+0000","last_undegraded":"2024-04-22T13:27:10.399166+0000","last_fullsized":"2024-04-22T13:27:10.399166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448498+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:27:09.448498+0000","last_peered":"2024-04-22T13:27:09.448498+0000","last_clean":"2024-04-22T13:27:09.448498+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:27:09.448498+0000","last_undegraded":"2024-04-22T13:27:09.448498+0000","last_fullsized":"2024-04-22T13:27:09.448498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446492+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:27:09.446492+0000","last_peered":"2024-04-22T13:27:09.446492+0000","last_clean":"2024-04-22T13:27:09.446492+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:27:09.446492+0000","last_undegraded":"2024-04-22T13:27:09.446492+0000","last_fullsized":"2024-04-22T13:27:09.446492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":141,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563174+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:27:09.563174+0000","last_peered":"2024-04-22T13:27:09.563174+0000","last_clean":"2024-04-22T13:27:09.563174+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:27:09.563174+0000","last_undegraded":"2024-04-22T13:27:09.563174+0000","last_fullsized":"2024-04-22T13:27:09.563174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540568+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:27:09.540568+0000","last_peered":"2024-04-22T13:27:09.540568+0000","last_clean":"2024-04-22T13:27:09.540568+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:27:09.540568+0000","last_undegraded":"2024-04-22T13:27:09.540568+0000","last_fullsized":"2024-04-22T13:27:09.540568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563433+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:27:09.563433+0000","last_peered":"2024-04-22T13:27:09.563433+0000","last_clean":"2024-04-22T13:27:09.563433+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:27:09.563433+0000","last_undegraded":"2024-04-22T13:27:09.563433+0000","last_fullsized":"2024-04-22T13:27:09.563433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564057+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:27:09.564057+0000","last_peered":"2024-04-22T13:27:09.564057+0000","last_clean":"2024-04-22T13:27:09.564057+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:27:09.564057+0000","last_undegraded":"2024-04-22T13:27:09.564057+0000","last_fullsized":"2024-04-22T13:27:09.564057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562773+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:27:09.562773+0000","last_peered":"2024-04-22T13:27:09.562773+0000","last_clean":"2024-04-22T13:27:09.562773+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:27:09.562773+0000","last_undegraded":"2024-04-22T13:27:09.562773+0000","last_fullsized":"2024-04-22T13:27:09.562773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564209+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:27:09.564209+0000","last_peered":"2024-04-22T13:27:09.564209+0000","last_clean":"2024-04-22T13:27:09.564209+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:27:09.564209+0000","last_undegraded":"2024-04-22T13:27:09.564209+0000","last_fullsized":"2024-04-22T13:27:09.564209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563221+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:27:09.563221+0000","last_peered":"2024-04-22T13:27:09.563221+0000","last_clean":"2024-04-22T13:27:09.563221+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:27:09.563221+0000","last_undegraded":"2024-04-22T13:27:09.563221+0000","last_fullsized":"2024-04-22T13:27:09.563221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.657844+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:27:09.657844+0000","last_peered":"2024-04-22T13:27:09.657844+0000","last_clean":"2024-04-22T13:27:09.657844+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:27:09.657844+0000","last_undegraded":"2024-04-22T13:27:09.657844+0000","last_fullsized":"2024-04-22T13:27:09.657844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541266+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:27:09.541266+0000","last_peered":"2024-04-22T13:27:09.541266+0000","last_clean":"2024-04-22T13:27:09.541266+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:27:09.541266+0000","last_undegraded":"2024-04-22T13:27:09.541266+0000","last_fullsized":"2024-04-22T13:27:09.541266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563070+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:27:09.563070+0000","last_peered":"2024-04-22T13:27:09.563070+0000","last_clean":"2024-04-22T13:27:09.563070+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:27:09.563070+0000","last_undegraded":"2024-04-22T13:27:09.563070+0000","last_fullsized":"2024-04-22T13:27:09.563070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564799+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:27:09.564799+0000","last_peered":"2024-04-22T13:27:09.564799+0000","last_clean":"2024-04-22T13:27:09.564799+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:27:09.564799+0000","last_undegraded":"2024-04-22T13:27:09.564799+0000","last_fullsized":"2024-04-22T13:27:09.564799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562808+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:27:09.562808+0000","last_peered":"2024-04-22T13:27:09.562808+0000","last_clean":"2024-04-22T13:27:09.562808+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:27:09.562808+0000","last_undegraded":"2024-04-22T13:27:09.562808+0000","last_fullsized":"2024-04-22T13:27:09.562808+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450691+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:27:09.450691+0000","last_peered":"2024-04-22T13:27:09.450691+0000","last_clean":"2024-04-22T13:27:09.450691+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:27:09.450691+0000","last_undegraded":"2024-04-22T13:27:09.450691+0000","last_fullsized":"2024-04-22T13:27:09.450691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540308+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:27:09.540308+0000","last_peered":"2024-04-22T13:27:09.540308+0000","last_clean":"2024-04-22T13:27:09.540308+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:27:09.540308+0000","last_undegraded":"2024-04-22T13:27:09.540308+0000","last_fullsized":"2024-04-22T13:27:09.540308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562679+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:27:09.562679+0000","last_peered":"2024-04-22T13:27:09.562679+0000","last_clean":"2024-04-22T13:27:09.562679+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:27:09.562679+0000","last_undegraded":"2024-04-22T13:27:09.562679+0000","last_fullsized":"2024-04-22T13:27:09.562679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562789+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:27:09.562789+0000","last_peered":"2024-04-22T13:27:09.562789+0000","last_clean":"2024-04-22T13:27:09.562789+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:27:09.562789+0000","last_undegraded":"2024-04-22T13:27:09.562789+0000","last_fullsized":"2024-04-22T13:27:09.562789+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540822+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:27:09.540822+0000","last_peered":"2024-04-22T13:27:09.540822+0000","last_clean":"2024-04-22T13:27:09.540822+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:27:09.540822+0000","last_undegraded":"2024-04-22T13:27:09.540822+0000","last_fullsized":"2024-04-22T13:27:09.540822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450783+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:27:09.450783+0000","last_peered":"2024-04-22T13:27:09.450783+0000","last_clean":"2024-04-22T13:27:09.450783+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:27:09.450783+0000","last_undegraded":"2024-04-22T13:27:09.450783+0000","last_fullsized":"2024-04-22T13:27:09.450783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564270+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:27:09.564270+0000","last_peered":"2024-04-22T13:27:09.564270+0000","last_clean":"2024-04-22T13:27:09.564270+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:27:09.564270+0000","last_undegraded":"2024-04-22T13:27:09.564270+0000","last_fullsized":"2024-04-22T13:27:09.564270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":146,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399643+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:27:10.399643+0000","last_peered":"2024-04-22T13:27:10.399643+0000","last_clean":"2024-04-22T13:27:10.399643+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:27:10.399643+0000","last_undegraded":"2024-04-22T13:27:10.399643+0000","last_fullsized":"2024-04-22T13:27:10.399643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564284+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.564284+0000","last_peered":"2024-04-22T13:27:09.564284+0000","last_clean":"2024-04-22T13:27:09.564284+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:27:09.564284+0000","last_undegraded":"2024-04-22T13:27:09.564284+0000","last_fullsized":"2024-04-22T13:27:09.564284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.651858+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:27:09.651858+0000","last_peered":"2024-04-22T13:27:09.651858+0000","last_clean":"2024-04-22T13:27:09.651858+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:27:09.651858+0000","last_undegraded":"2024-04-22T13:27:09.651858+0000","last_fullsized":"2024-04-22T13:27:09.651858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564302+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:27:09.564302+0000","last_peered":"2024-04-22T13:27:09.564302+0000","last_clean":"2024-04-22T13:27:09.564302+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:27:09.564302+0000","last_undegraded":"2024-04-22T13:27:09.564302+0000","last_fullsized":"2024-04-22T13:27:09.564302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563457+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:27:09.563457+0000","last_peered":"2024-04-22T13:27:09.563457+0000","last_clean":"2024-04-22T13:27:09.563457+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:27:09.563457+0000","last_undegraded":"2024-04-22T13:27:09.563457+0000","last_fullsized":"2024-04-22T13:27:09.563457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564533+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:27:09.564533+0000","last_peered":"2024-04-22T13:27:09.564533+0000","last_clean":"2024-04-22T13:27:09.564533+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:27:09.564533+0000","last_undegraded":"2024-04-22T13:27:09.564533+0000","last_fullsized":"2024-04-22T13:27:09.564533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562753+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:27:09.562753+0000","last_peered":"2024-04-22T13:27:09.562753+0000","last_clean":"2024-04-22T13:27:09.562753+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:27:09.562753+0000","last_undegraded":"2024-04-22T13:27:09.562753+0000","last_fullsized":"2024-04-22T13:27:09.562753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563291+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:27:09.563291+0000","last_peered":"2024-04-22T13:27:09.563291+0000","last_clean":"2024-04-22T13:27:09.563291+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:27:09.563291+0000","last_undegraded":"2024-04-22T13:27:09.563291+0000","last_fullsized":"2024-04-22T13:27:09.563291+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563267+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:27:09.563267+0000","last_peered":"2024-04-22T13:27:09.563267+0000","last_clean":"2024-04-22T13:27:09.563267+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:27:09.563267+0000","last_undegraded":"2024-04-22T13:27:09.563267+0000","last_fullsized":"2024-04-22T13:27:09.563267+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564293+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:27:09.564293+0000","last_peered":"2024-04-22T13:27:09.564293+0000","last_clean":"2024-04-22T13:27:09.564293+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:27:09.564293+0000","last_undegraded":"2024-04-22T13:27:09.564293+0000","last_fullsized":"2024-04-22T13:27:09.564293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563391+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:27:09.563391+0000","last_peered":"2024-04-22T13:27:09.563391+0000","last_clean":"2024-04-22T13:27:09.563391+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:27:09.563391+0000","last_undegraded":"2024-04-22T13:27:09.563391+0000","last_fullsized":"2024-04-22T13:27:09.563391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563022+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:27:09.563022+0000","last_peered":"2024-04-22T13:27:09.563022+0000","last_clean":"2024-04-22T13:27:09.563022+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:27:09.563022+0000","last_undegraded":"2024-04-22T13:27:09.563022+0000","last_fullsized":"2024-04-22T13:27:09.563022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563649+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:27:09.563649+0000","last_peered":"2024-04-22T13:27:09.563649+0000","last_clean":"2024-04-22T13:27:09.563649+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:27:09.563649+0000","last_undegraded":"2024-04-22T13:27:09.563649+0000","last_fullsized":"2024-04-22T13:27:09.563649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541195+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:27:09.541195+0000","last_peered":"2024-04-22T13:27:09.541195+0000","last_clean":"2024-04-22T13:27:09.541195+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:27:09.541195+0000","last_undegraded":"2024-04-22T13:27:09.541195+0000","last_fullsized":"2024-04-22T13:27:09.541195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562758+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:27:09.562758+0000","last_peered":"2024-04-22T13:27:09.562758+0000","last_clean":"2024-04-22T13:27:09.562758+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:27:09.562758+0000","last_undegraded":"2024-04-22T13:27:09.562758+0000","last_fullsized":"2024-04-22T13:27:09.562758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540273+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:27:09.540273+0000","last_peered":"2024-04-22T13:27:09.540273+0000","last_clean":"2024-04-22T13:27:09.540273+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:27:09.540273+0000","last_undegraded":"2024-04-22T13:27:09.540273+0000","last_fullsized":"2024-04-22T13:27:09.540273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562941+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:27:09.562941+0000","last_peered":"2024-04-22T13:27:09.562941+0000","last_clean":"2024-04-22T13:27:09.562941+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:27:09.562941+0000","last_undegraded":"2024-04-22T13:27:09.562941+0000","last_fullsized":"2024-04-22T13:27:09.562941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563605+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:27:09.563605+0000","last_peered":"2024-04-22T13:27:09.563605+0000","last_clean":"2024-04-22T13:27:09.563605+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:27:09.563605+0000","last_undegraded":"2024-04-22T13:27:09.563605+0000","last_fullsized":"2024-04-22T13:27:09.563605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562834+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:27:09.562834+0000","last_peered":"2024-04-22T13:27:09.562834+0000","last_clean":"2024-04-22T13:27:09.562834+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:27:09.562834+0000","last_undegraded":"2024-04-22T13:27:09.562834+0000","last_fullsized":"2024-04-22T13:27:09.562834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597354,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1275,"num_read_kb":751,"num_write":366,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252899,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":485126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514536,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1134854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1036288,"data_stored":484357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939700,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1425408,"data_stored":877951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41092,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704060,"statfs":{"total":95995035648,"available":95952957440,"internally_reserved":0,"allocated":1630208,"data_stored":1077785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":549872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":486112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":550932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40492,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704660,"statfs":{"total":95995035648,"available":95953571840,"internally_reserved":0,"allocated":1019904,"data_stored":474972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37252,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707900,"statfs":{"total":95995035648,"available":95956889600,"internally_reserved":0,"allocated":1777664,"data_stored":1208672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":197830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:27:14.308 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-22T13:27:14.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/2010620546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 msgr2=0x7fdca810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/2010620546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca810ee10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009ca0 tx=0x7fdca002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/2010620546 shutdown_connections 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/2010620546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca810ee10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/2010620546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdca8102b20 0x7fdca8106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/2010620546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca8102170 0x7fdca8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/2010620546 >> 172.21.15.5:0/2010620546 conn(0x7fdca80fd9d0 msgr2=0x7fdca80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/2010620546 shutdown_connections 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.452+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/2010620546 wait complete. 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 Processor -- start 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 -- start start 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca8102170 0x7fdca8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdca8074cd0 0x7fdca81a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8112190 con 0x7fdca8107240 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8112010 con 0x7fdca8102170 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcaf7d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8112310 con 0x7fdca8074cd0 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50922/0 (socket says 172.21.15.5:50922) 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 -- 172.21.15.5:0/1539054952 learned_addr learned my addr 172.21.15.5:0/1539054952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcad571640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca8102170 0x7fdca8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcadd72640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdca8074cd0 0x7fdca81a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 -- 172.21.15.5:0/1539054952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdca8074cd0 msgr2=0x7fdca81a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdca8074cd0 0x7fdca81a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 -- 172.21.15.5:0/1539054952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca8102170 msgr2=0x7fdca8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca8102170 0x7fdca8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.453+0000 7fdcacd70640 1 -- 172.21.15.5:0/1539054952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca81a6fa0 con 0x7fdca8107240 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.454+0000 7fdcacd70640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca8074790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e9c0 tx=0x7fdc9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.454+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800cd80 con 0x7fdca8107240 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.454+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc980054d0 con 0x7fdca8107240 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.454+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca81a7290 con 0x7fdca8107240 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.454+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98010430 con 0x7fdca8107240 2024-04-22T13:27:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.454+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca81a7830 con 0x7fdca8107240 2024-04-22T13:27:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.455+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca8069250 con 0x7fdca8107240 2024-04-22T13:27:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.456+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fdc98004f90 con 0x7fdca8107240 2024-04-22T13:27:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.457+0000 7fdc9e7fc640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc940a3ef0 0x7fdc940a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.457+0000 7fdcad571640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc940a3ef0 0x7fdc940a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.457+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fdc980d0d40 con 0x7fdca8107240 2024-04-22T13:27:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.457+0000 7fdcad571640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc940a3ef0 0x7fdc940a63b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdca400bbe0 tx=0x7fdca400b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.461+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc9809d1c0 con 0x7fdca8107240 2024-04-22T13:27:14.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.646+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdca810a540 con 0x7fdc940a3ef0 2024-04-22T13:27:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.653+0000 7fdc9e7fc640 1 -- 172.21.15.5:0/1539054952 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257789 (secure 0 0 0) 0x7fdca4010c30 con 0x7fdc940a3ef0 2024-04-22T13:27:14.652 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:14.653 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:27:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc940a3ef0 msgr2=0x7fdc940a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc940a3ef0 0x7fdc940a63b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdca400bbe0 tx=0x7fdca400b390 comp rx=0 tx=0).stop 2024-04-22T13:27:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 msgr2=0x7fdca8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca8074790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e9c0 tx=0x7fdc9800ee80 comp rx=0 tx=0).stop 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 shutdown_connections 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdca8107240 0x7fdca8074790 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc940a3ef0 0x7fdc940a63b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.656+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdca8074cd0 0x7fdca81a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.657+0000 7fdcaf7d5640 1 --2- 172.21.15.5:0/1539054952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdca8102170 0x7fdca8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.657+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 >> 172.21.15.5:0/1539054952 conn(0x7fdca80fd9d0 msgr2=0x7fdca80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.657+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 shutdown_connections 2024-04-22T13:27:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.657+0000 7fdcaf7d5640 1 -- 172.21.15.5:0/1539054952 wait complete. 2024-04-22T13:27:14.663 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T13:27:13.460500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187956,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1411,"num_read_kb":870,"num_write":549,"num_write_kb":4397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":933,"ondisk_log_size":933,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":413824,"kb_used_data":14536,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528000,"statfs":{"total":1151940427776,"available":1151516672000,"internally_reserved":0,"allocated":14884864,"data_stored":8299033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988589"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563085+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:27:09.563085+0000","last_peered":"2024-04-22T13:27:09.563085+0000","last_clean":"2024-04-22T13:27:09.563085+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:27:09.563085+0000","last_undegraded":"2024-04-22T13:27:09.563085+0000","last_fullsized":"2024-04-22T13:27:09.563085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564651+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:27:09.564651+0000","last_peered":"2024-04-22T13:27:09.564651+0000","last_clean":"2024-04-22T13:27:09.564651+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:27:09.564651+0000","last_undegraded":"2024-04-22T13:27:09.564651+0000","last_fullsized":"2024-04-22T13:27:09.564651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564621+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:27:09.564621+0000","last_peered":"2024-04-22T13:27:09.564621+0000","last_clean":"2024-04-22T13:27:09.564621+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:27:09.564621+0000","last_undegraded":"2024-04-22T13:27:09.564621+0000","last_fullsized":"2024-04-22T13:27:09.564621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563725+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:27:09.563725+0000","last_peered":"2024-04-22T13:27:09.563725+0000","last_clean":"2024-04-22T13:27:09.563725+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:27:09.563725+0000","last_undegraded":"2024-04-22T13:27:09.563725+0000","last_fullsized":"2024-04-22T13:27:09.563725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448296+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:27:09.448296+0000","last_peered":"2024-04-22T13:27:09.448296+0000","last_clean":"2024-04-22T13:27:09.448296+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:27:09.448296+0000","last_undegraded":"2024-04-22T13:27:09.448296+0000","last_fullsized":"2024-04-22T13:27:09.448296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564063+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:27:09.564063+0000","last_peered":"2024-04-22T13:27:09.564063+0000","last_clean":"2024-04-22T13:27:09.564063+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:27:09.564063+0000","last_undegraded":"2024-04-22T13:27:09.564063+0000","last_fullsized":"2024-04-22T13:27:09.564063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540227+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:27:09.540227+0000","last_peered":"2024-04-22T13:27:09.540227+0000","last_clean":"2024-04-22T13:27:09.540227+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:27:09.540227+0000","last_undegraded":"2024-04-22T13:27:09.540227+0000","last_fullsized":"2024-04-22T13:27:09.540227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563416+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:27:09.563416+0000","last_peered":"2024-04-22T13:27:09.563416+0000","last_clean":"2024-04-22T13:27:09.563416+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:27:09.563416+0000","last_undegraded":"2024-04-22T13:27:09.563416+0000","last_fullsized":"2024-04-22T13:27:09.563416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448852+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:27:09.448852+0000","last_peered":"2024-04-22T13:27:09.448852+0000","last_clean":"2024-04-22T13:27:09.448852+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:27:09.448852+0000","last_undegraded":"2024-04-22T13:27:09.448852+0000","last_fullsized":"2024-04-22T13:27:09.448852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564466+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:27:09.564466+0000","last_peered":"2024-04-22T13:27:09.564466+0000","last_clean":"2024-04-22T13:27:09.564466+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:27:09.564466+0000","last_undegraded":"2024-04-22T13:27:09.564466+0000","last_fullsized":"2024-04-22T13:27:09.564466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563562+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:27:09.563562+0000","last_peered":"2024-04-22T13:27:09.563562+0000","last_clean":"2024-04-22T13:27:09.563562+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:27:09.563562+0000","last_undegraded":"2024-04-22T13:27:09.563562+0000","last_fullsized":"2024-04-22T13:27:09.563562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563593+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:27:09.563593+0000","last_peered":"2024-04-22T13:27:09.563593+0000","last_clean":"2024-04-22T13:27:09.563593+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:27:09.563593+0000","last_undegraded":"2024-04-22T13:27:09.563593+0000","last_fullsized":"2024-04-22T13:27:09.563593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563776+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:27:09.563776+0000","last_peered":"2024-04-22T13:27:09.563776+0000","last_clean":"2024-04-22T13:27:09.563776+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:27:09.563776+0000","last_undegraded":"2024-04-22T13:27:09.563776+0000","last_fullsized":"2024-04-22T13:27:09.563776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562794+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:27:09.562794+0000","last_peered":"2024-04-22T13:27:09.562794+0000","last_clean":"2024-04-22T13:27:09.562794+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:27:09.562794+0000","last_undegraded":"2024-04-22T13:27:09.562794+0000","last_fullsized":"2024-04-22T13:27:09.562794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562713+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:27:09.562713+0000","last_peered":"2024-04-22T13:27:09.562713+0000","last_clean":"2024-04-22T13:27:09.562713+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:27:09.562713+0000","last_undegraded":"2024-04-22T13:27:09.562713+0000","last_fullsized":"2024-04-22T13:27:09.562713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563250+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.563250+0000","last_peered":"2024-04-22T13:27:09.563250+0000","last_clean":"2024-04-22T13:27:09.563250+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:27:09.563250+0000","last_undegraded":"2024-04-22T13:27:09.563250+0000","last_fullsized":"2024-04-22T13:27:09.563250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446872+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:27:09.446872+0000","last_peered":"2024-04-22T13:27:09.446872+0000","last_clean":"2024-04-22T13:27:09.446872+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:27:09.446872+0000","last_undegraded":"2024-04-22T13:27:09.446872+0000","last_fullsized":"2024-04-22T13:27:09.446872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563565+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:27:09.563565+0000","last_peered":"2024-04-22T13:27:09.563565+0000","last_clean":"2024-04-22T13:27:09.563565+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:27:09.563565+0000","last_undegraded":"2024-04-22T13:27:09.563565+0000","last_fullsized":"2024-04-22T13:27:09.563565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.695974+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:27:09.695974+0000","last_peered":"2024-04-22T13:27:09.695974+0000","last_clean":"2024-04-22T13:27:09.695974+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:27:09.695974+0000","last_undegraded":"2024-04-22T13:27:09.695974+0000","last_fullsized":"2024-04-22T13:27:09.695974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541055+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:27:09.541055+0000","last_peered":"2024-04-22T13:27:09.541055+0000","last_clean":"2024-04-22T13:27:09.541055+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:27:09.541055+0000","last_undegraded":"2024-04-22T13:27:09.541055+0000","last_fullsized":"2024-04-22T13:27:09.541055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541272+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:27:09.541272+0000","last_peered":"2024-04-22T13:27:09.541272+0000","last_clean":"2024-04-22T13:27:09.541272+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:27:09.541272+0000","last_undegraded":"2024-04-22T13:27:09.541272+0000","last_fullsized":"2024-04-22T13:27:09.541272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564793+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:27:09.564793+0000","last_peered":"2024-04-22T13:27:09.564793+0000","last_clean":"2024-04-22T13:27:09.564793+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:27:09.564793+0000","last_undegraded":"2024-04-22T13:27:09.564793+0000","last_fullsized":"2024-04-22T13:27:09.564793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562983+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:27:09.562983+0000","last_peered":"2024-04-22T13:27:09.562983+0000","last_clean":"2024-04-22T13:27:09.562983+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:27:09.562983+0000","last_undegraded":"2024-04-22T13:27:09.562983+0000","last_fullsized":"2024-04-22T13:27:09.562983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562953+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:27:09.562953+0000","last_peered":"2024-04-22T13:27:09.562953+0000","last_clean":"2024-04-22T13:27:09.562953+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:27:09.562953+0000","last_undegraded":"2024-04-22T13:27:09.562953+0000","last_fullsized":"2024-04-22T13:27:09.562953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.451042+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:27:09.451042+0000","last_peered":"2024-04-22T13:27:09.451042+0000","last_clean":"2024-04-22T13:27:09.451042+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:27:09.451042+0000","last_undegraded":"2024-04-22T13:27:09.451042+0000","last_fullsized":"2024-04-22T13:27:09.451042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547806+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:27:09.547806+0000","last_peered":"2024-04-22T13:27:09.547806+0000","last_clean":"2024-04-22T13:27:09.547806+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:27:09.547806+0000","last_undegraded":"2024-04-22T13:27:09.547806+0000","last_fullsized":"2024-04-22T13:27:09.547806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547716+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:27:09.547716+0000","last_peered":"2024-04-22T13:27:09.547716+0000","last_clean":"2024-04-22T13:27:09.547716+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:27:09.547716+0000","last_undegraded":"2024-04-22T13:27:09.547716+0000","last_fullsized":"2024-04-22T13:27:09.547716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450920+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:27:09.450920+0000","last_peered":"2024-04-22T13:27:09.450920+0000","last_clean":"2024-04-22T13:27:09.450920+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:27:09.450920+0000","last_undegraded":"2024-04-22T13:27:09.450920+0000","last_fullsized":"2024-04-22T13:27:09.450920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448243+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:27:09.448243+0000","last_peered":"2024-04-22T13:27:09.448243+0000","last_clean":"2024-04-22T13:27:09.448243+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:27:09.448243+0000","last_undegraded":"2024-04-22T13:27:09.448243+0000","last_fullsized":"2024-04-22T13:27:09.448243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547784+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:27:09.547784+0000","last_peered":"2024-04-22T13:27:09.547784+0000","last_clean":"2024-04-22T13:27:09.547784+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:27:09.547784+0000","last_undegraded":"2024-04-22T13:27:09.547784+0000","last_fullsized":"2024-04-22T13:27:09.547784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562955+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:27:09.562955+0000","last_peered":"2024-04-22T13:27:09.562955+0000","last_clean":"2024-04-22T13:27:09.562955+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:27:09.562955+0000","last_undegraded":"2024-04-22T13:27:09.562955+0000","last_fullsized":"2024-04-22T13:27:09.562955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540293+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:27:09.540293+0000","last_peered":"2024-04-22T13:27:09.540293+0000","last_clean":"2024-04-22T13:27:09.540293+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:27:09.540293+0000","last_undegraded":"2024-04-22T13:27:09.540293+0000","last_fullsized":"2024-04-22T13:27:09.540293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547473+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:27:09.547473+0000","last_peered":"2024-04-22T13:27:09.547473+0000","last_clean":"2024-04-22T13:27:09.547473+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:27:09.547473+0000","last_undegraded":"2024-04-22T13:27:09.547473+0000","last_fullsized":"2024-04-22T13:27:09.547473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446526+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:27:09.446526+0000","last_peered":"2024-04-22T13:27:09.446526+0000","last_clean":"2024-04-22T13:27:09.446526+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:27:09.446526+0000","last_undegraded":"2024-04-22T13:27:09.446526+0000","last_fullsized":"2024-04-22T13:27:09.446526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":255,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399316+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:27:10.399316+0000","last_peered":"2024-04-22T13:27:10.399316+0000","last_clean":"2024-04-22T13:27:10.399316+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:27:10.399316+0000","last_undegraded":"2024-04-22T13:27:10.399316+0000","last_fullsized":"2024-04-22T13:27:10.399316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563427+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:27:09.563427+0000","last_peered":"2024-04-22T13:27:09.563427+0000","last_clean":"2024-04-22T13:27:09.563427+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:27:09.563427+0000","last_undegraded":"2024-04-22T13:27:09.563427+0000","last_fullsized":"2024-04-22T13:27:09.563427+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564441+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:27:09.564441+0000","last_peered":"2024-04-22T13:27:09.564441+0000","last_clean":"2024-04-22T13:27:09.564441+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:27:09.564441+0000","last_undegraded":"2024-04-22T13:27:09.564441+0000","last_fullsized":"2024-04-22T13:27:09.564441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564869+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:27:09.564869+0000","last_peered":"2024-04-22T13:27:09.564869+0000","last_clean":"2024-04-22T13:27:09.564869+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:27:09.564869+0000","last_undegraded":"2024-04-22T13:27:09.564869+0000","last_fullsized":"2024-04-22T13:27:09.564869+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547887+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:27:09.547887+0000","last_peered":"2024-04-22T13:27:09.547887+0000","last_clean":"2024-04-22T13:27:09.547887+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:27:09.547887+0000","last_undegraded":"2024-04-22T13:27:09.547887+0000","last_fullsized":"2024-04-22T13:27:09.547887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562984+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:27:09.562984+0000","last_peered":"2024-04-22T13:27:09.562984+0000","last_clean":"2024-04-22T13:27:09.562984+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:27:09.562984+0000","last_undegraded":"2024-04-22T13:27:09.562984+0000","last_fullsized":"2024-04-22T13:27:09.562984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":293,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.404428+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:27:10.404428+0000","last_peered":"2024-04-22T13:27:10.404428+0000","last_clean":"2024-04-22T13:27:10.404428+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:27:10.404428+0000","last_undegraded":"2024-04-22T13:27:10.404428+0000","last_fullsized":"2024-04-22T13:27:10.404428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":237,"num_read_kb":182,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450855+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:27:09.450855+0000","last_peered":"2024-04-22T13:27:09.450855+0000","last_clean":"2024-04-22T13:27:09.450855+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:27:09.450855+0000","last_undegraded":"2024-04-22T13:27:09.450855+0000","last_fullsized":"2024-04-22T13:27:09.450855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563392+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:27:09.563392+0000","last_peered":"2024-04-22T13:27:09.563392+0000","last_clean":"2024-04-22T13:27:09.563392+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:27:09.563392+0000","last_undegraded":"2024-04-22T13:27:09.563392+0000","last_fullsized":"2024-04-22T13:27:09.563392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562910+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:27:09.562910+0000","last_peered":"2024-04-22T13:27:09.562910+0000","last_clean":"2024-04-22T13:27:09.562910+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:27:09.562910+0000","last_undegraded":"2024-04-22T13:27:09.562910+0000","last_fullsized":"2024-04-22T13:27:09.562910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547599+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:27:09.547599+0000","last_peered":"2024-04-22T13:27:09.547599+0000","last_clean":"2024-04-22T13:27:09.547599+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:27:09.547599+0000","last_undegraded":"2024-04-22T13:27:09.547599+0000","last_fullsized":"2024-04-22T13:27:09.547599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564267+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:27:09.564267+0000","last_peered":"2024-04-22T13:27:09.564267+0000","last_clean":"2024-04-22T13:27:09.564267+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:27:09.564267+0000","last_undegraded":"2024-04-22T13:27:09.564267+0000","last_fullsized":"2024-04-22T13:27:09.564267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563351+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:27:09.563351+0000","last_peered":"2024-04-22T13:27:09.563351+0000","last_clean":"2024-04-22T13:27:09.563351+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:27:09.563351+0000","last_undegraded":"2024-04-22T13:27:09.563351+0000","last_fullsized":"2024-04-22T13:27:09.563351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563985+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:27:09.563985+0000","last_peered":"2024-04-22T13:27:09.563985+0000","last_clean":"2024-04-22T13:27:09.563985+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:27:09.563985+0000","last_undegraded":"2024-04-22T13:27:09.563985+0000","last_fullsized":"2024-04-22T13:27:09.563985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450657+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:27:09.450657+0000","last_peered":"2024-04-22T13:27:09.450657+0000","last_clean":"2024-04-22T13:27:09.450657+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:27:09.450657+0000","last_undegraded":"2024-04-22T13:27:09.450657+0000","last_fullsized":"2024-04-22T13:27:09.450657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547522+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:27:09.547522+0000","last_peered":"2024-04-22T13:27:09.547522+0000","last_clean":"2024-04-22T13:27:09.547522+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:27:09.547522+0000","last_undegraded":"2024-04-22T13:27:09.547522+0000","last_fullsized":"2024-04-22T13:27:09.547522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450885+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:27:09.450885+0000","last_peered":"2024-04-22T13:27:09.450885+0000","last_clean":"2024-04-22T13:27:09.450885+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:27:09.450885+0000","last_undegraded":"2024-04-22T13:27:09.450885+0000","last_fullsized":"2024-04-22T13:27:09.450885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563035+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:27:09.563035+0000","last_peered":"2024-04-22T13:27:09.563035+0000","last_clean":"2024-04-22T13:27:09.563035+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:27:09.563035+0000","last_undegraded":"2024-04-22T13:27:09.563035+0000","last_fullsized":"2024-04-22T13:27:09.563035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446714+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:27:09.446714+0000","last_peered":"2024-04-22T13:27:09.446714+0000","last_clean":"2024-04-22T13:27:09.446714+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:27:09.446714+0000","last_undegraded":"2024-04-22T13:27:09.446714+0000","last_fullsized":"2024-04-22T13:27:09.446714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448687+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:27:09.448687+0000","last_peered":"2024-04-22T13:27:09.448687+0000","last_clean":"2024-04-22T13:27:09.448687+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:27:09.448687+0000","last_undegraded":"2024-04-22T13:27:09.448687+0000","last_fullsized":"2024-04-22T13:27:09.448687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399166+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:27:10.399166+0000","last_peered":"2024-04-22T13:27:10.399166+0000","last_clean":"2024-04-22T13:27:10.399166+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:27:10.399166+0000","last_undegraded":"2024-04-22T13:27:10.399166+0000","last_fullsized":"2024-04-22T13:27:10.399166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448498+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:27:09.448498+0000","last_peered":"2024-04-22T13:27:09.448498+0000","last_clean":"2024-04-22T13:27:09.448498+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:27:09.448498+0000","last_undegraded":"2024-04-22T13:27:09.448498+0000","last_fullsized":"2024-04-22T13:27:09.448498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446492+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:27:09.446492+0000","last_peered":"2024-04-22T13:27:09.446492+0000","last_clean":"2024-04-22T13:27:09.446492+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:27:09.446492+0000","last_undegraded":"2024-04-22T13:27:09.446492+0000","last_fullsized":"2024-04-22T13:27:09.446492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":141,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563174+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:27:09.563174+0000","last_peered":"2024-04-22T13:27:09.563174+0000","last_clean":"2024-04-22T13:27:09.563174+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:27:09.563174+0000","last_undegraded":"2024-04-22T13:27:09.563174+0000","last_fullsized":"2024-04-22T13:27:09.563174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540568+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:27:09.540568+0000","last_peered":"2024-04-22T13:27:09.540568+0000","last_clean":"2024-04-22T13:27:09.540568+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:27:09.540568+0000","last_undegraded":"2024-04-22T13:27:09.540568+0000","last_fullsized":"2024-04-22T13:27:09.540568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563433+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:27:09.563433+0000","last_peered":"2024-04-22T13:27:09.563433+0000","last_clean":"2024-04-22T13:27:09.563433+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:27:09.563433+0000","last_undegraded":"2024-04-22T13:27:09.563433+0000","last_fullsized":"2024-04-22T13:27:09.563433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564057+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:27:09.564057+0000","last_peered":"2024-04-22T13:27:09.564057+0000","last_clean":"2024-04-22T13:27:09.564057+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:27:09.564057+0000","last_undegraded":"2024-04-22T13:27:09.564057+0000","last_fullsized":"2024-04-22T13:27:09.564057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562773+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:27:09.562773+0000","last_peered":"2024-04-22T13:27:09.562773+0000","last_clean":"2024-04-22T13:27:09.562773+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:27:09.562773+0000","last_undegraded":"2024-04-22T13:27:09.562773+0000","last_fullsized":"2024-04-22T13:27:09.562773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564209+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:27:09.564209+0000","last_peered":"2024-04-22T13:27:09.564209+0000","last_clean":"2024-04-22T13:27:09.564209+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:27:09.564209+0000","last_undegraded":"2024-04-22T13:27:09.564209+0000","last_fullsized":"2024-04-22T13:27:09.564209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563221+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:27:09.563221+0000","last_peered":"2024-04-22T13:27:09.563221+0000","last_clean":"2024-04-22T13:27:09.563221+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:27:09.563221+0000","last_undegraded":"2024-04-22T13:27:09.563221+0000","last_fullsized":"2024-04-22T13:27:09.563221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.657844+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:27:09.657844+0000","last_peered":"2024-04-22T13:27:09.657844+0000","last_clean":"2024-04-22T13:27:09.657844+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:27:09.657844+0000","last_undegraded":"2024-04-22T13:27:09.657844+0000","last_fullsized":"2024-04-22T13:27:09.657844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541266+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:27:09.541266+0000","last_peered":"2024-04-22T13:27:09.541266+0000","last_clean":"2024-04-22T13:27:09.541266+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:27:09.541266+0000","last_undegraded":"2024-04-22T13:27:09.541266+0000","last_fullsized":"2024-04-22T13:27:09.541266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563070+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:27:09.563070+0000","last_peered":"2024-04-22T13:27:09.563070+0000","last_clean":"2024-04-22T13:27:09.563070+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:27:09.563070+0000","last_undegraded":"2024-04-22T13:27:09.563070+0000","last_fullsized":"2024-04-22T13:27:09.563070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564799+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:27:09.564799+0000","last_peered":"2024-04-22T13:27:09.564799+0000","last_clean":"2024-04-22T13:27:09.564799+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:27:09.564799+0000","last_undegraded":"2024-04-22T13:27:09.564799+0000","last_fullsized":"2024-04-22T13:27:09.564799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562808+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:27:09.562808+0000","last_peered":"2024-04-22T13:27:09.562808+0000","last_clean":"2024-04-22T13:27:09.562808+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:27:09.562808+0000","last_undegraded":"2024-04-22T13:27:09.562808+0000","last_fullsized":"2024-04-22T13:27:09.562808+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450691+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:27:09.450691+0000","last_peered":"2024-04-22T13:27:09.450691+0000","last_clean":"2024-04-22T13:27:09.450691+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:27:09.450691+0000","last_undegraded":"2024-04-22T13:27:09.450691+0000","last_fullsized":"2024-04-22T13:27:09.450691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540308+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:27:09.540308+0000","last_peered":"2024-04-22T13:27:09.540308+0000","last_clean":"2024-04-22T13:27:09.540308+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:27:09.540308+0000","last_undegraded":"2024-04-22T13:27:09.540308+0000","last_fullsized":"2024-04-22T13:27:09.540308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562679+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:27:09.562679+0000","last_peered":"2024-04-22T13:27:09.562679+0000","last_clean":"2024-04-22T13:27:09.562679+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:27:09.562679+0000","last_undegraded":"2024-04-22T13:27:09.562679+0000","last_fullsized":"2024-04-22T13:27:09.562679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562789+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:27:09.562789+0000","last_peered":"2024-04-22T13:27:09.562789+0000","last_clean":"2024-04-22T13:27:09.562789+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:27:09.562789+0000","last_undegraded":"2024-04-22T13:27:09.562789+0000","last_fullsized":"2024-04-22T13:27:09.562789+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540822+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:27:09.540822+0000","last_peered":"2024-04-22T13:27:09.540822+0000","last_clean":"2024-04-22T13:27:09.540822+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:27:09.540822+0000","last_undegraded":"2024-04-22T13:27:09.540822+0000","last_fullsized":"2024-04-22T13:27:09.540822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450783+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:27:09.450783+0000","last_peered":"2024-04-22T13:27:09.450783+0000","last_clean":"2024-04-22T13:27:09.450783+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:27:09.450783+0000","last_undegraded":"2024-04-22T13:27:09.450783+0000","last_fullsized":"2024-04-22T13:27:09.450783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564270+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:27:09.564270+0000","last_peered":"2024-04-22T13:27:09.564270+0000","last_clean":"2024-04-22T13:27:09.564270+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:27:09.564270+0000","last_undegraded":"2024-04-22T13:27:09.564270+0000","last_fullsized":"2024-04-22T13:27:09.564270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":146,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:10.399643+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:27:10.399643+0000","last_peered":"2024-04-22T13:27:10.399643+0000","last_clean":"2024-04-22T13:27:10.399643+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:27:10.399643+0000","last_undegraded":"2024-04-22T13:27:10.399643+0000","last_fullsized":"2024-04-22T13:27:10.399643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564284+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.564284+0000","last_peered":"2024-04-22T13:27:09.564284+0000","last_clean":"2024-04-22T13:27:09.564284+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:27:09.564284+0000","last_undegraded":"2024-04-22T13:27:09.564284+0000","last_fullsized":"2024-04-22T13:27:09.564284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.651858+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:27:09.651858+0000","last_peered":"2024-04-22T13:27:09.651858+0000","last_clean":"2024-04-22T13:27:09.651858+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:27:09.651858+0000","last_undegraded":"2024-04-22T13:27:09.651858+0000","last_fullsized":"2024-04-22T13:27:09.651858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564302+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:27:09.564302+0000","last_peered":"2024-04-22T13:27:09.564302+0000","last_clean":"2024-04-22T13:27:09.564302+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:27:09.564302+0000","last_undegraded":"2024-04-22T13:27:09.564302+0000","last_fullsized":"2024-04-22T13:27:09.564302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563457+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:27:09.563457+0000","last_peered":"2024-04-22T13:27:09.563457+0000","last_clean":"2024-04-22T13:27:09.563457+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:27:09.563457+0000","last_undegraded":"2024-04-22T13:27:09.563457+0000","last_fullsized":"2024-04-22T13:27:09.563457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564533+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:27:09.564533+0000","last_peered":"2024-04-22T13:27:09.564533+0000","last_clean":"2024-04-22T13:27:09.564533+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:27:09.564533+0000","last_undegraded":"2024-04-22T13:27:09.564533+0000","last_fullsized":"2024-04-22T13:27:09.564533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562753+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:27:09.562753+0000","last_peered":"2024-04-22T13:27:09.562753+0000","last_clean":"2024-04-22T13:27:09.562753+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:27:09.562753+0000","last_undegraded":"2024-04-22T13:27:09.562753+0000","last_fullsized":"2024-04-22T13:27:09.562753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563291+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:27:09.563291+0000","last_peered":"2024-04-22T13:27:09.563291+0000","last_clean":"2024-04-22T13:27:09.563291+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:27:09.563291+0000","last_undegraded":"2024-04-22T13:27:09.563291+0000","last_fullsized":"2024-04-22T13:27:09.563291+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563267+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:27:09.563267+0000","last_peered":"2024-04-22T13:27:09.563267+0000","last_clean":"2024-04-22T13:27:09.563267+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:27:09.563267+0000","last_undegraded":"2024-04-22T13:27:09.563267+0000","last_fullsized":"2024-04-22T13:27:09.563267+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564293+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:27:09.564293+0000","last_peered":"2024-04-22T13:27:09.564293+0000","last_clean":"2024-04-22T13:27:09.564293+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:27:09.564293+0000","last_undegraded":"2024-04-22T13:27:09.564293+0000","last_fullsized":"2024-04-22T13:27:09.564293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563391+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:27:09.563391+0000","last_peered":"2024-04-22T13:27:09.563391+0000","last_clean":"2024-04-22T13:27:09.563391+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:27:09.563391+0000","last_undegraded":"2024-04-22T13:27:09.563391+0000","last_fullsized":"2024-04-22T13:27:09.563391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563022+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:27:09.563022+0000","last_peered":"2024-04-22T13:27:09.563022+0000","last_clean":"2024-04-22T13:27:09.563022+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:27:09.563022+0000","last_undegraded":"2024-04-22T13:27:09.563022+0000","last_fullsized":"2024-04-22T13:27:09.563022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563649+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:27:09.563649+0000","last_peered":"2024-04-22T13:27:09.563649+0000","last_clean":"2024-04-22T13:27:09.563649+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:27:09.563649+0000","last_undegraded":"2024-04-22T13:27:09.563649+0000","last_fullsized":"2024-04-22T13:27:09.563649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541195+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:27:09.541195+0000","last_peered":"2024-04-22T13:27:09.541195+0000","last_clean":"2024-04-22T13:27:09.541195+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:27:09.541195+0000","last_undegraded":"2024-04-22T13:27:09.541195+0000","last_fullsized":"2024-04-22T13:27:09.541195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562758+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:27:09.562758+0000","last_peered":"2024-04-22T13:27:09.562758+0000","last_clean":"2024-04-22T13:27:09.562758+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:27:09.562758+0000","last_undegraded":"2024-04-22T13:27:09.562758+0000","last_fullsized":"2024-04-22T13:27:09.562758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540273+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:27:09.540273+0000","last_peered":"2024-04-22T13:27:09.540273+0000","last_clean":"2024-04-22T13:27:09.540273+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:27:09.540273+0000","last_undegraded":"2024-04-22T13:27:09.540273+0000","last_fullsized":"2024-04-22T13:27:09.540273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562941+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:27:09.562941+0000","last_peered":"2024-04-22T13:27:09.562941+0000","last_clean":"2024-04-22T13:27:09.562941+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:27:09.562941+0000","last_undegraded":"2024-04-22T13:27:09.562941+0000","last_fullsized":"2024-04-22T13:27:09.562941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563605+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:27:09.563605+0000","last_peered":"2024-04-22T13:27:09.563605+0000","last_clean":"2024-04-22T13:27:09.563605+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:27:09.563605+0000","last_undegraded":"2024-04-22T13:27:09.563605+0000","last_fullsized":"2024-04-22T13:27:09.563605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562834+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:27:09.562834+0000","last_peered":"2024-04-22T13:27:09.562834+0000","last_clean":"2024-04-22T13:27:09.562834+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:27:09.562834+0000","last_undegraded":"2024-04-22T13:27:09.562834+0000","last_fullsized":"2024-04-22T13:27:09.562834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597354,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1275,"num_read_kb":751,"num_write":366,"num_write_kb":1457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252899,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":485126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514536,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1134854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1036288,"data_stored":484357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939700,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1425408,"data_stored":877951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41092,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704060,"statfs":{"total":95995035648,"available":95952957440,"internally_reserved":0,"allocated":1630208,"data_stored":1077785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":549872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":486112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":550932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40492,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704660,"statfs":{"total":95995035648,"available":95953571840,"internally_reserved":0,"allocated":1019904,"data_stored":474972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37252,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707900,"statfs":{"total":95995035648,"available":95956889600,"internally_reserved":0,"allocated":1777664,"data_stored":1208672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":197830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:27:14.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 -- 172.21.15.5:0/2201689842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc68104d70 msgr2=0x7fdc68108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 --2- 172.21.15.5:0/2201689842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc68104d70 0x7fdc68108db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009900 tx=0x7fdc5802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 -- 172.21.15.5:0/2201689842 shutdown_connections 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 --2- 172.21.15.5:0/2201689842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc68104d70 0x7fdc68108db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 --2- 172.21.15.5:0/2201689842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc68109490 0x7fdc68111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 --2- 172.21.15.5:0/2201689842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc681043c0 0x7fdc681047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 -- 172.21.15.5:0/2201689842 >> 172.21.15.5:0/2201689842 conn(0x7fdc680ffc00 msgr2=0x7fdc68102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 -- 172.21.15.5:0/2201689842 shutdown_connections 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.810+0000 7fdc7045c640 1 -- 172.21.15.5:0/2201689842 wait complete. 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 Processor -- start 2024-04-22T13:27:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 -- start start 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc681043c0 0x7fdc681a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68109490 0x7fdc681a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 0x7fdc6819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68114400 con 0x7fdc6819a740 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68114280 con 0x7fdc681043c0 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc7045c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68114580 con 0x7fdc68109490 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc6e9f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 0x7fdc6819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc6e9f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 0x7fdc6819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50934/0 (socket says 172.21.15.5:50934) 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc6e9f9640 1 -- 172.21.15.5:0/1893841309 learned_addr learned my addr 172.21.15.5:0/1893841309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc6e1f8640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc681043c0 0x7fdc681a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.811+0000 7fdc6d9f7640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68109490 0x7fdc681a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc6e9f9640 1 -- 172.21.15.5:0/1893841309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68109490 msgr2=0x7fdc681a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc6e9f9640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68109490 0x7fdc681a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc6e9f9640 1 -- 172.21.15.5:0/1893841309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc681043c0 msgr2=0x7fdc681a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc6e9f9640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc681043c0 0x7fdc681a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc6e9f9640 1 -- 172.21.15.5:0/1893841309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6819b340 con 0x7fdc6819a740 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc6e9f9640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 0x7fdc6819abc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000eaa0 tx=0x7fdc6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000cdb0 con 0x7fdc6819a740 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc60005020 con 0x7fdc6819a740 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc681a7420 con 0x7fdc6819a740 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600116d0 con 0x7fdc6819a740 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.812+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc681a77d0 con 0x7fdc6819a740 2024-04-22T13:27:14.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.813+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc68105890 con 0x7fdc6819a740 2024-04-22T13:27:14.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.814+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fdc600051d0 con 0x7fdc6819a740 2024-04-22T13:27:14.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.815+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc540a3e40 0x7fdc540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:14.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.815+0000 7fdc6e1f8640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc540a3e40 0x7fdc540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:14.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.815+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fdc600d0b30 con 0x7fdc6819a740 2024-04-22T13:27:14.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.815+0000 7fdc6e1f8640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc540a3e40 0x7fdc540a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400bbe0 tx=0x7fdc6400b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:14.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:14.819+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc60016030 con 0x7fdc6819a740 2024-04-22T13:27:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.006+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc68069250 con 0x7fdc6819a740 2024-04-22T13:27:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.008+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1893841309 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7fdc6009c960 con 0x7fdc6819a740 2024-04-22T13:27:15.004 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:15.004 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc540a3e40 msgr2=0x7fdc540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc540a3e40 0x7fdc540a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400bbe0 tx=0x7fdc6400b390 comp rx=0 tx=0).stop 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 msgr2=0x7fdc6819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 0x7fdc6819abc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000eaa0 tx=0x7fdc6000ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 shutdown_connections 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc6819a740 0x7fdc6819abc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdc540a3e40 0x7fdc540a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68109490 0x7fdc681a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 --2- 172.21.15.5:0/1893841309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc681043c0 0x7fdc681a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 >> 172.21.15.5:0/1893841309 conn(0x7fdc680ffc00 msgr2=0x7fdc68101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 shutdown_connections 2024-04-22T13:27:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.010+0000 7fdc7045c640 1 -- 172.21.15.5:0/1893841309 wait complete. 2024-04-22T13:27:15.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:15.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:15.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:15.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:27:15.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 -- 172.21.15.5:0/3032278897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c101a40 msgr2=0x7fb67c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 --2- 172.21.15.5:0/3032278897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c101a40 0x7fb67c105a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb670009900 tx=0x7fb67002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 -- 172.21.15.5:0/3032278897 shutdown_connections 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 --2- 172.21.15.5:0/3032278897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c101a40 0x7fb67c105a80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 --2- 172.21.15.5:0/3032278897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c106160 0x7fb67c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 --2- 172.21.15.5:0/3032278897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb67c101090 0x7fb67c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 -- 172.21.15.5:0/3032278897 >> 172.21.15.5:0/3032278897 conn(0x7fb67c078000 msgr2=0x7fb67c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 -- 172.21.15.5:0/3032278897 shutdown_connections 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.129+0000 7fb682411640 1 -- 172.21.15.5:0/3032278897 wait complete. 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 Processor -- start 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 -- start start 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb67c101090 0x7fb67c10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 0x7fb67c10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c110360 0x7fb67c10f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1121e0 con 0x7fb67c106160 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c112060 con 0x7fb67c101090 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb682411640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c112360 con 0x7fb67c110360 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 0x7fb67c10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 0x7fb67c10ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50956/0 (socket says 172.21.15.5:50956) 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.130+0000 7fb67b7fe640 1 -- 172.21.15.5:0/1341538997 learned_addr learned my addr 172.21.15.5:0/1341538997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67bfff640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb67c101090 0x7fb67c10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb6809ae640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c110360 0x7fb67c10f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67b7fe640 1 -- 172.21.15.5:0/1341538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c110360 msgr2=0x7fb67c10f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67b7fe640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c110360 0x7fb67c10f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67b7fe640 1 -- 172.21.15.5:0/1341538997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb67c101090 msgr2=0x7fb67c10e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67b7fe640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb67c101090 0x7fb67c10e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67b7fe640 1 -- 172.21.15.5:0/1341538997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c1ab3f0 con 0x7fb67c106160 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb6809ae640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c110360 0x7fb67c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb67b7fe640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 0x7fb67c10ec00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb670036040 tx=0x7fb6700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670043020 con 0x7fb67c106160 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c1ab680 con 0x7fb67c106160 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb67002cb70 con 0x7fb67c106160 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670003bf0 con 0x7fb67c106160 2024-04-22T13:27:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.131+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c1abbd0 con 0x7fb67c106160 2024-04-22T13:27:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.132+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67c102560 con 0x7fb67c106160 2024-04-22T13:27:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.133+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fb670046020 con 0x7fb67c106160 2024-04-22T13:27:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.134+0000 7fb6797fa640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6640a3e40 0x7fb6640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.134+0000 7fb67bfff640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6640a3e40 0x7fb6640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.134+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb6700f3070 con 0x7fb67c106160 2024-04-22T13:27:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.135+0000 7fb67bfff640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6640a3e40 0x7fb6640a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005fd0 tx=0x7fb66c005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.138+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb67004d020 con 0x7fb67c106160 2024-04-22T13:27:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.272 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:15.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:27:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: Updating smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:15.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:27:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.324+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb67c101a40 con 0x7fb67c106160 2024-04-22T13:27:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.324+0000 7fb6797fa640 1 -- 172.21.15.5:0/1341538997 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fb6700bf5a0 con 0x7fb67c106160 2024-04-22T13:27:15.320 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:27:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6640a3e40 msgr2=0x7fb6640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6640a3e40 0x7fb6640a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005fd0 tx=0x7fb66c005850 comp rx=0 tx=0).stop 2024-04-22T13:27:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 msgr2=0x7fb67c10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 0x7fb67c10ec00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb670036040 tx=0x7fb6700045e0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 shutdown_connections 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb67c106160 0x7fb67c10ec00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6640a3e40 0x7fb6640a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb67c110360 0x7fb67c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 --2- 172.21.15.5:0/1341538997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb67c101090 0x7fb67c10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 >> 172.21.15.5:0/1341538997 conn(0x7fb67c078000 msgr2=0x7fb67c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 shutdown_connections 2024-04-22T13:27:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.326+0000 7fb682411640 1 -- 172.21.15.5:0/1341538997 wait complete. 2024-04-22T13:27:15.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:27:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 -- 172.21.15.5:0/428490186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3784107250 msgr2=0x7f378410ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 --2- 172.21.15.5:0/428490186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3784107250 0x7f378410ee20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ca0 tx=0x7f377c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 -- 172.21.15.5:0/428490186 shutdown_connections 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 --2- 172.21.15.5:0/428490186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3784107250 0x7f378410ee20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 --2- 172.21.15.5:0/428490186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784102b30 0x7f3784106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 --2- 172.21.15.5:0/428490186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784102180 0x7f3784102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.442+0000 7f378c585640 1 -- 172.21.15.5:0/428490186 >> 172.21.15.5:0/428490186 conn(0x7f37840fda00 msgr2=0x7f37840ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 -- 172.21.15.5:0/428490186 shutdown_connections 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 -- 172.21.15.5:0/428490186 wait complete. 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 Processor -- start 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 -- start start 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784102180 0x7f37841a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784107250 0x7f37841a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 0x7f378419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37841121a0 con 0x7f378419a760 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3784112020 con 0x7f3784107250 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.443+0000 7f378c585640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3784112320 con 0x7f3784102180 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 0x7f378419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 0x7f378419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50968/0 (socket says 172.21.15.5:50968) 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 -- 172.21.15.5:0/1060613175 learned_addr learned my addr 172.21.15.5:0/1060613175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378a321640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784102180 0x7f37841a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f3789b20640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784107250 0x7f37841a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 -- 172.21.15.5:0/1060613175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784102180 msgr2=0x7f37841a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784102180 0x7f37841a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 -- 172.21.15.5:0/1060613175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784107250 msgr2=0x7f37841a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784107250 0x7f37841a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 -- 172.21.15.5:0/1060613175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419b360 con 0x7f378419a760 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f3789b20640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784107250 0x7f37841a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378ab22640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 0x7f378419abe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f377c005e30 tx=0x7f377c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c041020 con 0x7f378419a760 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419b5f0 con 0x7f378419a760 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377c032970 con 0x7f378419a760 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c039040 con 0x7f378419a760 2024-04-22T13:27:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.444+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37841a7780 con 0x7f378419a760 2024-04-22T13:27:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.445+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3784103650 con 0x7f378419a760 2024-04-22T13:27:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.447+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f377c044020 con 0x7f378419a760 2024-04-22T13:27:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.448+0000 7f377b7fe640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37700a3e40 0x7f37700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.448+0000 7f378a321640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37700a3e40 0x7f37700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.448+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f377c0f2c90 con 0x7f378419a760 2024-04-22T13:27:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.448+0000 7f378a321640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37700a3e40 0x7f37700a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3780005fd0 tx=0x7f378000b590 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.451+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f377c04b020 con 0x7f378419a760 2024-04-22T13:27:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.638+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3784102b30 con 0x7f378419a760 2024-04-22T13:27:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.638+0000 7f377b7fe640 1 -- 172.21.15.5:0/1060613175 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f377c045e40 con 0x7f378419a760 2024-04-22T13:27:15.634 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:27:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37700a3e40 msgr2=0x7f37700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37700a3e40 0x7f37700a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3780005fd0 tx=0x7f378000b590 comp rx=0 tx=0).stop 2024-04-22T13:27:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 msgr2=0x7f378419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 0x7f378419abe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f377c005e30 tx=0x7f377c034670 comp rx=0 tx=0).stop 2024-04-22T13:27:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 shutdown_connections 2024-04-22T13:27:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378419a760 0x7f378419abe0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37700a3e40 0x7f37700a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3784107250 0x7f37841a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 --2- 172.21.15.5:0/1060613175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784102180 0x7f37841a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 >> 172.21.15.5:0/1060613175 conn(0x7f37840fda00 msgr2=0x7f37840ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 shutdown_connections 2024-04-22T13:27:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.640+0000 7f378c585640 1 -- 172.21.15.5:0/1060613175 wait complete. 2024-04-22T13:27:15.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:27:15.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 -- 172.21.15.5:0/1656710887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 msgr2=0x7f910c103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 --2- 172.21.15.5:0/1656710887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c103fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9100009930 tx=0x7f910002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 -- 172.21.15.5:0/1656710887 shutdown_connections 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 --2- 172.21.15.5:0/1656710887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c103fb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 --2- 172.21.15.5:0/1656710887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c1044f0 0x7f910c10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 --2- 172.21.15.5:0/1656710887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c102270 0x7f910c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 -- 172.21.15.5:0/1656710887 >> 172.21.15.5:0/1656710887 conn(0x7f910c0fda10 msgr2=0x7f910c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 -- 172.21.15.5:0/1656710887 shutdown_connections 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.757+0000 7f9112385640 1 -- 172.21.15.5:0/1656710887 wait complete. 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 Processor -- start 2024-04-22T13:27:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 -- start start 2024-04-22T13:27:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c102270 0x7f910c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c1044f0 0x7f910c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f910c112290 con 0x7f910c103b50 2024-04-22T13:27:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f910c112110 con 0x7f910c102270 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f9112385640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f910c112410 con 0x7f910c1044f0 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f910b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f910b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50992/0 (socket says 172.21.15.5:50992) 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.758+0000 7f910b7fe640 1 -- 172.21.15.5:0/2671460512 learned_addr learned my addr 172.21.15.5:0/2671460512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910bfff640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c102270 0x7f910c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f9110922640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c1044f0 0x7f910c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910b7fe640 1 -- 172.21.15.5:0/2671460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c1044f0 msgr2=0x7f910c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910b7fe640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c1044f0 0x7f910c19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910b7fe640 1 -- 172.21.15.5:0/2671460512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c102270 msgr2=0x7f910c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910b7fe640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c102270 0x7f910c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910b7fe640 1 -- 172.21.15.5:0/2671460512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c19d740 con 0x7f910c103b50 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910bfff640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c102270 0x7f910c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f9110922640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c1044f0 0x7f910c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f910b7fe640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c198c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9100009230 tx=0x7f91000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100043020 con 0x7f910c103b50 2024-04-22T13:27:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c19d9d0 con 0x7f910c103b50 2024-04-22T13:27:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910002cb70 con 0x7f910c103b50 2024-04-22T13:27:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100003b50 con 0x7f910c103b50 2024-04-22T13:27:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.759+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c1a52c0 con 0x7f910c103b50 2024-04-22T13:27:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.760+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c069250 con 0x7f910c103b50 2024-04-22T13:27:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.761+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f9100046020 con 0x7f910c103b50 2024-04-22T13:27:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.762+0000 7f91097fa640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90f40a3e40 0x7f90f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.762+0000 7f910bfff640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90f40a3e40 0x7f90f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.762+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f91000f2c60 con 0x7f910c103b50 2024-04-22T13:27:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.762+0000 7f910bfff640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90f40a3e40 0x7f90f40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009b40 tx=0x7f90fc005e50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.766+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f910004d020 con 0x7f910c103b50 2024-04-22T13:27:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.956+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f910c10a540 con 0x7f910c103b50 2024-04-22T13:27:15.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.956+0000 7f91097fa640 1 -- 172.21.15.5:0/2671460512 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f91000bf1f0 con 0x7f910c103b50 2024-04-22T13:27:15.952 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:27:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90f40a3e40 msgr2=0x7f90f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90f40a3e40 0x7f90f40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009b40 tx=0x7f90fc005e50 comp rx=0 tx=0).stop 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 msgr2=0x7f910c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c198c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9100009230 tx=0x7f91000045e0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 shutdown_connections 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f910c103b50 0x7f910c198c90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90f40a3e40 0x7f90f40a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c1044f0 0x7f910c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 --2- 172.21.15.5:0/2671460512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c102270 0x7f910c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 >> 172.21.15.5:0/2671460512 conn(0x7f910c0fda10 msgr2=0x7f910c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 shutdown_connections 2024-04-22T13:27:15.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:15.958+0000 7f9112385640 1 -- 172.21.15.5:0/2671460512 wait complete. 2024-04-22T13:27:15.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:15 smithi005 ceph-mon[28006]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:15.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:15 smithi005 ceph-mon[28006]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:15.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1893841309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:15.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1341538997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:15.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1060613175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:15.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.071+0000 7f1f17b15640 1 -- 172.21.15.5:0/2488146192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f10107260 msgr2=0x7f1f1010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.071+0000 7f1f17b15640 1 --2- 172.21.15.5:0/2488146192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f10107260 0x7f1f1010ee30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009ca0 tx=0x7f1f0802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- 172.21.15.5:0/2488146192 shutdown_connections 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 --2- 172.21.15.5:0/2488146192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f10107260 0x7f1f1010ee30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 --2- 172.21.15.5:0/2488146192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f10102b40 0x7f1f10106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 --2- 172.21.15.5:0/2488146192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f10102190 0x7f1f10102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- 172.21.15.5:0/2488146192 >> 172.21.15.5:0/2488146192 conn(0x7f1f100793e0 msgr2=0x7f1f100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- 172.21.15.5:0/2488146192 shutdown_connections 2024-04-22T13:27:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- 172.21.15.5:0/2488146192 wait complete. 2024-04-22T13:27:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 Processor -- start 2024-04-22T13:27:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- start start 2024-04-22T13:27:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f10102190 0x7f1f101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f10107260 0x7f1f10198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 0x7f1f1019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101121d0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10112050 con 0x7f1f10107260 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.072+0000 7f1f17b15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10112350 con 0x7f1f10102190 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 0x7f1f1019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 0x7f1f1019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50998/0 (socket says 172.21.15.5:50998) 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 -- 172.21.15.5:0/1984090865 learned_addr learned my addr 172.21.15.5:0/1984090865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f150b0640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f10107260 0x7f1f10198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f158b1640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f10102190 0x7f1f101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 -- 172.21.15.5:0/1984090865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f10102190 msgr2=0x7f1f101986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f10102190 0x7f1f101986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 -- 172.21.15.5:0/1984090865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f10107260 msgr2=0x7f1f10198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f10107260 0x7f1f10198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 -- 172.21.15.5:0/1984090865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1019daa0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f160b2640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 0x7f1f1019d460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1f08005e30 tx=0x7f1f08034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08039040 con 0x7f1f1019cfb0 2024-04-22T13:27:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1019dd30 con 0x7f1f1019cfb0 2024-04-22T13:27:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f08044070 con 0x7f1f1019cfb0 2024-04-22T13:27:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08003e60 con 0x7f1f1019cfb0 2024-04-22T13:27:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.073+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f101a54f0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.074+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f10068570 con 0x7f1f1019cfb0 2024-04-22T13:27:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.075+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f1f080324c0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.076+0000 7f1f06ffd640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1efc0a3e40 0x7f1efc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.076+0000 7f1f158b1640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1efc0a3e40 0x7f1efc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.076+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1f080f2bc0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.076+0000 7f1f158b1640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1efc0a3e40 0x7f1efc0a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005d00 tx=0x7f1f0c00b410 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.080+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f080bf0f0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:15 smithi049 ceph-mon[31379]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:15 smithi049 ceph-mon[31379]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1893841309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1341538997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:16.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1060613175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:15 smithi107 ceph-mon[31859]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:15 smithi107 ceph-mon[31859]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1893841309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1341538997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1060613175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.307+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f10102b40 con 0x7f1f1019cfb0 2024-04-22T13:27:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.308+0000 7f1f06ffd640 1 -- 172.21.15.5:0/1984090865 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f1f080be9f0 con 0x7f1f1019cfb0 2024-04-22T13:27:16.304 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:16.304 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1efc0a3e40 msgr2=0x7f1efc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1efc0a3e40 0x7f1efc0a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005d00 tx=0x7f1f0c00b410 comp rx=0 tx=0).stop 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 msgr2=0x7f1f1019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 0x7f1f1019d460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1f08005e30 tx=0x7f1f08034760 comp rx=0 tx=0).stop 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 shutdown_connections 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f1019cfb0 0x7f1f1019d460 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1efc0a3e40 0x7f1efc0a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f10107260 0x7f1f10198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 --2- 172.21.15.5:0/1984090865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f10102190 0x7f1f101986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 >> 172.21.15.5:0/1984090865 conn(0x7f1f100793e0 msgr2=0x7f1f100ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 shutdown_connections 2024-04-22T13:27:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.310+0000 7f1f17b15640 1 -- 172.21.15.5:0/1984090865 wait complete. 2024-04-22T13:27:16.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 -- 172.21.15.5:0/2226760342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc06a640 msgr2=0x7facdc06aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 --2- 172.21.15.5:0/2226760342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc06a640 0x7facdc06aac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7facd0009930 tx=0x7facd002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 -- 172.21.15.5:0/2226760342 shutdown_connections 2024-04-22T13:27:16.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 --2- 172.21.15.5:0/2226760342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc06a640 0x7facdc06aac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 --2- 172.21.15.5:0/2226760342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 0x7facdc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 --2- 172.21.15.5:0/2226760342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facdc108060 0x7facdc108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 -- 172.21.15.5:0/2226760342 >> 172.21.15.5:0/2226760342 conn(0x7facdc077c40 msgr2=0x7facdc078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 -- 172.21.15.5:0/2226760342 shutdown_connections 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.423+0000 7face1fdb640 1 -- 172.21.15.5:0/2226760342 wait complete. 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 Processor -- start 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 -- start start 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facdc06a640 0x7facdc1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 0x7facdc1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 0x7facdc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1121a0 con 0x7facdc108060 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7facdc112020 con 0x7facdc06a640 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.424+0000 7face1fdb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7facdc112320 con 0x7facdc06b000 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 0x7facdc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 0x7facdc19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51008/0 (socket says 172.21.15.5:51008) 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facdc06a640 0x7facdc1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 -- 172.21.15.5:0/2436106418 learned_addr learned my addr 172.21.15.5:0/2436106418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdaffd640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 0x7facdc1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 -- 172.21.15.5:0/2436106418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 msgr2=0x7facdc1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 0x7facdc1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 -- 172.21.15.5:0/2436106418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facdc06a640 msgr2=0x7facdc1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facdc06a640 0x7facdc1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 -- 172.21.15.5:0/2436106418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc19af80 con 0x7facdc108060 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdaffd640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 0x7facdc1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facdbfff640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 0x7facdc19a6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7facd400eb10 tx=0x7facd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4004b20 con 0x7facdc108060 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc19b270 con 0x7facdc108060 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd4004cc0 con 0x7facdc108060 2024-04-22T13:27:16.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4010690 con 0x7facdc108060 2024-04-22T13:27:16.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.425+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc1a7580 con 0x7facdc108060 2024-04-22T13:27:16.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.426+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc069e50 con 0x7facdc108060 2024-04-22T13:27:16.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.427+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7facd4005020 con 0x7facdc108060 2024-04-22T13:27:16.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.428+0000 7facd8ff9640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7facc40a3e40 0x7facc40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.428+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7facd40d1070 con 0x7facdc108060 2024-04-22T13:27:16.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.428+0000 7facdb7fe640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7facc40a3e40 0x7facc40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.429+0000 7facdb7fe640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7facc40a3e40 0x7facc40a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faccc00bbe0 tx=0x7faccc00b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.433+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facd409d4f0 con 0x7facdc108060 2024-04-22T13:27:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.627+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facdc069250 con 0x7facdc108060 2024-04-22T13:27:16.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.629+0000 7facd8ff9640 1 -- 172.21.15.5:0/2436106418 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7facd409cdf0 con 0x7facdc108060 2024-04-22T13:27:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:16.626 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7facc40a3e40 msgr2=0x7facc40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7facc40a3e40 0x7facc40a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faccc00bbe0 tx=0x7faccc00b390 comp rx=0 tx=0).stop 2024-04-22T13:27:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 msgr2=0x7facdc19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 0x7facdc19a6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7facd400eb10 tx=0x7facd400c2d0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 shutdown_connections 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facdc108060 0x7facdc19a6c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7facc40a3e40 0x7facc40a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facdc06b000 0x7facdc1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 --2- 172.21.15.5:0/2436106418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facdc06a640 0x7facdc1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 >> 172.21.15.5:0/2436106418 conn(0x7facdc077c40 msgr2=0x7facdc10d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 shutdown_connections 2024-04-22T13:27:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.631+0000 7face1fdb640 1 -- 172.21.15.5:0/2436106418 wait complete. 2024-04-22T13:27:16.639 DEBUG:tasks.fwd_scrub:Yielding 2024-04-22T13:27:16.639 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T13:27:16.648 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:27:16.648 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:27:16.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-22T13:27:16.649 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-22T13:27:16.649 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:27:16.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:16.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:27:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 -- 172.21.15.5:0/2700981451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c10c590 msgr2=0x7f1a4c10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2700981451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c10c590 0x7f1a4c10e9e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009ca0 tx=0x7f1a4402c180 comp rx=0 tx=0).stop 2024-04-22T13:27:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 -- 172.21.15.5:0/2700981451 shutdown_connections 2024-04-22T13:27:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2700981451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c10c590 0x7f1a4c10e9e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2700981451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2700981451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 0x7f1a4c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 -- 172.21.15.5:0/2700981451 >> 172.21.15.5:0/2700981451 conn(0x7f1a4c077fa0 msgr2=0x7f1a4c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 -- 172.21.15.5:0/2700981451 shutdown_connections 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7f1a5340f640 1 -- 172.21.15.5:0/2700981451 wait complete. 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7fe6c994a640 1 -- 172.21.15.5:0/1808739299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4102b40 msgr2=0x7fe6c4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7fe6c994a640 1 --2- 172.21.15.5:0/1808739299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4102b40 0x7fe6c4106b80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009900 tx=0x7fe6b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.758+0000 7fe6c994a640 1 -- 172.21.15.5:0/1808739299 shutdown_connections 2024-04-22T13:27:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 --2- 172.21.15.5:0/1808739299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4102b40 0x7fe6c4106b80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 --2- 172.21.15.5:0/1808739299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c4107260 0x7fe6c410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 --2- 172.21.15.5:0/1808739299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c4102190 0x7fe6c4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 -- 172.21.15.5:0/1808739299 >> 172.21.15.5:0/1808739299 conn(0x7fe6c40fda10 msgr2=0x7fe6c40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 Processor -- start 2024-04-22T13:27:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 -- 172.21.15.5:0/1808739299 shutdown_connections 2024-04-22T13:27:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 -- 172.21.15.5:0/1808739299 wait complete. 2024-04-22T13:27:16.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 -- start start 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 0x7f1a4c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a4c10c590 0x7f1a4c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c1121a0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c112020 con 0x7f1a4c10c590 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a5340f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c112320 con 0x7f1a4c1031c0 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51032/0 (socket says 172.21.15.5:51032) 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 -- 172.21.15.5:0/2130371262 learned_addr learned my addr 172.21.15.5:0/2130371262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a511ab640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 0x7f1a4c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a519ac640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a4c10c590 0x7f1a4c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7fe6c994a640 1 Processor -- start 2024-04-22T13:27:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 -- 172.21.15.5:0/2130371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 msgr2=0x7f1a4c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 0x7f1a4c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 -- 172.21.15.5:0/2130371262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a4c10c590 msgr2=0x7f1a4c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a4c10c590 0x7f1a4c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a509aa640 1 -- 172.21.15.5:0/2130371262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c19d6b0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.759+0000 7f1a511ab640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 0x7f1a4c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7f1a509aa640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c198c00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800ea60 tx=0x7f1a4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 -- start start 2024-04-22T13:27:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800cde0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a480035f0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48010690 con 0x7f1a4c103ae0 2024-04-22T13:27:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c19d9a0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c4102190 0x7fe6c419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 0x7fe6c419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c419f3a0 0x7fe6c419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c1a5230 con 0x7f1a4c103ae0 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41121f0 con 0x7fe6c4107260 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4112070 con 0x7fe6c419f3a0 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c994a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4112370 con 0x7fe6c4102190 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 0x7fe6c419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 0x7fe6c419aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51044/0 (socket says 172.21.15.5:51044) 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 -- 172.21.15.5:0/3169803983 learned_addr learned my addr 172.21.15.5:0/3169803983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c419f3a0 0x7fe6c419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c4102190 0x7fe6c419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 -- 172.21.15.5:0/3169803983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c4102190 msgr2=0x7fe6c419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c4102190 0x7fe6c419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 -- 172.21.15.5:0/3169803983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c419f3a0 msgr2=0x7fe6c419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.760+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c419f3a0 0x7fe6c419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c27fc640 1 -- 172.21.15.5:0/3169803983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c419fe90 con 0x7fe6c4107260 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c419f3a0 0x7fe6c419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 0x7fe6c419aff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8000c00 tx=0x7fe6b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c8948640 1 -- 172.21.15.5:0/3169803983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8043020 con 0x7fe6c4107260 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c8948640 1 -- 172.21.15.5:0/3169803983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b802cb70 con 0x7fe6c4107260 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c8948640 1 -- 172.21.15.5:0/3169803983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8003b50 con 0x7fe6c4107260 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c41a0120 con 0x7fe6c4107260 2024-04-22T13:27:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c050530 con 0x7f1a4c103ae0 2024-04-22T13:27:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.761+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c41a78f0 con 0x7fe6c4107260 2024-04-22T13:27:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.763+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f1a480037a0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7f1a427fc640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a380a3e30 0x7f1a380a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1a480d0e30 con 0x7f1a4c103ae0 2024-04-22T13:27:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7f1a511ab640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a380a3e30 0x7f1a380a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 Processor -- start 2024-04-22T13:27:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 -- start start 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2ccf8e0 0x5568f2ccfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 0x5568f2cd8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd8e10 0x5568f2cdb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568f2c33f00 con 0x5568f2cd0290 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5568f2bd1590 con 0x5568f2ccf8e0 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568f2c2fec0 con 0x5568f2cd8e10 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 0x5568f2cd8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 0x5568f2cd8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51048/0 (socket says 172.21.15.5:51048) 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.764+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/426530057 learned_addr learned my addr 172.21.15.5:0/426530057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2ccf8e0 0x5568f2ccfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/426530057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd8e10 msgr2=0x5568f2cdb260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd8e10 0x5568f2cdb260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/426530057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2ccf8e0 msgr2=0x5568f2ccfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2ccf8e0 0x5568f2ccfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/426530057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f2a0e7e0 con 0x5568f2cd0290 2024-04-22T13:27:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 0x5568f2cd8840 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009d20 tx=0x7fe6b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8558633346521a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7f1a511ab640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a380a3e30 0x7f1a380a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c30 tx=0x7f1a3c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.765+0000 7fe6c8948640 1 -- 172.21.15.5:0/3169803983 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fe6b8046020 con 0x7fe6c4107260 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.766+0000 7fe6c8948640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6ac0a3ec0 0x7fe6ac0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.766+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6ac0a3ec0 0x7fe6ac0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.766+0000 7fe6c8948640 1 -- 172.21.15.5:0/3169803983 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe6b80f2960 con 0x7fe6c4107260 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.766+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6ac0a3ec0 0x7fe6ac0a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005fd0 tx=0x7fe6bc0096a0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6a9ffb640 1 -- 172.21.15.5:0/426530057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b401e070 con 0x5568f2cd0290 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6a9ffb640 1 -- 172.21.15.5:0/426530057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b4019ae0 con 0x5568f2cd0290 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6a9ffb640 1 -- 172.21.15.5:0/426530057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400ea60 con 0x5568f2cd0290 2024-04-22T13:27:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a4809d2b0 con 0x7f1a4c103ae0 2024-04-22T13:27:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6cc60e740 1 -- 172.21.15.5:0/426530057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 msgr2=0x5568f2cd8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 0x5568f2cd8840 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009d20 tx=0x7fe6b4011810 comp rx=0 tx=0).stop 2024-04-22T13:27:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6cc60e740 1 -- 172.21.15.5:0/426530057 shutdown_connections 2024-04-22T13:27:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd0290 0x5568f2cd8840 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.769+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd8e10 0x5568f2cdb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.770+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/426530057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2ccf8e0 0x5568f2ccfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.770+0000 7fe6cc60e740 1 -- 172.21.15.5:0/426530057 >> 172.21.15.5:0/426530057 conn(0x5568f2cc9fb0 msgr2=0x5568f2cca390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.770+0000 7fe6cc60e740 1 -- 172.21.15.5:0/426530057 shutdown_connections 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.770+0000 7fe6cc60e740 1 -- 172.21.15.5:0/426530057 wait complete. 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.770+0000 7fe6cc60e740 1 -- start start 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.770+0000 7fe6cc60e740 1 Processor -- start 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cd5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cd6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cd68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568f2cdbd80 con 0x5568f2ccf8e0 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5568f2bed580 con 0x5568f2cd8e10 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568f2cdbf00 con 0x5568f2cd0290 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6cc60e740 10 client.0 _ll_register_callbacks cb 0x5568f2bf3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cd5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cd5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51056/0 (socket says 172.21.15.5:51056) 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/743544600 learned_addr learned my addr 172.21.15.5:0/743544600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cd68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cd6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/743544600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 msgr2=0x5568f2cd6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cd6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 msgr2=0x5568f2cd68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cd68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/743544600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f2a6ef70 con 0x5568f2ccf8e0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cd5e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400e740 tx=0x7fe6b4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1275d7c6237b05d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.771+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/743544600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4027070 con 0x5568f2ccf8e0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/743544600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe6b4019840 con 0x5568f2ccf8e0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b4019ae0 con 0x5568f2ccf8e0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/743544600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6b4033050 con 0x5568f2ccf8e0 2024-04-22T13:27:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 10 client.15558 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568f2906170 con 0x5568f2ccf8e0 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/743544600 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b4019ae0 con 0x5568f2ccf8e0 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/743544600 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe6b400ec00 con 0x5568f2ccf8e0 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 20 client.15558 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2ce9260 0x5568f2ceb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 4 client.15558 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568f2c41920 con 0x5568f2ce9260 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.772+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2ce9260 0x5568f2ceb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.773+0000 7fe6a2ffd640 10 client.15558 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.773+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2ce9260 0x5568f2ceb650 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be79880b3fd8d92 server_cookie=fad9c16f70f1cc2d in_seq=0 out_seq=0 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/743544600 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568f2c41920 con 0x5568f2ce9260 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6a2ffd640 10 client.15558 handle_command_reply: tid=0 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 client.15558 shutdown 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 msgr2=0x5568f2cd5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cd5e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400e740 tx=0x7fe6b4009f60 comp rx=0 tx=0).stop 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 shutdown_connections 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2ce9260 0x5568f2ceb650 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cd68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cd6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6a2ffd640 0 client.15558 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/743544600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cd5e50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 >> 172.21.15.5:0/743544600 conn(0x5568f2cc9fb0 msgr2=0x5568f2ccbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 shutdown_connections 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 1 -- 172.21.15.5:0/743544600 wait complete. 2024-04-22T13:27:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.774+0000 7fe6cc60e740 20 client.15558 trim_cache size 0 max 16384 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 Processor -- start 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 -- start start 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2ccf8e0 0x5568f2ce4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd0290 0x5568f2cd5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 0x5568f2cf0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41afdb0 con 0x5568f2cd8e10 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b18f0 con 0x5568f2cd0290 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b0480 con 0x5568f2ccf8e0 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 0x5568f2cf0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 0x5568f2cf0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51058/0 (socket says 172.21.15.5:51058) 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 -- 172.21.15.5:0/582720746 learned_addr learned my addr 172.21.15.5:0/582720746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2ccf8e0 0x5568f2ce4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 -- 172.21.15.5:0/582720746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2ccf8e0 msgr2=0x5568f2ce4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2ccf8e0 0x5568f2ce4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 -- 172.21.15.5:0/582720746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd0290 msgr2=0x5568f2cd5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd0290 0x5568f2cd5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.810+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd0290 0x5568f2cd5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.811+0000 7fe6c37fe640 1 -- 172.21.15.5:0/582720746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f2cd6db0 con 0x5568f2cd8e10 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.811+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 0x5568f2cf0dc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc010d70 tx=0x7fe6bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07e03ac80521e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.811+0000 7fe6a97fa640 1 -- 172.21.15.5:0/582720746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc010da0 con 0x5568f2cd8e10 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.811+0000 7fe6a97fa640 1 -- 172.21.15.5:0/582720746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6bc003e20 con 0x5568f2cd8e10 2024-04-22T13:27:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.811+0000 7fe6a97fa640 1 -- 172.21.15.5:0/582720746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc01c650 con 0x5568f2cd8e10 2024-04-22T13:27:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- 172.21.15.5:0/582720746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 msgr2=0x5568f2cf0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 0x5568f2cf0dc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc010d70 tx=0x7fe6bc019000 comp rx=0 tx=0).stop 2024-04-22T13:27:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- 172.21.15.5:0/582720746 shutdown_connections 2024-04-22T13:27:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2cd8e10 0x5568f2cf0dc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd0290 0x5568f2cd5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/582720746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2ccf8e0 0x5568f2ce4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- 172.21.15.5:0/582720746 >> 172.21.15.5:0/582720746 conn(0x5568f2cc9fb0 msgr2=0x5568f2cd66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- 172.21.15.5:0/582720746 shutdown_connections 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- 172.21.15.5:0/582720746 wait complete. 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- start start 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 Processor -- start 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cdcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cdd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cdd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b1220 con 0x5568f2ccf8e0 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b0b50 con 0x5568f2cd8e10 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.812+0000 7fe6cc60e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41add90 con 0x5568f2cd0290 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6cc60e740 10 client.0 _ll_register_callbacks cb 0x7fe6b401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cdcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cdcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51074/0 (socket says 172.21.15.5:51074) 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/1116368250 learned_addr learned my addr 172.21.15.5:0/1116368250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cdd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/1116368250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 msgr2=0x5568f2cdd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cdd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cdd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 msgr2=0x5568f2cdd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cdd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/1116368250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f2d4e880 con 0x5568f2ccf8e0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c37fe640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cdcdb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400e740 tx=0x7fe6b40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81605465b47a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/1116368250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4027040 con 0x5568f2ccf8e0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/1116368250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe6b4004440 con 0x5568f2ccf8e0 2024-04-22T13:27:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc003e20 con 0x5568f2ccf8e0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/1116368250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6b401e040 con 0x5568f2ccf8e0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6cc60e740 10 client.15564 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.813+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568f2d4fb80 con 0x5568f2ccf8e0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/1116368250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6bc003e20 con 0x5568f2ccf8e0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/1116368250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe6b40052e0 con 0x5568f2ccf8e0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6cc60e740 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6cc60e740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6cc60e740 20 client.15564 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2d502d0 0x5568f2d526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6cc60e740 4 client.15564 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5568f2c3b790 con 0x5568f2d502d0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.814+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2d502d0 0x5568f2d526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.815+0000 7fe6c27fc640 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2d502d0 0x5568f2d526f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438124fc99433254 server_cookie=98bddf51d02bebec in_seq=0 out_seq=0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.815+0000 7fe6a2ffd640 10 client.15564 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6a2ffd640 1 -- 172.21.15.5:0/1116368250 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5568f2c3b790 con 0x5568f2d502d0 2024-04-22T13:27:16.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6a2ffd640 10 client.15564 handle_command_reply: tid=0 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 client.15564 shutdown 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 msgr2=0x5568f2cdcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cdcdb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400e740 tx=0x7fe6b40096a0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 shutdown_connections 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568f2d502d0 0x5568f2d526f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568f2cd8e10 0x5568f2cdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568f2cd0290 0x5568f2cdd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6a2ffd640 0 client.15564 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 --2- 172.21.15.5:0/1116368250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568f2ccf8e0 0x5568f2cdcdb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 >> 172.21.15.5:0/1116368250 conn(0x5568f2cc9fb0 msgr2=0x5568f2ce3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 shutdown_connections 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 1 -- 172.21.15.5:0/1116368250 wait complete. 2024-04-22T13:27:16.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.816+0000 7fe6cc60e740 20 client.15564 trim_cache size 0 max 16384 2024-04-22T13:27:16.817 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:27:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "581b4b18-30a5-43e9-97cb-afdbbd40413b", 2024-04-22T13:27:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:27:16.817 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6ac0a3ec0 msgr2=0x7fe6ac0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6ac0a3ec0 0x7fe6ac0a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005fd0 tx=0x7fe6bc0096a0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 msgr2=0x7fe6c419aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 0x7fe6c419aff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8000c00 tx=0x7fe6b80045e0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 shutdown_connections 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4107260 0x7fe6c419aff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6ac0a3ec0 0x7fe6ac0a6380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c419f3a0 0x7fe6c419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 --2- 172.21.15.5:0/3169803983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c4102190 0x7fe6c419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 >> 172.21.15.5:0/3169803983 conn(0x7fe6c40fda10 msgr2=0x7fe6c40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 shutdown_connections 2024-04-22T13:27:16.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.817+0000 7fe6c994a640 1 -- 172.21.15.5:0/3169803983 wait complete. 2024-04-22T13:27:16.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:27:16.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 -- 172.21.15.5:0/3263482918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb038101050 msgr2=0x7fb038101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/3263482918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb038101050 0x7fb038101430 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb028009900 tx=0x7fb02802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 -- 172.21.15.5:0/3263482918 shutdown_connections 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/3263482918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb038106120 0x7fb03810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/3263482918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb038101a00 0x7fb038105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/3263482918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb038101050 0x7fb038101430 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 -- 172.21.15.5:0/3263482918 >> 172.21.15.5:0/3263482918 conn(0x7fb038077fa0 msgr2=0x7fb0380783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 -- 172.21.15.5:0/3263482918 shutdown_connections 2024-04-22T13:27:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 -- 172.21.15.5:0/3263482918 wait complete. 2024-04-22T13:27:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.934+0000 7fb03dd7e640 1 Processor -- start 2024-04-22T13:27:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 -- start start 2024-04-22T13:27:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb038101050 0x7fb0381986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb038106120 0x7fb038198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 0x7fb03819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb038111060 con 0x7fb03819cf90 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb038110ee0 con 0x7fb038106120 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb03dd7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381111e0 con 0x7fb038101050 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 0x7fb03819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 0x7fb03819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51102/0 (socket says 172.21.15.5:51102) 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 -- 172.21.15.5:0/857484579 learned_addr learned my addr 172.21.15.5:0/857484579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb0377fe640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb038101050 0x7fb0381986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb036ffd640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb038106120 0x7fb038198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 -- 172.21.15.5:0/857484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb038101050 msgr2=0x7fb0381986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb038101050 0x7fb0381986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 -- 172.21.15.5:0/857484579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb038106120 msgr2=0x7fb038198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb038106120 0x7fb038198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 -- 172.21.15.5:0/857484579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819da80 con 0x7fb03819cf90 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.935+0000 7fb037fff640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 0x7fb03819d440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb03000cce0 tx=0x7fb03000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.936+0000 7fb034ff9640 1 -- 172.21.15.5:0/857484579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03000f450 con 0x7fb03819cf90 2024-04-22T13:27:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.936+0000 7fb034ff9640 1 -- 172.21.15.5:0/857484579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03000faa0 con 0x7fb03819cf90 2024-04-22T13:27:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.936+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819dd70 con 0x7fb03819cf90 2024-04-22T13:27:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.936+0000 7fb034ff9640 1 -- 172.21.15.5:0/857484579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030018b70 con 0x7fb03819cf90 2024-04-22T13:27:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.936+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a5560 con 0x7fb03819cf90 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb034ff9640 1 -- 172.21.15.5:0/857484579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fb030018d10 con 0x7fb03819cf90 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 Processor -- start 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 -- start start 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b53f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53f910 0x55ea1b54c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b54f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea1b4a41e0 con 0x55ea1b53ef60 2024-04-22T13:27:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea1b441390 con 0x55ea1b54ce90 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb04096e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea1b4a01a0 con 0x55ea1b53f910 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb034ff9640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0200a3ec0 0x7fb0200a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b53f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.938+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b53f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51104/0 (socket says 172.21.15.5:51104) 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 -- 172.21.15.5:0/4235752549 learned_addr learned my addr 172.21.15.5:0/4235752549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb0377fe640 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53f910 0x55ea1b54c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb037fff640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0200a3ec0 0x7fb0200a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b54f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb034ff9640 1 -- 172.21.15.5:0/857484579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb0300d0ee0 con 0x7fb03819cf90 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 -- 172.21.15.5:0/4235752549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53f910 msgr2=0x55ea1b54c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53f910 0x55ea1b54c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 -- 172.21.15.5:0/4235752549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 msgr2=0x55ea1b54f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b54f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 -- 172.21.15.5:0/4235752549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1b2807d0 con 0x55ea1b53ef60 2024-04-22T13:27:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb036ffd640 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b53f340 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009e70 tx=0x7fb02c009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2b649ae5081af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb037fff640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0200a3ec0 0x7fb0200a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0300ce540 tx=0x7fb0300920f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb01dffb640 1 -- 172.21.15.5:0/4235752549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c012e60 con 0x55ea1b53ef60 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb01dffb640 1 -- 172.21.15.5:0/4235752549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02c0054d0 con 0x55ea1b53ef60 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.939+0000 7fb01dffb640 1 -- 172.21.15.5:0/4235752549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c010400 con 0x55ea1b53ef60 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 -- 172.21.15.5:0/4235752549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 msgr2=0x55ea1b53f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b53f340 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009e70 tx=0x7fb02c009760 comp rx=0 tx=0).stop 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 -- 172.21.15.5:0/4235752549 shutdown_connections 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b54f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53f910 0x55ea1b54c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 --2- 172.21.15.5:0/4235752549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b53f340 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 -- 172.21.15.5:0/4235752549 >> 172.21.15.5:0/4235752549 conn(0x55ea1b539cb0 msgr2=0x55ea1b53a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 -- 172.21.15.5:0/4235752549 shutdown_connections 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 -- 172.21.15.5:0/4235752549 wait complete. 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.940+0000 7fb04096e740 1 -- start start 2024-04-22T13:27:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 Processor -- start 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b549e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b54a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b54ce90 0x55ea1b54a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea1b54fc50 con 0x55ea1b53ef60 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ea1b45d2f0 con 0x55ea1b53f910 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea1b54fdd0 con 0x55ea1b54ce90 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb04096e740 10 client.0 _ll_register_callbacks cb 0x55ea1b46a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b549e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b549e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51110/0 (socket says 172.21.15.5:51110) 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 -- 172.21.15.5:0/936073332 learned_addr learned my addr 172.21.15.5:0/936073332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb0377fe640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b54a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b54ce90 0x55ea1b54a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 -- 172.21.15.5:0/936073332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b54ce90 msgr2=0x55ea1b54a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b54ce90 0x55ea1b54a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 -- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 msgr2=0x55ea1b54a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b54a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 -- 172.21.15.5:0/936073332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1b1b63f0 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb036ffd640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b549e40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009b10 tx=0x7fb02c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8dff48a8af3ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.941+0000 7fb016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb016ffd640 1 -- 172.21.15.5:0/936073332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c021020 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb016ffd640 1 -- 172.21.15.5:0/936073332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb02c0046c0 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02c0054d0 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb016ffd640 1 -- 172.21.15.5:0/936073332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb02c005000 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea1b1760f0 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb016ffd640 1 -- 172.21.15.5:0/936073332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02c0054d0 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb016ffd640 1 -- 172.21.15.5:0/936073332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb02c00f040 con 0x55ea1b53ef60 2024-04-22T13:27:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 20 client.15576 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b55cdb0 0x55ea1b55f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 4 client.15576 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.942+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1b4b1c00 con 0x55ea1b55cdb0 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.943+0000 7fb0377fe640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b55cdb0 0x55ea1b55f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.943+0000 7fb016ffd640 10 client.15576 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.943+0000 7fb0377fe640 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b55cdb0 0x55ea1b55f1a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c69206261f718f3 server_cookie=f487d6aa322119c0 in_seq=0 out_seq=0 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb016ffd640 1 -- 172.21.15.5:0/936073332 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea1b4b1c00 con 0x55ea1b55cdb0 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb016ffd640 10 client.15576 handle_command_reply: tid=0 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 client.15576 shutdown 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 msgr2=0x55ea1b549e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b549e40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009b10 tx=0x7fb02c005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 shutdown_connections 2024-04-22T13:27:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b55cdb0 0x55ea1b55f1a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b54ce90 0x55ea1b54a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b54a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 --2- 172.21.15.5:0/936073332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53ef60 0x55ea1b549e40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb016ffd640 0 client.15576 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 >> 172.21.15.5:0/936073332 conn(0x55ea1b539cb0 msgr2=0x55ea1b53ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 shutdown_connections 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 1 -- 172.21.15.5:0/936073332 wait complete. 2024-04-22T13:27:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.944+0000 7fb04096e740 20 client.15576 trim_cache size 0 max 16384 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 Processor -- start 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 -- start start 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b558b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b538bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 0x55ea1b5493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381ab860 con 0x55ea1b54ce90 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381ad3a0 con 0x55ea1b53f910 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb04096e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381abf30 con 0x55ea1b53ef60 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 0x55ea1b5493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 0x55ea1b5493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51126/0 (socket says 172.21.15.5:51126) 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 -- 172.21.15.5:0/2389779217 learned_addr learned my addr 172.21.15.5:0/2389779217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b558b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb0377fe640 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b538bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 -- 172.21.15.5:0/2389779217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 msgr2=0x55ea1b558b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b558b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 -- 172.21.15.5:0/2389779217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 msgr2=0x55ea1b538bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b538bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb036ffd640 1 -- 172.21.15.5:0/2389779217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1b55f700 con 0x55ea1b54ce90 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.962+0000 7fb0377fe640 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b538bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb036ffd640 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 0x55ea1b5493a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55ea1b54b3e0 tx=0x7fb02c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87832c2dcb886432 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb01d7fa640 1 -- 172.21.15.5:0/2389779217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c021070 con 0x55ea1b54ce90 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb01d7fa640 1 -- 172.21.15.5:0/2389779217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02c009bd0 con 0x55ea1b54ce90 2024-04-22T13:27:16.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb01d7fa640 1 -- 172.21.15.5:0/2389779217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0120d0 con 0x55ea1b54ce90 2024-04-22T13:27:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 -- 172.21.15.5:0/2389779217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 msgr2=0x55ea1b5493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 0x55ea1b5493a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55ea1b54b3e0 tx=0x7fb02c004ad0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 -- 172.21.15.5:0/2389779217 shutdown_connections 2024-04-22T13:27:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b54ce90 0x55ea1b5493a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b53f910 0x55ea1b538bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 --2- 172.21.15.5:0/2389779217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b558b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 -- 172.21.15.5:0/2389779217 >> 172.21.15.5:0/2389779217 conn(0x55ea1b539cb0 msgr2=0x55ea1b5384e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.963+0000 7fb04096e740 1 -- 172.21.15.5:0/2389779217 shutdown_connections 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 -- 172.21.15.5:0/2389779217 wait complete. 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 -- start start 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 Processor -- start 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b555250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 0x55ea1b546670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b546bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381accd0 con 0x55ea1b53f910 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381ac600 con 0x55ea1b54ce90 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381a9840 con 0x55ea1b53ef60 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb04096e740 10 client.0 _ll_register_callbacks cb 0x7fb02c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb0377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 0x55ea1b546670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb0377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 0x55ea1b546670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51142/0 (socket says 172.21.15.5:51142) 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb0377fe640 1 -- 172.21.15.5:0/3608257881 learned_addr learned my addr 172.21.15.5:0/3608257881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.964+0000 7fb036ffd640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b546bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb036ffd640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b555250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb0377fe640 1 -- 172.21.15.5:0/3608257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 msgr2=0x55ea1b555250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb0377fe640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b555250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb0377fe640 1 -- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 msgr2=0x55ea1b546bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb0377fe640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b546bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb0377fe640 1 -- 172.21.15.5:0/3608257881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1b5470f0 con 0x55ea1b53f910 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb0377fe640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 0x55ea1b546670 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb02802c170 tx=0x7fb028005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccfa216f3fbcdd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb016ffd640 1 -- 172.21.15.5:0/3608257881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028004480 con 0x55ea1b53f910 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb016ffd640 1 -- 172.21.15.5:0/3608257881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb02804abd0 con 0x55ea1b53f910 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02c009bd0 con 0x55ea1b53f910 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb016ffd640 1 -- 172.21.15.5:0/3608257881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb02804add0 con 0x55ea1b53f910 2024-04-22T13:27:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb04096e740 10 client.15582 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea1b5c2dc0 con 0x55ea1b53f910 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb016ffd640 1 -- 172.21.15.5:0/3608257881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02c009bd0 con 0x55ea1b53f910 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.965+0000 7fb016ffd640 1 -- 172.21.15.5:0/3608257881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb028005200 con 0x55ea1b53f910 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 20 client.15582 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b5c3540 0x55ea1b5c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 4 client.15582 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea1b4aba70 con 0x55ea1b5c3540 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb036ffd640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b5c3540 0x55ea1b5c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb016ffd640 10 client.15582 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb036ffd640 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b5c3540 0x55ea1b5c5960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ee33b5715230ae server_cookie=7bf8009bbe01c880 in_seq=0 out_seq=0 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb016ffd640 1 -- 172.21.15.5:0/3608257881 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ea1b4aba70 con 0x55ea1b5c3540 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb016ffd640 10 client.15582 handle_command_reply: tid=0 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.966+0000 7fb04096e740 1 client.15582 shutdown 2024-04-22T13:27:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 msgr2=0x55ea1b546670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 0x55ea1b546670 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb02802c170 tx=0x7fb028005be0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 shutdown_connections 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ea1b5c3540 0x55ea1b5c5960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1b53f910 0x55ea1b546670 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb016ffd640 0 client.15582 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ea1b54ce90 0x55ea1b546bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 --2- 172.21.15.5:0/3608257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1b53ef60 0x55ea1b555250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 >> 172.21.15.5:0/3608257881 conn(0x55ea1b539cb0 msgr2=0x55ea1b54a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 shutdown_connections 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 1 -- 172.21.15.5:0/3608257881 wait complete. 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb04096e740 20 client.15582 trim_cache size 0 max 16384 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:16.968 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: "581b4b18-30a5-43e9-97cb-afdbbd40413b": { 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "581b4b18-30a5-43e9-97cb-afdbbd40413b", 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:16.969 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0200a3ec0 msgr2=0x7fb0200a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0200a3ec0 0x7fb0200a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0300ce540 tx=0x7fb0300920f0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 msgr2=0x7fb03819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 0x7fb03819d440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb03000cce0 tx=0x7fb03000a620 comp rx=0 tx=0).stop 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 shutdown_connections 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb03819cf90 0x7fb03819d440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0200a3ec0 0x7fb0200a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb038106120 0x7fb038198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 --2- 172.21.15.5:0/857484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb038101050 0x7fb0381986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.967+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 >> 172.21.15.5:0/857484579 conn(0x7fb038077fa0 msgr2=0x7fb038069640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.968+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 shutdown_connections 2024-04-22T13:27:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:16.968+0000 7fb03dd7e640 1 -- 172.21.15.5:0/857484579 wait complete. 2024-04-22T13:27:16.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:16 smithi005 ceph-mon[28006]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:16.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2671460512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:16.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1984090865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:16.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2436106418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:16.974 INFO:tasks.cephfs.filesystem:scrub status for tag:581b4b18-30a5-43e9-97cb-afdbbd40413b - {'path': '/', 'tag': '581b4b18-30a5-43e9-97cb-afdbbd40413b', 'options': 'recursive,force'} 2024-04-22T13:27:17.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.005+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1a4c1076e0 con 0x7f1a380a3e30 2024-04-22T13:27:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.073+0000 7f1a427fc640 1 -- 172.21.15.5:0/2130371262 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1a4c1076e0 con 0x7f1a380a3e30 2024-04-22T13:27:17.069 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2024-04-22T13:27:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a380a3e30 msgr2=0x7f1a380a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a380a3e30 0x7f1a380a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c30 tx=0x7f1a3c005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 msgr2=0x7f1a4c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c198c00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800ea60 tx=0x7f1a4800ef20 comp rx=0 tx=0).stop 2024-04-22T13:27:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 shutdown_connections 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c103ae0 0x7f1a4c198c00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a380a3e30 0x7f1a380a62f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a4c10c590 0x7f1a4c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 --2- 172.21.15.5:0/2130371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a4c1031c0 0x7f1a4c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 >> 172.21.15.5:0/2130371262 conn(0x7f1a4c077fa0 msgr2=0x7f1a4c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 shutdown_connections 2024-04-22T13:27:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.075+0000 7f1a5340f640 1 -- 172.21.15.5:0/2130371262 wait complete. 2024-04-22T13:27:17.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-22T13:27:17.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.231+0000 7f105ac3f640 1 -- 172.21.15.5:0/3421939268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 msgr2=0x7f1054101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.231+0000 7f105ac3f640 1 --2- 172.21.15.5:0/3421939268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f1054101500 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1044009900 tx=0x7f104402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.231+0000 7f105ac3f640 1 -- 172.21.15.5:0/3421939268 shutdown_connections 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 --2- 172.21.15.5:0/3421939268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10541061a0 0x7f105410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 --2- 172.21.15.5:0/3421939268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054101ad0 0x7f1054105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 --2- 172.21.15.5:0/3421939268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f1054101500 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- 172.21.15.5:0/3421939268 >> 172.21.15.5:0/3421939268 conn(0x7f1054077fa0 msgr2=0x7f10540783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- 172.21.15.5:0/3421939268 shutdown_connections 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- 172.21.15.5:0/3421939268 wait complete. 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 Processor -- start 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- start start 2024-04-22T13:27:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f10541a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10541061a0 0x7f10541a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105419a7c0 0x7f105419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1054111100 con 0x7f1054101120 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1054110f80 con 0x7f10541061a0 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.232+0000 7f105ac3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1054111280 con 0x7f105419a7c0 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f10541a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f10541a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51164/0 (socket says 172.21.15.5:51164) 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 -- 172.21.15.5:0/2328159138 learned_addr learned my addr 172.21.15.5:0/2328159138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f1053fff640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10541061a0 0x7f10541a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10591dc640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105419a7c0 0x7f105419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 -- 172.21.15.5:0/2328159138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105419a7c0 msgr2=0x7f105419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105419a7c0 0x7f105419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 -- 172.21.15.5:0/2328159138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10541061a0 msgr2=0x7f10541a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10541061a0 0x7f10541a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 -- 172.21.15.5:0/2328159138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105419b350 con 0x7f1054101120 2024-04-22T13:27:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f10589db640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f10541a0650 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1044005940 tx=0x7f104402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044043020 con 0x7f1054101120 2024-04-22T13:27:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f104402cbe0 con 0x7f1054101120 2024-04-22T13:27:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105419b5e0 con 0x7f1054101120 2024-04-22T13:27:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044003bf0 con 0x7f1054101120 2024-04-22T13:27:17.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.233+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10541a77c0 con 0x7f1054101120 2024-04-22T13:27:17.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.235+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10541025a0 con 0x7f1054101120 2024-04-22T13:27:17.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.236+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f1044046020 con 0x7f1054101120 2024-04-22T13:27:17.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.237+0000 7f1051ffb640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f103c0a3e30 0x7f103c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.237+0000 7f1053fff640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f103c0a3e30 0x7f103c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.237+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f10440f30a0 con 0x7f1054101120 2024-04-22T13:27:17.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.237+0000 7f1053fff640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f103c0a3e30 0x7f103c0a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f105406b670 tx=0x7f104800b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:17.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.241+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f104404d020 con 0x7f1054101120 2024-04-22T13:27:17.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:16 smithi049 ceph-mon[31379]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2671460512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1984090865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:17.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2436106418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:16 smithi107 ceph-mon[31859]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:27:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2671460512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1984090865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:17.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2436106418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:17.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.451+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f1054101c80 con 0x7f103c0a3e30 2024-04-22T13:27:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.515+0000 7f1051ffb640 1 -- 172.21.15.5:0/2328159138 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f1054101c80 con 0x7f103c0a3e30 2024-04-22T13:27:17.511 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f103c0a3e30 msgr2=0x7f103c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f103c0a3e30 0x7f103c0a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f105406b670 tx=0x7f104800b390 comp rx=0 tx=0).stop 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 msgr2=0x7f10541a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f10541a0650 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1044005940 tx=0x7f104402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 shutdown_connections 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f103c0a3e30 0x7f103c0a62f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105419a7c0 0x7f105419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10541061a0 0x7f10541a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 --2- 172.21.15.5:0/2328159138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1054101120 0x7f10541a0650 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 >> 172.21.15.5:0/2328159138 conn(0x7f1054077fa0 msgr2=0x7f10540fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 shutdown_connections 2024-04-22T13:27:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.518+0000 7f105ac3f640 1 -- 172.21.15.5:0/2328159138 wait complete. 2024-04-22T13:27:17.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-22T13:27:17.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 -- 172.21.15.5:0/5352586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 msgr2=0x7fe37c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 --2- 172.21.15.5:0/5352586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c10bfe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe364009900 tx=0x7fe36402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 -- 172.21.15.5:0/5352586 shutdown_connections 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 --2- 172.21.15.5:0/5352586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c10bfe0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 --2- 172.21.15.5:0/5352586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe37c10c5b0 0x7fe37c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 --2- 172.21.15.5:0/5352586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe37c1031e0 0x7fe37c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 -- 172.21.15.5:0/5352586 >> 172.21.15.5:0/5352586 conn(0x7fe37c077fa0 msgr2=0x7fe37c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 -- 172.21.15.5:0/5352586 shutdown_connections 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.631+0000 7fe381426640 1 -- 172.21.15.5:0/5352586 wait complete. 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 Processor -- start 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 -- start start 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe37c1031e0 0x7fe37c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe37c10c5b0 0x7fe37c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1121e0 con 0x7fe37c103b00 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c112060 con 0x7fe37c10c5b0 2024-04-22T13:27:17.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe381426640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c112360 con 0x7fe37c1031e0 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c19e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51184/0 (socket says 172.21.15.5:51184) 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe373fff640 1 -- 172.21.15.5:0/3849138676 learned_addr learned my addr 172.21.15.5:0/3849138676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe37affd640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe37c1031e0 0x7fe37c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.632+0000 7fe37b7fe640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe37c10c5b0 0x7fe37c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe373fff640 1 -- 172.21.15.5:0/3849138676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe37c1031e0 msgr2=0x7fe37c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe373fff640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe37c1031e0 0x7fe37c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe373fff640 1 -- 172.21.15.5:0/3849138676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe37c10c5b0 msgr2=0x7fe37c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe373fff640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe37c10c5b0 0x7fe37c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe373fff640 1 -- 172.21.15.5:0/3849138676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c198d40 con 0x7fe37c103b00 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe373fff640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c19e950 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe364005940 tx=0x7fe36402d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364043020 con 0x7fe37c103b00 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36402dc90 con 0x7fe37c103b00 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c198fd0 con 0x7fe37c103b00 2024-04-22T13:27:17.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364003ab0 con 0x7fe37c103b00 2024-04-22T13:27:17.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.633+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a52d0 con 0x7fe37c103b00 2024-04-22T13:27:17.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.634+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c06adc0 con 0x7fe37c103b00 2024-04-22T13:27:17.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.635+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fe364046020 con 0x7fe37c103b00 2024-04-22T13:27:17.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.636+0000 7fe378ff9640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe3680a3eb0 0x7fe3680a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.636+0000 7fe37affd640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe3680a3eb0 0x7fe3680a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.636+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe3640f2ab0 con 0x7fe37c103b00 2024-04-22T13:27:17.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.636+0000 7fe37affd640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe3680a3eb0 0x7fe3680a6370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005fd0 tx=0x7fe36c005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.641+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe36404d020 con 0x7fe37c103b00 2024-04-22T13:27:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.854+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe37c062390 con 0x7fe3680a3eb0 2024-04-22T13:27:17.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.862+0000 7fe378ff9640 1 -- 172.21.15.5:0/3849138676 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fe37c062390 con 0x7fe3680a3eb0 2024-04-22T13:27:17.858 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-22T00:00:00", "created": "2024-04-22T13:27:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-22T13:27:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe3680a3eb0 msgr2=0x7fe3680a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe3680a3eb0 0x7fe3680a6370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005fd0 tx=0x7fe36c005850 comp rx=0 tx=0).stop 2024-04-22T13:27:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 msgr2=0x7fe37c19e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c19e950 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe364005940 tx=0x7fe36402d890 comp rx=0 tx=0).stop 2024-04-22T13:27:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 shutdown_connections 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe37c103b00 0x7fe37c19e950 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe3680a3eb0 0x7fe3680a6370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe37c10c5b0 0x7fe37c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 --2- 172.21.15.5:0/3849138676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe37c1031e0 0x7fe37c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 >> 172.21.15.5:0/3849138676 conn(0x7fe37c077fa0 msgr2=0x7fe37c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 shutdown_connections 2024-04-22T13:27:17.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.864+0000 7fe381426640 1 -- 172.21.15.5:0/3849138676 wait complete. 2024-04-22T13:27:17.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 -- 172.21.15.5:0/982812966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 msgr2=0x7f02901035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 --2- 172.21.15.5:0/982812966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f02901035c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0280009900 tx=0x7f028002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 -- 172.21.15.5:0/982812966 shutdown_connections 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 --2- 172.21.15.5:0/982812966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f029010c5b0 0x7f029010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 --2- 172.21.15.5:0/982812966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0290103b00 0x7f029010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 --2- 172.21.15.5:0/982812966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f02901035c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 -- 172.21.15.5:0/982812966 >> 172.21.15.5:0/982812966 conn(0x7f0290077fa0 msgr2=0x7f02900783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 -- 172.21.15.5:0/982812966 shutdown_connections 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.979+0000 7f02968c1640 1 -- 172.21.15.5:0/982812966 wait complete. 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 Processor -- start 2024-04-22T13:27:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 -- start start 2024-04-22T13:27:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f0290100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0290103b00 0x7f0290101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f029010c5b0 0x7f02901018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02901121e0 con 0x7f02901031e0 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0290112060 con 0x7f0290103b00 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f02968c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0290112360 con 0x7f029010c5b0 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.980+0000 7f028ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f0290100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f0290100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51202/0 (socket says 172.21.15.5:51202) 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 -- 172.21.15.5:0/2556318331 learned_addr learned my addr 172.21.15.5:0/2556318331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028f7fe640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0290103b00 0x7f0290101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f0294e5e640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f029010c5b0 0x7f02901018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 -- 172.21.15.5:0/2556318331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f029010c5b0 msgr2=0x7f02901018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f029010c5b0 0x7f02901018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 -- 172.21.15.5:0/2556318331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0290103b00 msgr2=0x7f0290101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0290103b00 0x7f0290101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 -- 172.21.15.5:0/2556318331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02901a6be0 con 0x7f02901031e0 2024-04-22T13:27:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f0294e5e640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f029010c5b0 0x7f02901018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028ffff640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f0290100e30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0280005940 tx=0x7f028002d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280043020 con 0x7f02901031e0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901a6e70 con 0x7f02901031e0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028002dd10 con 0x7f02901031e0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280003b80 con 0x7f02901031e0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.981+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a7330 con 0x7f02901031e0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.982+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029006adc0 con 0x7f02901031e0 2024-04-22T13:27:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.983+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f0280046020 con 0x7f02901031e0 2024-04-22T13:27:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.984+0000 7f028d7fa640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f02780a3e60 0x7f02780a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.984+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f02800f2b40 con 0x7f02901031e0 2024-04-22T13:27:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.984+0000 7f028f7fe640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f02780a3e60 0x7f02780a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.984+0000 7f028f7fe640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f02780a3e60 0x7f02780a6320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0290102870 tx=0x7f028400a250 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:17.989+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f028004d020 con 0x7f02901031e0 2024-04-22T13:27:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.198+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0290062390 con 0x7f02780a3e60 2024-04-22T13:27:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.205+0000 7f028d7fa640 1 -- 172.21.15.5:0/2556318331 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0290062390 con 0x7f02780a3e60 2024-04-22T13:27:18.202 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.207+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f02780a3e60 msgr2=0x7f02780a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.207+0000 7f02968c1640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f02780a3e60 0x7f02780a6320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0290102870 tx=0x7f028400a250 comp rx=0 tx=0).stop 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.207+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 msgr2=0x7f0290100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.207+0000 7f02968c1640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f0290100e30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0280005940 tx=0x7f028002d910 comp rx=0 tx=0).stop 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 shutdown_connections 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f02780a3e60 0x7f02780a6320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f029010c5b0 0x7f02901018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0290103b00 0x7f0290101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 --2- 172.21.15.5:0/2556318331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02901031e0 0x7f0290100e30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 >> 172.21.15.5:0/2556318331 conn(0x7f0290077fa0 msgr2=0x7f02900fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 shutdown_connections 2024-04-22T13:27:18.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.208+0000 7f02968c1640 1 -- 172.21.15.5:0/2556318331 wait complete. 2024-04-22T13:27:18.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:17 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:27:18.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:17 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:18.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:17 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:27:18.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:17 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:18.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:17 smithi005 ceph-mon[28006]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:18.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-22T13:27:18.241 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-22T13:27:18.251 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-22T13:27:18.260 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:27:18.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-22T13:27:18.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:17 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:27:18.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:17 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:17 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:27:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:17 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:18.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:17 smithi049 ceph-mon[31379]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:18.283 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-22T13:27:18.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:18.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:17 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:27:18.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:17 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:18.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:17 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:27:18.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:17 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:18.295 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:17 smithi107 ceph-mon[31859]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 -- 172.21.15.5:0/587141084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 msgr2=0x7f1ea81061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 --2- 172.21.15.5:0/587141084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81061b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009900 tx=0x7f1ea402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 -- 172.21.15.5:0/587141084 shutdown_connections 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 --2- 172.21.15.5:0/587141084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ea8101a40 0x7f1ea810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 --2- 172.21.15.5:0/587141084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ea81010a0 0x7f1ea8101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 --2- 172.21.15.5:0/587141084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81061b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 -- 172.21.15.5:0/587141084 >> 172.21.15.5:0/587141084 conn(0x7f1ea8077c40 msgr2=0x7f1ea8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 -- 172.21.15.5:0/587141084 shutdown_connections 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.401+0000 7f1eaf493640 1 -- 172.21.15.5:0/587141084 wait complete. 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 Processor -- start 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 -- start start 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ea81010a0 0x7f1ea819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ea8101a40 0x7f1ea819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8111170 con 0x7f1ea8105dd0 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8110ff0 con 0x7f1ea81010a0 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaf493640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81112f0 con 0x7f1ea8101a40 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51226/0 (socket says 172.21.15.5:51226) 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 -- 172.21.15.5:0/3469649043 learned_addr learned my addr 172.21.15.5:0/3469649043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1ead22f640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ea81010a0 0x7f1ea819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eaca2e640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ea8101a40 0x7f1ea819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 -- 172.21.15.5:0/3469649043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ea8101a40 msgr2=0x7f1ea819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ea8101a40 0x7f1ea819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 -- 172.21.15.5:0/3469649043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ea81010a0 msgr2=0x7f1ea819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ea81010a0 0x7f1ea819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.402+0000 7f1eada30640 1 -- 172.21.15.5:0/3469649043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea8198d20 con 0x7f1ea8105dd0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.403+0000 7f1eada30640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81985a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000eaa0 tx=0x7f1ea000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.403+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000cdb0 con 0x7f1ea8105dd0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.403+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea0005020 con 0x7f1ea8105dd0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.403+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00116d0 con 0x7f1ea8105dd0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.403+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea8199010 con 0x7f1ea8105dd0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.403+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea81a52b0 con 0x7f1ea8105dd0 2024-04-22T13:27:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.404+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea8102540 con 0x7f1ea8105dd0 2024-04-22T13:27:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.405+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f1ea00051d0 con 0x7f1ea8105dd0 2024-04-22T13:27:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.406+0000 7f1e9e7fc640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e940a3e40 0x7f1e940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.406+0000 7f1ead22f640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e940a3e40 0x7f1e940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.406+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1ea00d0be0 con 0x7f1ea8105dd0 2024-04-22T13:27:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.406+0000 7f1ead22f640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e940a3e40 0x7f1e940a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006010 tx=0x7f1ea4005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.411+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ea0016030 con 0x7f1ea8105dd0 2024-04-22T13:27:18.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.597+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ea810a370 con 0x7f1ea8105dd0 2024-04-22T13:27:18.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.598+0000 7f1e9e7fc640 1 -- 172.21.15.5:0/3469649043 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7f1ea009d060 con 0x7f1ea8105dd0 2024-04-22T13:27:18.594 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:18.595 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:18.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e940a3e40 msgr2=0x7f1e940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e940a3e40 0x7f1e940a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006010 tx=0x7f1ea4005e40 comp rx=0 tx=0).stop 2024-04-22T13:27:18.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 msgr2=0x7f1ea81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81985a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000eaa0 tx=0x7f1ea000ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 shutdown_connections 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea8105dd0 0x7f1ea81985a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e940a3e40 0x7f1e940a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ea8101a40 0x7f1ea819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 --2- 172.21.15.5:0/3469649043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ea81010a0 0x7f1ea819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 >> 172.21.15.5:0/3469649043 conn(0x7f1ea8077c40 msgr2=0x7f1ea806a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 shutdown_connections 2024-04-22T13:27:18.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.600+0000 7f1eaf493640 1 -- 172.21.15.5:0/3469649043 wait complete. 2024-04-22T13:27:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:27:18.608 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 -- 172.21.15.5:0/991479737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4400101a40 msgr2=0x7f4400105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 --2- 172.21.15.5:0/991479737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4400101a40 0x7f4400105a80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009900 tx=0x7f43e802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 -- 172.21.15.5:0/991479737 shutdown_connections 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 --2- 172.21.15.5:0/991479737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4400101a40 0x7f4400105a80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 --2- 172.21.15.5:0/991479737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 0x7f440010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 --2- 172.21.15.5:0/991479737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400101090 0x7f4400101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 -- 172.21.15.5:0/991479737 >> 172.21.15.5:0/991479737 conn(0x7f4400078000 msgr2=0x7f4400078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 -- 172.21.15.5:0/991479737 shutdown_connections 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 -- 172.21.15.5:0/991479737 wait complete. 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 Processor -- start 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.721+0000 7f44051e6640 1 -- start start 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400101090 0x7f4400102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 0x7f4400103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 0x7f44001a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44001110a0 con 0x7f44001035a0 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4400110f20 con 0x7f4400106160 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111220 con 0x7f4400101090 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 0x7f44001a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 0x7f44001a6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51240/0 (socket says 172.21.15.5:51240) 2024-04-22T13:27:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 -- 172.21.15.5:0/416856352 learned_addr learned my addr 172.21.15.5:0/416856352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43fed9d640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400101090 0x7f4400102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43f7fff640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 0x7f4400103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 -- 172.21.15.5:0/416856352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400101090 msgr2=0x7f4400102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400101090 0x7f4400102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 -- 172.21.15.5:0/416856352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 msgr2=0x7f4400103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 0x7f4400103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 -- 172.21.15.5:0/416856352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44001a7080 con 0x7f44001035a0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43f7fff640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 0x7f4400103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43ff59e640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 0x7f44001a6980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f43f800cab0 tx=0x7f43f800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f800adc0 con 0x7f44001035a0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f800f900 con 0x7f44001035a0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001a7370 con 0x7f44001035a0 2024-04-22T13:27:18.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f80189a0 con 0x7f44001035a0 2024-04-22T13:27:18.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.722+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44001a7870 con 0x7f44001035a0 2024-04-22T13:27:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.724+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4400068e00 con 0x7f44001035a0 2024-04-22T13:27:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.725+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f43f800f450 con 0x7f44001035a0 2024-04-22T13:27:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.725+0000 7f43fcd99640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f43ec0a3e70 0x7f43ec0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.726+0000 7f43fed9d640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f43ec0a3e70 0x7f43ec0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.726+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f43f80d0dd0 con 0x7f44001035a0 2024-04-22T13:27:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.726+0000 7f43fed9d640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f43ec0a3e70 0x7f43ec0a6330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005fd0 tx=0x7f43f0005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.730+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43f8014030 con 0x7f44001035a0 2024-04-22T13:27:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.918+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4400101a40 con 0x7f44001035a0 2024-04-22T13:27:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.918+0000 7f43fcd99640 1 -- 172.21.15.5:0/416856352 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f43f809d1f0 con 0x7f44001035a0 2024-04-22T13:27:18.914 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:27:18.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f43ec0a3e70 msgr2=0x7f43ec0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f43ec0a3e70 0x7f43ec0a6330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005fd0 tx=0x7f43f0005850 comp rx=0 tx=0).stop 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 msgr2=0x7f44001a6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 0x7f44001a6980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f43f800cab0 tx=0x7f43f800cf70 comp rx=0 tx=0).stop 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 shutdown_connections 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44001035a0 0x7f44001a6980 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f43ec0a3e70 0x7f43ec0a6330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400106160 0x7f4400103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 --2- 172.21.15.5:0/416856352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400101090 0x7f4400102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 >> 172.21.15.5:0/416856352 conn(0x7f4400078000 msgr2=0x7f4400069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 shutdown_connections 2024-04-22T13:27:18.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:18.920+0000 7f44051e6640 1 -- 172.21.15.5:0/416856352 wait complete. 2024-04-22T13:27:18.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:27:19.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 -- 172.21.15.5:0/829870624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f390810e910 msgr2=0x7f3908110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 --2- 172.21.15.5:0/829870624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f390810e910 0x7f3908110d00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3900009ca0 tx=0x7f390002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 -- 172.21.15.5:0/829870624 shutdown_connections 2024-04-22T13:27:19.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 --2- 172.21.15.5:0/829870624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f390810e910 0x7f3908110d00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 --2- 172.21.15.5:0/829870624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3908074e20 0x7f390810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 --2- 172.21.15.5:0/829870624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908074500 0x7f39080748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 -- 172.21.15.5:0/829870624 >> 172.21.15.5:0/829870624 conn(0x7f39080fda10 msgr2=0x7f39080ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 -- 172.21.15.5:0/829870624 shutdown_connections 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 -- 172.21.15.5:0/829870624 wait complete. 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.037+0000 7f390dd67640 1 Processor -- start 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 -- start start 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 0x7f39081a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908074e20 0x7f39081a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810e910 0x7f390819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39081144a0 con 0x7f3908074500 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114320 con 0x7f390810e910 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f390dd67640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114620 con 0x7f3908074e20 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 0x7f39081a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 0x7f39081a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46630/0 (socket says 172.21.15.5:46630) 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 -- 172.21.15.5:0/1538677682 learned_addr learned my addr 172.21.15.5:0/1538677682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:19.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f3906ffd640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908074e20 0x7f39081a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f3907fff640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810e910 0x7f390819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 -- 172.21.15.5:0/1538677682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908074e20 msgr2=0x7f39081a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908074e20 0x7f39081a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 -- 172.21.15.5:0/1538677682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810e910 msgr2=0x7f390819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810e910 0x7f390819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 -- 172.21.15.5:0/1538677682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390819b050 con 0x7f3908074500 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f3907fff640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810e910 0x7f390819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.038+0000 7f39077fe640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 0x7f39081a06c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f38f800d770 tx=0x7f38f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.039+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8017070 con 0x7f3908074500 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.039+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f80054d0 con 0x7f3908074500 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.039+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390819b340 con 0x7f3908074500 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.039+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004c30 con 0x7f3908074500 2024-04-22T13:27:19.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.039+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a7510 con 0x7f3908074500 2024-04-22T13:27:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.040+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39080759b0 con 0x7f3908074500 2024-04-22T13:27:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.041+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f38f8004e10 con 0x7f3908074500 2024-04-22T13:27:19.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.041+0000 7f3904ff9640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e40 0x7f38f00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.041+0000 7f3906ffd640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e40 0x7f38f00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.042+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f38f80d0550 con 0x7f3908074500 2024-04-22T13:27:19.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.042+0000 7f3906ffd640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e40 0x7f38f00a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f390810fcc0 tx=0x7f38fc00b410 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.046+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f809c9d0 con 0x7f3908074500 2024-04-22T13:27:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.233+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3908109960 con 0x7f3908074500 2024-04-22T13:27:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.233+0000 7f3904ff9640 1 -- 172.21.15.5:0/1538677682 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f38f809c2d0 con 0x7f3908074500 2024-04-22T13:27:19.229 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e40 msgr2=0x7f38f00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e40 0x7f38f00a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f390810fcc0 tx=0x7f38fc00b410 comp rx=0 tx=0).stop 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 msgr2=0x7f39081a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 0x7f39081a06c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f38f800d770 tx=0x7f38f800dc30 comp rx=0 tx=0).stop 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 shutdown_connections 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e40 0x7f38f00a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810e910 0x7f390819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908074e20 0x7f39081a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 --2- 172.21.15.5:0/1538677682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908074500 0x7f39081a06c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 >> 172.21.15.5:0/1538677682 conn(0x7f39080fda10 msgr2=0x7f39080fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 shutdown_connections 2024-04-22T13:27:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.235+0000 7f390dd67640 1 -- 172.21.15.5:0/1538677682 wait complete. 2024-04-22T13:27:19.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:18 smithi005 ceph-mon[28006]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:19.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:18 smithi005 ceph-mon[28006]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-04-22T13:27:19.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:18 smithi005 ceph-mon[28006]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:19.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:18 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3469649043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:19.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:27:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:18 smithi049 ceph-mon[31379]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:18 smithi049 ceph-mon[31379]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-04-22T13:27:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:18 smithi049 ceph-mon[31379]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:19.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:18 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3469649043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:18 smithi107 ceph-mon[31859]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:18 smithi107 ceph-mon[31859]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-04-22T13:27:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:18 smithi107 ceph-mon[31859]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:19.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:18 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3469649043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 -- 172.21.15.5:0/1411150341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a1010c500 msgr2=0x7f7a1010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 --2- 172.21.15.5:0/1411150341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a1010c500 0x7f7a1010e9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009ca0 tx=0x7f7a0802c180 comp rx=0 tx=0).stop 2024-04-22T13:27:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 -- 172.21.15.5:0/1411150341 shutdown_connections 2024-04-22T13:27:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 --2- 172.21.15.5:0/1411150341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a1010c500 0x7f7a1010e9a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 --2- 172.21.15.5:0/1411150341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a10103ae0 0x7f7a1010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 --2- 172.21.15.5:0/1411150341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a101031c0 0x7f7a101035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 -- 172.21.15.5:0/1411150341 >> 172.21.15.5:0/1411150341 conn(0x7f7a10078000 msgr2=0x7f7a10078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 -- 172.21.15.5:0/1411150341 shutdown_connections 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.353+0000 7f7a177ce640 1 -- 172.21.15.5:0/1411150341 wait complete. 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 Processor -- start 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 -- start start 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a101031c0 0x7f7a101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 0x7f7a10198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a1010c500 0x7f7a1019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10112190 con 0x7f7a10103ae0 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10112010 con 0x7f7a1010c500 2024-04-22T13:27:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a177ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10112310 con 0x7f7a101031c0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a14d69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 0x7f7a10198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a14d69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 0x7f7a10198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46654/0 (socket says 172.21.15.5:46654) 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a14d69640 1 -- 172.21.15.5:0/2741237339 learned_addr learned my addr 172.21.15.5:0/2741237339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a1556a640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a101031c0 0x7f7a101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a15d6b640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a1010c500 0x7f7a1019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a14d69640 1 -- 172.21.15.5:0/2741237339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a101031c0 msgr2=0x7f7a101986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a14d69640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a101031c0 0x7f7a101986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.354+0000 7f7a14d69640 1 -- 172.21.15.5:0/2741237339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a1010c500 msgr2=0x7f7a1019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a14d69640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a1010c500 0x7f7a1019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a14d69640 1 -- 172.21.15.5:0/2741237339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1019d690 con 0x7f7a10103ae0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a15d6b640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a1010c500 0x7f7a1019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a14d69640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 0x7f7a10198c30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d770 tx=0x7f7a0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00017070 con 0x7f7a10103ae0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1019d980 con 0x7f7a10103ae0 2024-04-22T13:27:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a000054d0 con 0x7f7a10103ae0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00004c30 con 0x7f7a10103ae0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.355+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a5210 con 0x7f7a10103ae0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.356+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10069250 con 0x7f7a10103ae0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.357+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f7a00004df0 con 0x7f7a10103ae0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.358+0000 7f7a067fc640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f79fc0a3e40 0x7f79fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.358+0000 7f7a1556a640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f79fc0a3e40 0x7f79fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.358+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7a000d0520 con 0x7f7a10103ae0 2024-04-22T13:27:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.358+0000 7f7a1556a640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f79fc0a3e40 0x7f79fc0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009bd0 tx=0x7f7a0c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.363+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a0009c940 con 0x7f7a10103ae0 2024-04-22T13:27:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.549+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a101070f0 con 0x7f7a10103ae0 2024-04-22T13:27:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.550+0000 7f7a067fc640 1 -- 172.21.15.5:0/2741237339 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f7a0009c240 con 0x7f7a10103ae0 2024-04-22T13:27:19.546 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f79fc0a3e40 msgr2=0x7f79fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f79fc0a3e40 0x7f79fc0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009bd0 tx=0x7f7a0c005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 msgr2=0x7f7a10198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 0x7f7a10198c30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d770 tx=0x7f7a0000dc30 comp rx=0 tx=0).stop 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 shutdown_connections 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10103ae0 0x7f7a10198c30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f79fc0a3e40 0x7f79fc0a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a1010c500 0x7f7a1019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 --2- 172.21.15.5:0/2741237339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a101031c0 0x7f7a101986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 >> 172.21.15.5:0/2741237339 conn(0x7f7a10078000 msgr2=0x7f7a100fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 shutdown_connections 2024-04-22T13:27:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.552+0000 7f7a177ce640 1 -- 172.21.15.5:0/2741237339 wait complete. 2024-04-22T13:27:19.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:19.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 -- 172.21.15.5:0/82360408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 msgr2=0x7f557c102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 --2- 172.21.15.5:0/82360408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c102550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f556c009900 tx=0x7f556c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:19.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 -- 172.21.15.5:0/82360408 shutdown_connections 2024-04-22T13:27:19.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 --2- 172.21.15.5:0/82360408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c107240 0x7f557c10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 --2- 172.21.15.5:0/82360408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f557c102b20 0x7f557c106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 --2- 172.21.15.5:0/82360408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c102550 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 -- 172.21.15.5:0/82360408 >> 172.21.15.5:0/82360408 conn(0x7f557c0fd9d0 msgr2=0x7f557c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 -- 172.21.15.5:0/82360408 shutdown_connections 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.665+0000 7f558205c640 1 -- 172.21.15.5:0/82360408 wait complete. 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 Processor -- start 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 -- start start 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c107240 0x7f557c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f557c074cd0 0x7f557c1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f557c1121b0 con 0x7f557c102170 2024-04-22T13:27:19.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f557c112030 con 0x7f557c074cd0 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f558205c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f557c112330 con 0x7f557c107240 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46674/0 (socket says 172.21.15.5:46674) 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 -- 172.21.15.5:0/1862617958 learned_addr learned my addr 172.21.15.5:0/1862617958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557affd640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c107240 0x7f557c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 -- 172.21.15.5:0/1862617958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c107240 msgr2=0x7f557c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c107240 0x7f557c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 -- 172.21.15.5:0/1862617958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f557c074cd0 msgr2=0x7f557c1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f557c074cd0 0x7f557c1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.666+0000 7f557b7fe640 1 -- 172.21.15.5:0/1862617958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1a7000 con 0x7f557c102170 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.667+0000 7f557b7fe640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c074250 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f556c005940 tx=0x7f556c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.667+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c043020 con 0x7f557c102170 2024-04-22T13:27:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.667+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1a7290 con 0x7f557c102170 2024-04-22T13:27:19.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.667+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c02cbe0 con 0x7f557c102170 2024-04-22T13:27:19.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.667+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c003bf0 con 0x7f557c102170 2024-04-22T13:27:19.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.667+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1a7830 con 0x7f557c102170 2024-04-22T13:27:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.668+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c069250 con 0x7f557c102170 2024-04-22T13:27:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.669+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f556c046020 con 0x7f557c102170 2024-04-22T13:27:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.670+0000 7f5578ff9640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f55640a3e70 0x7f55640a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.670+0000 7f557affd640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f55640a3e70 0x7f55640a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.670+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f556c0f2c70 con 0x7f557c102170 2024-04-22T13:27:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.670+0000 7f557affd640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f55640a3e70 0x7f55640a6330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5570005fd0 tx=0x7f5570009470 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.674+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f556c04d020 con 0x7f557c102170 2024-04-22T13:27:19.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.864+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f557c102b20 con 0x7f557c102170 2024-04-22T13:27:19.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.865+0000 7f5578ff9640 1 -- 172.21.15.5:0/1862617958 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7f556c03c350 con 0x7f557c102170 2024-04-22T13:27:19.861 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:19.862 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f55640a3e70 msgr2=0x7f55640a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f55640a3e70 0x7f55640a6330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5570005fd0 tx=0x7f5570009470 comp rx=0 tx=0).stop 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 msgr2=0x7f557c074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c074250 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f556c005940 tx=0x7f556c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 shutdown_connections 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f55640a3e70 0x7f55640a6330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f557c074cd0 0x7f557c1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c107240 0x7f557c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 --2- 172.21.15.5:0/1862617958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557c102170 0x7f557c074250 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.867+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 >> 172.21.15.5:0/1862617958 conn(0x7f557c0fd9d0 msgr2=0x7f557c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.868+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 shutdown_connections 2024-04-22T13:27:19.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.868+0000 7f558205c640 1 -- 172.21.15.5:0/1862617958 wait complete. 2024-04-22T13:27:19.875 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:27:19.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:27:19.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 -- 172.21.15.5:0/3918019049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c103b70 msgr2=0x7fb17c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 --2- 172.21.15.5:0/3918019049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c103b70 0x7fb17c10bfc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb164009900 tx=0x7fb16402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 -- 172.21.15.5:0/3918019049 shutdown_connections 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 --2- 172.21.15.5:0/3918019049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c103b70 0x7fb17c10bfc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 --2- 172.21.15.5:0/3918019049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb17c10c6d0 0x7fb17c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 --2- 172.21.15.5:0/3918019049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c1031c0 0x7fb17c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 -- 172.21.15.5:0/3918019049 >> 172.21.15.5:0/3918019049 conn(0x7fb17c077f70 msgr2=0x7fb17c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 -- 172.21.15.5:0/3918019049 shutdown_connections 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.984+0000 7fb1819a4640 1 -- 172.21.15.5:0/3918019049 wait complete. 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 Processor -- start 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 -- start start 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c1031c0 0x7fb17c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 0x7fb17c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb17c198690 0x7fb17c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112330 con 0x7fb17c10c6d0 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1121b0 con 0x7fb17c198690 2024-04-22T13:27:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb1819a4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1124b0 con 0x7fb17c1031c0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb173fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 0x7fb17c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb173fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 0x7fb17c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46690/0 (socket says 172.21.15.5:46690) 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.985+0000 7fb173fff640 1 -- 172.21.15.5:0/1854355113 learned_addr learned my addr 172.21.15.5:0/1854355113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb17b7fe640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb17c198690 0x7fb17c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb17affd640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c1031c0 0x7fb17c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb173fff640 1 -- 172.21.15.5:0/1854355113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c1031c0 msgr2=0x7fb17c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb173fff640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c1031c0 0x7fb17c19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb173fff640 1 -- 172.21.15.5:0/1854355113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb17c198690 msgr2=0x7fb17c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb173fff640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb17c198690 0x7fb17c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb173fff640 1 -- 172.21.15.5:0/1854355113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c199290 con 0x7fb17c10c6d0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb173fff640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 0x7fb17c19eb10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb164036040 tx=0x7fb16402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164043020 con 0x7fb17c10c6d0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16402cbe0 con 0x7fb17c10c6d0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c199520 con 0x7fb17c10c6d0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164003c30 con 0x7fb17c10c6d0 2024-04-22T13:27:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.986+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c1a56c0 con 0x7fb17c10c6d0 2024-04-22T13:27:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.987+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17c06ae40 con 0x7fb17c10c6d0 2024-04-22T13:27:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.988+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fb164046020 con 0x7fb17c10c6d0 2024-04-22T13:27:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.989+0000 7fb1809a2640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb15c0a3e40 0x7fb15c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.989+0000 7fb17affd640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb15c0a3e40 0x7fb15c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:19.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.989+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb1640f30c0 con 0x7fb17c10c6d0 2024-04-22T13:27:19.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.989+0000 7fb17affd640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb15c0a3e40 0x7fb15c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00ba40 tx=0x7fb16c005cc0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:19.992+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb16404d020 con 0x7fb17c10c6d0 2024-04-22T13:27:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.177+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb17c107680 con 0x7fb17c10c6d0 2024-04-22T13:27:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.178+0000 7fb1809a2640 1 -- 172.21.15.5:0/1854355113 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fb16403e450 con 0x7fb17c10c6d0 2024-04-22T13:27:20.174 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb15c0a3e40 msgr2=0x7fb15c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb15c0a3e40 0x7fb15c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00ba40 tx=0x7fb16c005cc0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 msgr2=0x7fb17c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 0x7fb17c19eb10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb164036040 tx=0x7fb16402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 shutdown_connections 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb17c10c6d0 0x7fb17c19eb10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb15c0a3e40 0x7fb15c0a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb17c198690 0x7fb17c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 --2- 172.21.15.5:0/1854355113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c1031c0 0x7fb17c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 >> 172.21.15.5:0/1854355113 conn(0x7fb17c077f70 msgr2=0x7fb17c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 shutdown_connections 2024-04-22T13:27:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.180+0000 7fb1819a4640 1 -- 172.21.15.5:0/1854355113 wait complete. 2024-04-22T13:27:20.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:19 smithi005 ceph-mon[28006]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:20.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:19 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/416856352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:19 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1538677682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:19 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2741237339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:19 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1862617958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:20.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:27:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:19 smithi049 ceph-mon[31379]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:19 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/416856352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:19 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1538677682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:19 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2741237339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:19 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1862617958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:19 smithi107 ceph-mon[31859]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:27:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:19 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/416856352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:19 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1538677682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:19 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2741237339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:19 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1862617958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/3291740691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac102180 msgr2=0x7f7cac102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/3291740691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac102180 0x7f7cac102560 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009900 tx=0x7f7c9c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/3291740691 shutdown_connections 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/3291740691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac107250 0x7f7cac10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/3291740691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cac102b30 0x7f7cac106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/3291740691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac102180 0x7f7cac102560 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/3291740691 >> 172.21.15.5:0/3291740691 conn(0x7f7cac0fda00 msgr2=0x7f7cac0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/3291740691 shutdown_connections 2024-04-22T13:27:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.296+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/3291740691 wait complete. 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 Processor -- start 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 -- start start 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac102180 0x7f7cac19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 0x7f7cac19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cac1985f0 0x7f7cac198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac112190 con 0x7f7cac107250 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac112010 con 0x7f7cac1985f0 2024-04-22T13:27:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.297+0000 7f7cb0fb6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac112310 con 0x7f7cac102180 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 0x7f7cac19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 0x7f7cac19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46716/0 (socket says 172.21.15.5:46716) 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 -- 172.21.15.5:0/1337867827 learned_addr learned my addr 172.21.15.5:0/1337867827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7caad9d640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cac1985f0 0x7f7cac198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7caa59c640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac102180 0x7f7cac19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 -- 172.21.15.5:0/1337867827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac102180 msgr2=0x7f7cac19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac102180 0x7f7cac19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 -- 172.21.15.5:0/1337867827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cac1985f0 msgr2=0x7f7cac198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cac1985f0 0x7f7cac198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 -- 172.21.15.5:0/1337867827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac199110 con 0x7f7cac107250 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7ca9d9b640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 0x7f7cac19e980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000e9c0 tx=0x7f7ca000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000cd80 con 0x7f7cac107250 2024-04-22T13:27:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca00054d0 con 0x7f7cac107250 2024-04-22T13:27:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0010430 con 0x7f7cac107250 2024-04-22T13:27:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac199400 con 0x7f7cac107250 2024-04-22T13:27:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.298+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1a56a0 con 0x7f7cac107250 2024-04-22T13:27:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.299+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac069250 con 0x7f7cac107250 2024-04-22T13:27:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.301+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f7ca0004fb0 con 0x7f7cac107250 2024-04-22T13:27:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.301+0000 7f7c9b7fe640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7c940a3ec0 0x7f7c940a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.301+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7ca00d10a0 con 0x7f7cac107250 2024-04-22T13:27:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.301+0000 7f7caa59c640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7c940a3ec0 0x7f7c940a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.302+0000 7f7caa59c640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7c940a3ec0 0x7f7c940a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c036040 tx=0x7f7c9c000bc0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.305+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ca0053ff0 con 0x7f7cac107250 2024-04-22T13:27:20.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.491+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7cac102b30 con 0x7f7cac107250 2024-04-22T13:27:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.492+0000 7f7c9b7fe640 1 -- 172.21.15.5:0/1337867827 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f7ca009d520 con 0x7f7cac107250 2024-04-22T13:27:20.488 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.493+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7c940a3ec0 msgr2=0x7f7c940a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.493+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7c940a3ec0 0x7f7c940a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c036040 tx=0x7f7c9c000bc0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.493+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 msgr2=0x7f7cac19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.493+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 0x7f7cac19e980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000e9c0 tx=0x7f7ca000ee80 comp rx=0 tx=0).stop 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 shutdown_connections 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cac107250 0x7f7cac19e980 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7c940a3ec0 0x7f7c940a6380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cac1985f0 0x7f7cac198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 --2- 172.21.15.5:0/1337867827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac102180 0x7f7cac19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 >> 172.21.15.5:0/1337867827 conn(0x7f7cac0fda00 msgr2=0x7f7cac0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 shutdown_connections 2024-04-22T13:27:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.494+0000 7f7cb0fb6640 1 -- 172.21.15.5:0/1337867827 wait complete. 2024-04-22T13:27:20.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:27:20.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 -- 172.21.15.5:0/1797355908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102b40 msgr2=0x7f7620106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 --2- 172.21.15.5:0/1797355908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102b40 0x7f7620106b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7614009900 tx=0x7f761402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 -- 172.21.15.5:0/1797355908 shutdown_connections 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 --2- 172.21.15.5:0/1797355908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102b40 0x7f7620106b80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 --2- 172.21.15.5:0/1797355908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7620107260 0x7f762010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 --2- 172.21.15.5:0/1797355908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7620102190 0x7f7620102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 -- 172.21.15.5:0/1797355908 >> 172.21.15.5:0/1797355908 conn(0x7f76200fda10 msgr2=0x7f76200ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 -- 172.21.15.5:0/1797355908 shutdown_connections 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 -- 172.21.15.5:0/1797355908 wait complete. 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 Processor -- start 2024-04-22T13:27:20.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.610+0000 7f7626226640 1 -- start start 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f7626226640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 0x7f762019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f7626226640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7620107260 0x7f762019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f7626226640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7620198640 0x7f7620198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f7626226640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7620112200 con 0x7f7620102190 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f7626226640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7620112080 con 0x7f7620198640 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f7626226640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7620112380 con 0x7f7620107260 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 0x7f762019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 0x7f762019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46740/0 (socket says 172.21.15.5:46740) 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 -- 172.21.15.5:0/1496088586 learned_addr learned my addr 172.21.15.5:0/1496088586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761ffff640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7620198640 0x7f7620198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761effd640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7620107260 0x7f762019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 -- 172.21.15.5:0/1496088586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7620107260 msgr2=0x7f762019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7620107260 0x7f762019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 -- 172.21.15.5:0/1496088586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7620198640 msgr2=0x7f7620198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7620198640 0x7f7620198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 -- 172.21.15.5:0/1496088586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7620199240 con 0x7f7620102190 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761f7fe640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 0x7f762019e580 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f761000b4a0 tx=0x7f761000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.611+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610014070 con 0x7f7620102190 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.612+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7610010ac0 con 0x7f7620102190 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.612+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76201a5320 con 0x7f7620102190 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.612+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610017c30 con 0x7f7620102190 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.612+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76201a56d0 con 0x7f7620102190 2024-04-22T13:27:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.612+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7620103660 con 0x7f7620102190 2024-04-22T13:27:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.614+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f7610017070 con 0x7f7620102190 2024-04-22T13:27:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.614+0000 7f761cff9640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f76080a3e40 0x7f76080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.614+0000 7f761effd640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f76080a3e40 0x7f76080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.615+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f76100d0030 con 0x7f7620102190 2024-04-22T13:27:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.615+0000 7f761effd640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f76080a3e40 0x7f76080a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7614005ec0 tx=0x7f7614005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.618+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f761009bf70 con 0x7f7620102190 2024-04-22T13:27:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.803+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7620068e00 con 0x7f7620102190 2024-04-22T13:27:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.804+0000 7f761cff9640 1 -- 172.21.15.5:0/1496088586 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f76100a0270 con 0x7f7620102190 2024-04-22T13:27:20.800 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.805+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f76080a3e40 msgr2=0x7f76080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.805+0000 7f7626226640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f76080a3e40 0x7f76080a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7614005ec0 tx=0x7f7614005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 msgr2=0x7f762019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 0x7f762019e580 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f761000b4a0 tx=0x7f761000b960 comp rx=0 tx=0).stop 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 shutdown_connections 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f76080a3e40 0x7f76080a6300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7620198640 0x7f7620198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7620107260 0x7f762019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 --2- 172.21.15.5:0/1496088586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102190 0x7f762019e580 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 >> 172.21.15.5:0/1496088586 conn(0x7f76200fda10 msgr2=0x7f76200fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 shutdown_connections 2024-04-22T13:27:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.806+0000 7f7626226640 1 -- 172.21.15.5:0/1496088586 wait complete. 2024-04-22T13:27:20.812 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-22T13:27:20.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 -- 172.21.15.5:0/4162594217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30102170 msgr2=0x7fab30102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 --2- 172.21.15.5:0/4162594217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30102170 0x7fab30102550 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009900 tx=0x7fab2c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 -- 172.21.15.5:0/4162594217 shutdown_connections 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 --2- 172.21.15.5:0/4162594217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab30107240 0x7fab3010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 --2- 172.21.15.5:0/4162594217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30102b20 0x7fab30106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 --2- 172.21.15.5:0/4162594217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30102170 0x7fab30102550 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 -- 172.21.15.5:0/4162594217 >> 172.21.15.5:0/4162594217 conn(0x7fab300fd9d0 msgr2=0x7fab300ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 -- 172.21.15.5:0/4162594217 shutdown_connections 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.920+0000 7fab36c5c640 1 -- 172.21.15.5:0/4162594217 wait complete. 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 Processor -- start 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 -- start start 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30102170 0x7fab30198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 0x7fab30198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab3019cfd0 0x7fab3019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112180 con 0x7fab30107240 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112000 con 0x7fab30102170 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab36c5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112300 con 0x7fab3019cfd0 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 0x7fab30198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 0x7fab30198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46762/0 (socket says 172.21.15.5:46762) 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 -- 172.21.15.5:0/460055011 learned_addr learned my addr 172.21.15.5:0/460055011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab349f8640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30102170 0x7fab30198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab351f9640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab3019cfd0 0x7fab3019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 -- 172.21.15.5:0/460055011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab3019cfd0 msgr2=0x7fab3019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab3019cfd0 0x7fab3019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 -- 172.21.15.5:0/460055011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30102170 msgr2=0x7fab30198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30102170 0x7fab30198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 -- 172.21.15.5:0/460055011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3019dac0 con 0x7fab30107240 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.921+0000 7fab27fff640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 0x7fab30198ca0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab20005fd0 tx=0x7fab2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.922+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20017070 con 0x7fab30107240 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.922+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20009c30 con 0x7fab30107240 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.922+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3019ddb0 con 0x7fab30107240 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.922+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20003d30 con 0x7fab30107240 2024-04-22T13:27:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.922+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a55f0 con 0x7fab30107240 2024-04-22T13:27:20.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.923+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab30069250 con 0x7fab30107240 2024-04-22T13:27:20.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.924+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fab20003ed0 con 0x7fab30107240 2024-04-22T13:27:20.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.924+0000 7fab25ffb640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab180a3ec0 0x7fab180a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:20.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.925+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab200d06b0 con 0x7fab30107240 2024-04-22T13:27:20.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.925+0000 7fab349f8640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab180a3ec0 0x7fab180a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:20.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.925+0000 7fab349f8640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab180a3ec0 0x7fab180a6380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006010 tx=0x7fab2c005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:20.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:20.929+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab20020030 con 0x7fab30107240 2024-04-22T13:27:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.111+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab3010a4d0 con 0x7fab180a3ec0 2024-04-22T13:27:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.119+0000 7fab25ffb640 1 -- 172.21.15.5:0/460055011 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257802 (secure 0 0 0) 0x7fab3010a4d0 con 0x7fab180a3ec0 2024-04-22T13:27:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab180a3ec0 msgr2=0x7fab180a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab180a3ec0 0x7fab180a6380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006010 tx=0x7fab2c005e40 comp rx=0 tx=0).stop 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 msgr2=0x7fab30198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 0x7fab30198ca0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab20005fd0 tx=0x7fab2000dde0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 shutdown_connections 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab30107240 0x7fab30198ca0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab180a3ec0 0x7fab180a6380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab3019cfd0 0x7fab3019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 --2- 172.21.15.5:0/460055011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30102170 0x7fab30198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 >> 172.21.15.5:0/460055011 conn(0x7fab300fd9d0 msgr2=0x7fab300fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:21.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 shutdown_connections 2024-04-22T13:27:21.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.122+0000 7fab36c5c640 1 -- 172.21.15.5:0/460055011 wait complete. 2024-04-22T13:27:21.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:20 smithi005 ceph-mon[28006]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-04-22T13:27:21.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:20 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1854355113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:20 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1337867827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:20 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:27:21.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:20 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1496088586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.129 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-22T13:27:19.461585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192040,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1566,"num_read_kb":1112,"num_write":579,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":952,"ondisk_log_size":952,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":418916,"kb_used_data":15600,"kb_used_omap":18,"kb_used_meta":403117,"kb_avail":1124522908,"statfs":{"total":1151940427776,"available":1151511457792,"internally_reserved":0,"allocated":15974400,"data_stored":9374940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":412792122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4084,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":242,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.989674"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563085+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:27:09.563085+0000","last_peered":"2024-04-22T13:27:09.563085+0000","last_clean":"2024-04-22T13:27:09.563085+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:27:09.563085+0000","last_undegraded":"2024-04-22T13:27:09.563085+0000","last_fullsized":"2024-04-22T13:27:09.563085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564703+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:27:09.564703+0000","last_peered":"2024-04-22T13:27:09.564703+0000","last_clean":"2024-04-22T13:27:09.564703+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:27:09.564703+0000","last_undegraded":"2024-04-22T13:27:09.564703+0000","last_fullsized":"2024-04-22T13:27:09.564703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564676+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:27:09.564676+0000","last_peered":"2024-04-22T13:27:09.564676+0000","last_clean":"2024-04-22T13:27:09.564676+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:27:09.564676+0000","last_undegraded":"2024-04-22T13:27:09.564676+0000","last_fullsized":"2024-04-22T13:27:09.564676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563774+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:27:09.563774+0000","last_peered":"2024-04-22T13:27:09.563774+0000","last_clean":"2024-04-22T13:27:09.563774+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:27:09.563774+0000","last_undegraded":"2024-04-22T13:27:09.563774+0000","last_fullsized":"2024-04-22T13:27:09.563774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448296+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:27:09.448296+0000","last_peered":"2024-04-22T13:27:09.448296+0000","last_clean":"2024-04-22T13:27:09.448296+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:27:09.448296+0000","last_undegraded":"2024-04-22T13:27:09.448296+0000","last_fullsized":"2024-04-22T13:27:09.448296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564063+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:27:09.564063+0000","last_peered":"2024-04-22T13:27:09.564063+0000","last_clean":"2024-04-22T13:27:09.564063+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:27:09.564063+0000","last_undegraded":"2024-04-22T13:27:09.564063+0000","last_fullsized":"2024-04-22T13:27:09.564063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540227+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:27:09.540227+0000","last_peered":"2024-04-22T13:27:09.540227+0000","last_clean":"2024-04-22T13:27:09.540227+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:27:09.540227+0000","last_undegraded":"2024-04-22T13:27:09.540227+0000","last_fullsized":"2024-04-22T13:27:09.540227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563416+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:27:09.563416+0000","last_peered":"2024-04-22T13:27:09.563416+0000","last_clean":"2024-04-22T13:27:09.563416+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:27:09.563416+0000","last_undegraded":"2024-04-22T13:27:09.563416+0000","last_fullsized":"2024-04-22T13:27:09.563416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448852+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:27:09.448852+0000","last_peered":"2024-04-22T13:27:09.448852+0000","last_clean":"2024-04-22T13:27:09.448852+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:27:09.448852+0000","last_undegraded":"2024-04-22T13:27:09.448852+0000","last_fullsized":"2024-04-22T13:27:09.448852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564466+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:27:09.564466+0000","last_peered":"2024-04-22T13:27:09.564466+0000","last_clean":"2024-04-22T13:27:09.564466+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:27:09.564466+0000","last_undegraded":"2024-04-22T13:27:09.564466+0000","last_fullsized":"2024-04-22T13:27:09.564466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563562+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:27:09.563562+0000","last_peered":"2024-04-22T13:27:09.563562+0000","last_clean":"2024-04-22T13:27:09.563562+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:27:09.563562+0000","last_undegraded":"2024-04-22T13:27:09.563562+0000","last_fullsized":"2024-04-22T13:27:09.563562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563593+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:27:09.563593+0000","last_peered":"2024-04-22T13:27:09.563593+0000","last_clean":"2024-04-22T13:27:09.563593+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:27:09.563593+0000","last_undegraded":"2024-04-22T13:27:09.563593+0000","last_fullsized":"2024-04-22T13:27:09.563593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563776+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:27:09.563776+0000","last_peered":"2024-04-22T13:27:09.563776+0000","last_clean":"2024-04-22T13:27:09.563776+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:27:09.563776+0000","last_undegraded":"2024-04-22T13:27:09.563776+0000","last_fullsized":"2024-04-22T13:27:09.563776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562794+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:27:09.562794+0000","last_peered":"2024-04-22T13:27:09.562794+0000","last_clean":"2024-04-22T13:27:09.562794+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:27:09.562794+0000","last_undegraded":"2024-04-22T13:27:09.562794+0000","last_fullsized":"2024-04-22T13:27:09.562794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562713+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:27:09.562713+0000","last_peered":"2024-04-22T13:27:09.562713+0000","last_clean":"2024-04-22T13:27:09.562713+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:27:09.562713+0000","last_undegraded":"2024-04-22T13:27:09.562713+0000","last_fullsized":"2024-04-22T13:27:09.562713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563250+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.563250+0000","last_peered":"2024-04-22T13:27:09.563250+0000","last_clean":"2024-04-22T13:27:09.563250+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:27:09.563250+0000","last_undegraded":"2024-04-22T13:27:09.563250+0000","last_fullsized":"2024-04-22T13:27:09.563250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446872+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:27:09.446872+0000","last_peered":"2024-04-22T13:27:09.446872+0000","last_clean":"2024-04-22T13:27:09.446872+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:27:09.446872+0000","last_undegraded":"2024-04-22T13:27:09.446872+0000","last_fullsized":"2024-04-22T13:27:09.446872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563565+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:27:09.563565+0000","last_peered":"2024-04-22T13:27:09.563565+0000","last_clean":"2024-04-22T13:27:09.563565+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:27:09.563565+0000","last_undegraded":"2024-04-22T13:27:09.563565+0000","last_fullsized":"2024-04-22T13:27:09.563565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:15.404954+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:27:15.404954+0000","last_peered":"2024-04-22T13:27:15.404954+0000","last_clean":"2024-04-22T13:27:15.404954+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:27:15.404954+0000","last_undegraded":"2024-04-22T13:27:15.404954+0000","last_fullsized":"2024-04-22T13:27:15.404954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541086+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:27:09.541086+0000","last_peered":"2024-04-22T13:27:09.541086+0000","last_clean":"2024-04-22T13:27:09.541086+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:27:09.541086+0000","last_undegraded":"2024-04-22T13:27:09.541086+0000","last_fullsized":"2024-04-22T13:27:09.541086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541272+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:27:09.541272+0000","last_peered":"2024-04-22T13:27:09.541272+0000","last_clean":"2024-04-22T13:27:09.541272+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:27:09.541272+0000","last_undegraded":"2024-04-22T13:27:09.541272+0000","last_fullsized":"2024-04-22T13:27:09.541272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564850+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:27:09.564850+0000","last_peered":"2024-04-22T13:27:09.564850+0000","last_clean":"2024-04-22T13:27:09.564850+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:27:09.564850+0000","last_undegraded":"2024-04-22T13:27:09.564850+0000","last_fullsized":"2024-04-22T13:27:09.564850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'13","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563038+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:27:09.563038+0000","last_peered":"2024-04-22T13:27:09.563038+0000","last_clean":"2024-04-22T13:27:09.563038+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:27:09.563038+0000","last_undegraded":"2024-04-22T13:27:09.563038+0000","last_fullsized":"2024-04-22T13:27:09.563038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562953+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:27:09.562953+0000","last_peered":"2024-04-22T13:27:09.562953+0000","last_clean":"2024-04-22T13:27:09.562953+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:27:09.562953+0000","last_undegraded":"2024-04-22T13:27:09.562953+0000","last_fullsized":"2024-04-22T13:27:09.562953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.451042+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:27:09.451042+0000","last_peered":"2024-04-22T13:27:09.451042+0000","last_clean":"2024-04-22T13:27:09.451042+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:27:09.451042+0000","last_undegraded":"2024-04-22T13:27:09.451042+0000","last_fullsized":"2024-04-22T13:27:09.451042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547806+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:27:09.547806+0000","last_peered":"2024-04-22T13:27:09.547806+0000","last_clean":"2024-04-22T13:27:09.547806+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:27:09.547806+0000","last_undegraded":"2024-04-22T13:27:09.547806+0000","last_fullsized":"2024-04-22T13:27:09.547806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:14.404861+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:27:14.404861+0000","last_peered":"2024-04-22T13:27:14.404861+0000","last_clean":"2024-04-22T13:27:14.404861+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:27:14.404861+0000","last_undegraded":"2024-04-22T13:27:14.404861+0000","last_fullsized":"2024-04-22T13:27:14.404861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450920+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:27:09.450920+0000","last_peered":"2024-04-22T13:27:09.450920+0000","last_clean":"2024-04-22T13:27:09.450920+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:27:09.450920+0000","last_undegraded":"2024-04-22T13:27:09.450920+0000","last_fullsized":"2024-04-22T13:27:09.450920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448243+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:27:09.448243+0000","last_peered":"2024-04-22T13:27:09.448243+0000","last_clean":"2024-04-22T13:27:09.448243+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:27:09.448243+0000","last_undegraded":"2024-04-22T13:27:09.448243+0000","last_fullsized":"2024-04-22T13:27:09.448243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547784+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:27:09.547784+0000","last_peered":"2024-04-22T13:27:09.547784+0000","last_clean":"2024-04-22T13:27:09.547784+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:27:09.547784+0000","last_undegraded":"2024-04-22T13:27:09.547784+0000","last_fullsized":"2024-04-22T13:27:09.547784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562955+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:27:09.562955+0000","last_peered":"2024-04-22T13:27:09.562955+0000","last_clean":"2024-04-22T13:27:09.562955+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:27:09.562955+0000","last_undegraded":"2024-04-22T13:27:09.562955+0000","last_fullsized":"2024-04-22T13:27:09.562955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540293+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:27:09.540293+0000","last_peered":"2024-04-22T13:27:09.540293+0000","last_clean":"2024-04-22T13:27:09.540293+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:27:09.540293+0000","last_undegraded":"2024-04-22T13:27:09.540293+0000","last_fullsized":"2024-04-22T13:27:09.540293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547473+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:27:09.547473+0000","last_peered":"2024-04-22T13:27:09.547473+0000","last_clean":"2024-04-22T13:27:09.547473+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:27:09.547473+0000","last_undegraded":"2024-04-22T13:27:09.547473+0000","last_fullsized":"2024-04-22T13:27:09.547473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446526+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:27:09.446526+0000","last_peered":"2024-04-22T13:27:09.446526+0000","last_clean":"2024-04-22T13:27:09.446526+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:27:09.446526+0000","last_undegraded":"2024-04-22T13:27:09.446526+0000","last_fullsized":"2024-04-22T13:27:09.446526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":266,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:15.405597+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:27:15.405597+0000","last_peered":"2024-04-22T13:27:15.405597+0000","last_clean":"2024-04-22T13:27:15.405597+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:27:15.405597+0000","last_undegraded":"2024-04-22T13:27:15.405597+0000","last_fullsized":"2024-04-22T13:27:15.405597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563427+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:27:09.563427+0000","last_peered":"2024-04-22T13:27:09.563427+0000","last_clean":"2024-04-22T13:27:09.563427+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:27:09.563427+0000","last_undegraded":"2024-04-22T13:27:09.563427+0000","last_fullsized":"2024-04-22T13:27:09.563427+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564441+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:27:09.564441+0000","last_peered":"2024-04-22T13:27:09.564441+0000","last_clean":"2024-04-22T13:27:09.564441+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:27:09.564441+0000","last_undegraded":"2024-04-22T13:27:09.564441+0000","last_fullsized":"2024-04-22T13:27:09.564441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564869+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:27:09.564869+0000","last_peered":"2024-04-22T13:27:09.564869+0000","last_clean":"2024-04-22T13:27:09.564869+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:27:09.564869+0000","last_undegraded":"2024-04-22T13:27:09.564869+0000","last_fullsized":"2024-04-22T13:27:09.564869+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547887+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:27:09.547887+0000","last_peered":"2024-04-22T13:27:09.547887+0000","last_clean":"2024-04-22T13:27:09.547887+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:27:09.547887+0000","last_undegraded":"2024-04-22T13:27:09.547887+0000","last_fullsized":"2024-04-22T13:27:09.547887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562984+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:27:09.562984+0000","last_peered":"2024-04-22T13:27:09.562984+0000","last_clean":"2024-04-22T13:27:09.562984+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:27:09.562984+0000","last_undegraded":"2024-04-22T13:27:09.562984+0000","last_fullsized":"2024-04-22T13:27:09.562984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":306,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:15.409853+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:27:15.409853+0000","last_peered":"2024-04-22T13:27:15.409853+0000","last_clean":"2024-04-22T13:27:15.409853+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:27:15.409853+0000","last_undegraded":"2024-04-22T13:27:15.409853+0000","last_fullsized":"2024-04-22T13:27:15.409853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67405,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":190,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450855+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:27:09.450855+0000","last_peered":"2024-04-22T13:27:09.450855+0000","last_clean":"2024-04-22T13:27:09.450855+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:27:09.450855+0000","last_undegraded":"2024-04-22T13:27:09.450855+0000","last_fullsized":"2024-04-22T13:27:09.450855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563392+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:27:09.563392+0000","last_peered":"2024-04-22T13:27:09.563392+0000","last_clean":"2024-04-22T13:27:09.563392+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:27:09.563392+0000","last_undegraded":"2024-04-22T13:27:09.563392+0000","last_fullsized":"2024-04-22T13:27:09.563392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562910+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:27:09.562910+0000","last_peered":"2024-04-22T13:27:09.562910+0000","last_clean":"2024-04-22T13:27:09.562910+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:27:09.562910+0000","last_undegraded":"2024-04-22T13:27:09.562910+0000","last_fullsized":"2024-04-22T13:27:09.562910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547599+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:27:09.547599+0000","last_peered":"2024-04-22T13:27:09.547599+0000","last_clean":"2024-04-22T13:27:09.547599+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:27:09.547599+0000","last_undegraded":"2024-04-22T13:27:09.547599+0000","last_fullsized":"2024-04-22T13:27:09.547599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564267+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:27:09.564267+0000","last_peered":"2024-04-22T13:27:09.564267+0000","last_clean":"2024-04-22T13:27:09.564267+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:27:09.564267+0000","last_undegraded":"2024-04-22T13:27:09.564267+0000","last_fullsized":"2024-04-22T13:27:09.564267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":147,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:14.404299+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:27:14.404299+0000","last_peered":"2024-04-22T13:27:14.404299+0000","last_clean":"2024-04-22T13:27:14.404299+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:27:14.404299+0000","last_undegraded":"2024-04-22T13:27:14.404299+0000","last_fullsized":"2024-04-22T13:27:14.404299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563985+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:27:09.563985+0000","last_peered":"2024-04-22T13:27:09.563985+0000","last_clean":"2024-04-22T13:27:09.563985+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:27:09.563985+0000","last_undegraded":"2024-04-22T13:27:09.563985+0000","last_fullsized":"2024-04-22T13:27:09.563985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450657+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:27:09.450657+0000","last_peered":"2024-04-22T13:27:09.450657+0000","last_clean":"2024-04-22T13:27:09.450657+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:27:09.450657+0000","last_undegraded":"2024-04-22T13:27:09.450657+0000","last_fullsized":"2024-04-22T13:27:09.450657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547522+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:27:09.547522+0000","last_peered":"2024-04-22T13:27:09.547522+0000","last_clean":"2024-04-22T13:27:09.547522+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:27:09.547522+0000","last_undegraded":"2024-04-22T13:27:09.547522+0000","last_fullsized":"2024-04-22T13:27:09.547522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450885+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:27:09.450885+0000","last_peered":"2024-04-22T13:27:09.450885+0000","last_clean":"2024-04-22T13:27:09.450885+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:27:09.450885+0000","last_undegraded":"2024-04-22T13:27:09.450885+0000","last_fullsized":"2024-04-22T13:27:09.450885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563035+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:27:09.563035+0000","last_peered":"2024-04-22T13:27:09.563035+0000","last_clean":"2024-04-22T13:27:09.563035+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:27:09.563035+0000","last_undegraded":"2024-04-22T13:27:09.563035+0000","last_fullsized":"2024-04-22T13:27:09.563035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446714+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:27:09.446714+0000","last_peered":"2024-04-22T13:27:09.446714+0000","last_clean":"2024-04-22T13:27:09.446714+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:27:09.446714+0000","last_undegraded":"2024-04-22T13:27:09.446714+0000","last_fullsized":"2024-04-22T13:27:09.446714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448687+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:27:09.448687+0000","last_peered":"2024-04-22T13:27:09.448687+0000","last_clean":"2024-04-22T13:27:09.448687+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:27:09.448687+0000","last_undegraded":"2024-04-22T13:27:09.448687+0000","last_fullsized":"2024-04-22T13:27:09.448687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:15.405682+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:27:15.405682+0000","last_peered":"2024-04-22T13:27:15.405682+0000","last_clean":"2024-04-22T13:27:15.405682+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:27:15.405682+0000","last_undegraded":"2024-04-22T13:27:15.405682+0000","last_fullsized":"2024-04-22T13:27:15.405682+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448498+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:27:09.448498+0000","last_peered":"2024-04-22T13:27:09.448498+0000","last_clean":"2024-04-22T13:27:09.448498+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:27:09.448498+0000","last_undegraded":"2024-04-22T13:27:09.448498+0000","last_fullsized":"2024-04-22T13:27:09.448498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446492+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:27:09.446492+0000","last_peered":"2024-04-22T13:27:09.446492+0000","last_clean":"2024-04-22T13:27:09.446492+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:27:09.446492+0000","last_undegraded":"2024-04-22T13:27:09.446492+0000","last_fullsized":"2024-04-22T13:27:09.446492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":147,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:15.404770+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:27:15.404770+0000","last_peered":"2024-04-22T13:27:15.404770+0000","last_clean":"2024-04-22T13:27:15.404770+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:27:15.404770+0000","last_undegraded":"2024-04-22T13:27:15.404770+0000","last_fullsized":"2024-04-22T13:27:15.404770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540568+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:27:09.540568+0000","last_peered":"2024-04-22T13:27:09.540568+0000","last_clean":"2024-04-22T13:27:09.540568+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:27:09.540568+0000","last_undegraded":"2024-04-22T13:27:09.540568+0000","last_fullsized":"2024-04-22T13:27:09.540568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563518+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:27:09.563518+0000","last_peered":"2024-04-22T13:27:09.563518+0000","last_clean":"2024-04-22T13:27:09.563518+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:27:09.563518+0000","last_undegraded":"2024-04-22T13:27:09.563518+0000","last_fullsized":"2024-04-22T13:27:09.563518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":403,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:11.559108+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:27:11.559108+0000","last_peered":"2024-04-22T13:27:11.559108+0000","last_clean":"2024-04-22T13:27:11.559108+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:27:11.559108+0000","last_undegraded":"2024-04-22T13:27:11.559108+0000","last_fullsized":"2024-04-22T13:27:11.559108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562824+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:27:09.562824+0000","last_peered":"2024-04-22T13:27:09.562824+0000","last_clean":"2024-04-22T13:27:09.562824+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:27:09.562824+0000","last_undegraded":"2024-04-22T13:27:09.562824+0000","last_fullsized":"2024-04-22T13:27:09.562824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564286+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:27:09.564286+0000","last_peered":"2024-04-22T13:27:09.564286+0000","last_clean":"2024-04-22T13:27:09.564286+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:27:09.564286+0000","last_undegraded":"2024-04-22T13:27:09.564286+0000","last_fullsized":"2024-04-22T13:27:09.564286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450990+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:27:09.450990+0000","last_peered":"2024-04-22T13:27:09.450990+0000","last_clean":"2024-04-22T13:27:09.450990+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:27:09.450990+0000","last_undegraded":"2024-04-22T13:27:09.450990+0000","last_fullsized":"2024-04-22T13:27:09.450990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563221+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:27:09.563221+0000","last_peered":"2024-04-22T13:27:09.563221+0000","last_clean":"2024-04-22T13:27:09.563221+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:27:09.563221+0000","last_undegraded":"2024-04-22T13:27:09.563221+0000","last_fullsized":"2024-04-22T13:27:09.563221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'50","reported_seq":94,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.916336+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:27:09.916336+0000","last_peered":"2024-04-22T13:27:09.916336+0000","last_clean":"2024-04-22T13:27:09.916336+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:27:09.916336+0000","last_undegraded":"2024-04-22T13:27:09.916336+0000","last_fullsized":"2024-04-22T13:27:09.916336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541266+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:27:09.541266+0000","last_peered":"2024-04-22T13:27:09.541266+0000","last_clean":"2024-04-22T13:27:09.541266+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:27:09.541266+0000","last_undegraded":"2024-04-22T13:27:09.541266+0000","last_fullsized":"2024-04-22T13:27:09.541266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563070+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:27:09.563070+0000","last_peered":"2024-04-22T13:27:09.563070+0000","last_clean":"2024-04-22T13:27:09.563070+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:27:09.563070+0000","last_undegraded":"2024-04-22T13:27:09.563070+0000","last_fullsized":"2024-04-22T13:27:09.563070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564799+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:27:09.564799+0000","last_peered":"2024-04-22T13:27:09.564799+0000","last_clean":"2024-04-22T13:27:09.564799+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:27:09.564799+0000","last_undegraded":"2024-04-22T13:27:09.564799+0000","last_fullsized":"2024-04-22T13:27:09.564799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562808+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:27:09.562808+0000","last_peered":"2024-04-22T13:27:09.562808+0000","last_clean":"2024-04-22T13:27:09.562808+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:27:09.562808+0000","last_undegraded":"2024-04-22T13:27:09.562808+0000","last_fullsized":"2024-04-22T13:27:09.562808+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450691+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:27:09.450691+0000","last_peered":"2024-04-22T13:27:09.450691+0000","last_clean":"2024-04-22T13:27:09.450691+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:27:09.450691+0000","last_undegraded":"2024-04-22T13:27:09.450691+0000","last_fullsized":"2024-04-22T13:27:09.450691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540308+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:27:09.540308+0000","last_peered":"2024-04-22T13:27:09.540308+0000","last_clean":"2024-04-22T13:27:09.540308+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:27:09.540308+0000","last_undegraded":"2024-04-22T13:27:09.540308+0000","last_fullsized":"2024-04-22T13:27:09.540308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562679+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:27:09.562679+0000","last_peered":"2024-04-22T13:27:09.562679+0000","last_clean":"2024-04-22T13:27:09.562679+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:27:09.562679+0000","last_undegraded":"2024-04-22T13:27:09.562679+0000","last_fullsized":"2024-04-22T13:27:09.562679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562789+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:27:09.562789+0000","last_peered":"2024-04-22T13:27:09.562789+0000","last_clean":"2024-04-22T13:27:09.562789+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:27:09.562789+0000","last_undegraded":"2024-04-22T13:27:09.562789+0000","last_fullsized":"2024-04-22T13:27:09.562789+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540822+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:27:09.540822+0000","last_peered":"2024-04-22T13:27:09.540822+0000","last_clean":"2024-04-22T13:27:09.540822+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:27:09.540822+0000","last_undegraded":"2024-04-22T13:27:09.540822+0000","last_fullsized":"2024-04-22T13:27:09.540822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450783+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:27:09.450783+0000","last_peered":"2024-04-22T13:27:09.450783+0000","last_clean":"2024-04-22T13:27:09.450783+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:27:09.450783+0000","last_undegraded":"2024-04-22T13:27:09.450783+0000","last_fullsized":"2024-04-22T13:27:09.450783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564270+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:27:09.564270+0000","last_peered":"2024-04-22T13:27:09.564270+0000","last_clean":"2024-04-22T13:27:09.564270+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:27:09.564270+0000","last_undegraded":"2024-04-22T13:27:09.564270+0000","last_fullsized":"2024-04-22T13:27:09.564270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":153,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:14.404696+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:27:14.404696+0000","last_peered":"2024-04-22T13:27:14.404696+0000","last_clean":"2024-04-22T13:27:14.404696+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:27:14.404696+0000","last_undegraded":"2024-04-22T13:27:14.404696+0000","last_fullsized":"2024-04-22T13:27:14.404696+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564284+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.564284+0000","last_peered":"2024-04-22T13:27:09.564284+0000","last_clean":"2024-04-22T13:27:09.564284+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:27:09.564284+0000","last_undegraded":"2024-04-22T13:27:09.564284+0000","last_fullsized":"2024-04-22T13:27:09.564284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.651858+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:27:09.651858+0000","last_peered":"2024-04-22T13:27:09.651858+0000","last_clean":"2024-04-22T13:27:09.651858+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:27:09.651858+0000","last_undegraded":"2024-04-22T13:27:09.651858+0000","last_fullsized":"2024-04-22T13:27:09.651858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564302+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:27:09.564302+0000","last_peered":"2024-04-22T13:27:09.564302+0000","last_clean":"2024-04-22T13:27:09.564302+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:27:09.564302+0000","last_undegraded":"2024-04-22T13:27:09.564302+0000","last_fullsized":"2024-04-22T13:27:09.564302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563457+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:27:09.563457+0000","last_peered":"2024-04-22T13:27:09.563457+0000","last_clean":"2024-04-22T13:27:09.563457+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:27:09.563457+0000","last_undegraded":"2024-04-22T13:27:09.563457+0000","last_fullsized":"2024-04-22T13:27:09.563457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564533+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:27:09.564533+0000","last_peered":"2024-04-22T13:27:09.564533+0000","last_clean":"2024-04-22T13:27:09.564533+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:27:09.564533+0000","last_undegraded":"2024-04-22T13:27:09.564533+0000","last_fullsized":"2024-04-22T13:27:09.564533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562753+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:27:09.562753+0000","last_peered":"2024-04-22T13:27:09.562753+0000","last_clean":"2024-04-22T13:27:09.562753+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:27:09.562753+0000","last_undegraded":"2024-04-22T13:27:09.562753+0000","last_fullsized":"2024-04-22T13:27:09.562753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563366+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:27:09.563366+0000","last_peered":"2024-04-22T13:27:09.563366+0000","last_clean":"2024-04-22T13:27:09.563366+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:27:09.563366+0000","last_undegraded":"2024-04-22T13:27:09.563366+0000","last_fullsized":"2024-04-22T13:27:09.563366+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563322+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:27:09.563322+0000","last_peered":"2024-04-22T13:27:09.563322+0000","last_clean":"2024-04-22T13:27:09.563322+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:27:09.563322+0000","last_undegraded":"2024-04-22T13:27:09.563322+0000","last_fullsized":"2024-04-22T13:27:09.563322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564293+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:27:09.564293+0000","last_peered":"2024-04-22T13:27:09.564293+0000","last_clean":"2024-04-22T13:27:09.564293+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:27:09.564293+0000","last_undegraded":"2024-04-22T13:27:09.564293+0000","last_fullsized":"2024-04-22T13:27:09.564293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563391+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:27:09.563391+0000","last_peered":"2024-04-22T13:27:09.563391+0000","last_clean":"2024-04-22T13:27:09.563391+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:27:09.563391+0000","last_undegraded":"2024-04-22T13:27:09.563391+0000","last_fullsized":"2024-04-22T13:27:09.563391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563022+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:27:09.563022+0000","last_peered":"2024-04-22T13:27:09.563022+0000","last_clean":"2024-04-22T13:27:09.563022+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:27:09.563022+0000","last_undegraded":"2024-04-22T13:27:09.563022+0000","last_fullsized":"2024-04-22T13:27:09.563022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563649+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:27:09.563649+0000","last_peered":"2024-04-22T13:27:09.563649+0000","last_clean":"2024-04-22T13:27:09.563649+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:27:09.563649+0000","last_undegraded":"2024-04-22T13:27:09.563649+0000","last_fullsized":"2024-04-22T13:27:09.563649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541195+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:27:09.541195+0000","last_peered":"2024-04-22T13:27:09.541195+0000","last_clean":"2024-04-22T13:27:09.541195+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:27:09.541195+0000","last_undegraded":"2024-04-22T13:27:09.541195+0000","last_fullsized":"2024-04-22T13:27:09.541195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562758+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:27:09.562758+0000","last_peered":"2024-04-22T13:27:09.562758+0000","last_clean":"2024-04-22T13:27:09.562758+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:27:09.562758+0000","last_undegraded":"2024-04-22T13:27:09.562758+0000","last_fullsized":"2024-04-22T13:27:09.562758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540273+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:27:09.540273+0000","last_peered":"2024-04-22T13:27:09.540273+0000","last_clean":"2024-04-22T13:27:09.540273+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:27:09.540273+0000","last_undegraded":"2024-04-22T13:27:09.540273+0000","last_fullsized":"2024-04-22T13:27:09.540273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562941+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:27:09.562941+0000","last_peered":"2024-04-22T13:27:09.562941+0000","last_clean":"2024-04-22T13:27:09.562941+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:27:09.562941+0000","last_undegraded":"2024-04-22T13:27:09.562941+0000","last_fullsized":"2024-04-22T13:27:09.562941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563605+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:27:09.563605+0000","last_peered":"2024-04-22T13:27:09.563605+0000","last_clean":"2024-04-22T13:27:09.563605+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:27:09.563605+0000","last_undegraded":"2024-04-22T13:27:09.563605+0000","last_fullsized":"2024-04-22T13:27:09.563605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562834+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:27:09.562834+0000","last_peered":"2024-04-22T13:27:09.562834+0000","last_clean":"2024-04-22T13:27:09.562834+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:27:09.562834+0000","last_undegraded":"2024-04-22T13:27:09.562834+0000","last_fullsized":"2024-04-22T13:27:09.562834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601438,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1329,"num_read_kb":782,"num_write":376,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1804314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":485517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514537,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41368,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703784,"statfs":{"total":95995035648,"available":95952674816,"internally_reserved":0,"allocated":1855488,"data_stored":1274461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678063,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40896,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704256,"statfs":{"total":95995035648,"available":95953158144,"internally_reserved":0,"allocated":1429504,"data_stored":877573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939701,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1425408,"data_stored":877951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41088,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704064,"statfs":{"total":95995035648,"available":95952961536,"internally_reserved":0,"allocated":1630208,"data_stored":1078567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":553174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":486503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":554625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41024,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704128,"statfs":{"total":95995035648,"available":95953027072,"internally_reserved":0,"allocated":1564672,"data_stored":1008715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1209063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:27:21.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:27:21.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- 172.21.15.5:0/1039208490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c101a20 msgr2=0x7f5c2c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1039208490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c101a20 0x7f5c2c105a60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009900 tx=0x7f5c1c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:21.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- 172.21.15.5:0/1039208490 shutdown_connections 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1039208490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c101a20 0x7f5c2c105a60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1039208490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c2c106140 0x7f5c2c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1039208490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2c101070 0x7f5c2c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- 172.21.15.5:0/1039208490 >> 172.21.15.5:0/1039208490 conn(0x7f5c2c0771d0 msgr2=0x7f5c2c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- 172.21.15.5:0/1039208490 shutdown_connections 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- 172.21.15.5:0/1039208490 wait complete. 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 Processor -- start 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- start start 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c2c101070 0x7f5c2c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 0x7f5c2c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2c19cf40 0x7f5c2c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112180 con 0x7f5c2c106140 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112000 con 0x7f5c2c101070 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.275+0000 7f5c335e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112300 con 0x7f5c2c19cf40 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 0x7f5c2c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 0x7f5c2c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46774/0 (socket says 172.21.15.5:46774) 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 -- 172.21.15.5:0/1294085291 learned_addr learned my addr 172.21.15.5:0/1294085291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c31b7f640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2c19cf40 0x7f5c2c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c3137e640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c2c101070 0x7f5c2c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 -- 172.21.15.5:0/1294085291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2c19cf40 msgr2=0x7f5c2c19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2c19cf40 0x7f5c2c19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 -- 172.21.15.5:0/1294085291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c2c101070 msgr2=0x7f5c2c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c2c101070 0x7f5c2c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 -- 172.21.15.5:0/1294085291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c19da30 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c30b7d640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 0x7f5c2c198bc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c036040 tx=0x7f5c1c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c043020 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c1c02cbe0 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c19dcc0 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c003bf0 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.276+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a5510 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.277+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c068e00 con 0x7f5c2c106140 2024-04-22T13:27:21.276 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:20 smithi049 ceph-mon[31379]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-04-22T13:27:21.276 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:20 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1854355113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.277 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:20 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1337867827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.277 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:20 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:27:21.277 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:20 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1496088586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.278+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f5c1c046020 con 0x7f5c2c106140 2024-04-22T13:27:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.279+0000 7f5c227fc640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c180a3ec0 0x7f5c180a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.279+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f5c1c0f2c20 con 0x7f5c2c106140 2024-04-22T13:27:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.279+0000 7f5c3137e640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c180a3ec0 0x7f5c180a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.280+0000 7f5c3137e640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c180a3ec0 0x7f5c180a6380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005e10 tx=0x7f5c28005da0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.284+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c1c04d020 con 0x7f5c2c106140 2024-04-22T13:27:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:20 smithi107 ceph-mon[31859]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-04-22T13:27:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:20 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1854355113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:20 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1337867827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:20 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:27:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:20 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1496088586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:21.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.467+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c2c101a20 con 0x7f5c180a3ec0 2024-04-22T13:27:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.474+0000 7f5c227fc640 1 -- 172.21.15.5:0/1294085291 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257819 (secure 0 0 0) 0x7f5c2c101a20 con 0x7f5c180a3ec0 2024-04-22T13:27:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:27:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c180a3ec0 msgr2=0x7f5c180a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c180a3ec0 0x7f5c180a6380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005e10 tx=0x7f5c28005da0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 msgr2=0x7f5c2c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 0x7f5c2c198bc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c036040 tx=0x7f5c1c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 shutdown_connections 2024-04-22T13:27:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c2c106140 0x7f5c2c198bc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c180a3ec0 0x7f5c180a6380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2c19cf40 0x7f5c2c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 --2- 172.21.15.5:0/1294085291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c2c101070 0x7f5c2c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 >> 172.21.15.5:0/1294085291 conn(0x7f5c2c0771d0 msgr2=0x7f5c2c107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.477+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 shutdown_connections 2024-04-22T13:27:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.478+0000 7f5c335e2640 1 -- 172.21.15.5:0/1294085291 wait complete. 2024-04-22T13:27:21.484 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-22T13:27:21.461967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389810,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1872,"num_read_kb":1371,"num_write":706,"num_write_kb":5323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1032,"ondisk_log_size":1032,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":423948,"kb_used_data":16176,"kb_used_omap":18,"kb_used_meta":407277,"kb_avail":1124517876,"statfs":{"total":1151940427776,"available":1151506305024,"internally_reserved":0,"allocated":16564224,"data_stored":9968250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":417051962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":461,"num_read_kb":501,"num_write":157,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.990056"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563135+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:27:09.563135+0000","last_peered":"2024-04-22T13:27:09.563135+0000","last_clean":"2024-04-22T13:27:09.563135+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:27:09.563135+0000","last_undegraded":"2024-04-22T13:27:09.563135+0000","last_fullsized":"2024-04-22T13:27:09.563135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564703+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:27:09.564703+0000","last_peered":"2024-04-22T13:27:09.564703+0000","last_clean":"2024-04-22T13:27:09.564703+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:27:09.564703+0000","last_undegraded":"2024-04-22T13:27:09.564703+0000","last_fullsized":"2024-04-22T13:27:09.564703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564676+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:27:09.564676+0000","last_peered":"2024-04-22T13:27:09.564676+0000","last_clean":"2024-04-22T13:27:09.564676+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:27:09.564676+0000","last_undegraded":"2024-04-22T13:27:09.564676+0000","last_fullsized":"2024-04-22T13:27:09.564676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563774+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:27:09.563774+0000","last_peered":"2024-04-22T13:27:09.563774+0000","last_clean":"2024-04-22T13:27:09.563774+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:27:09.563774+0000","last_undegraded":"2024-04-22T13:27:09.563774+0000","last_fullsized":"2024-04-22T13:27:09.563774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448296+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:27:09.448296+0000","last_peered":"2024-04-22T13:27:09.448296+0000","last_clean":"2024-04-22T13:27:09.448296+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:27:09.448296+0000","last_undegraded":"2024-04-22T13:27:09.448296+0000","last_fullsized":"2024-04-22T13:27:09.448296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564063+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:27:09.564063+0000","last_peered":"2024-04-22T13:27:09.564063+0000","last_clean":"2024-04-22T13:27:09.564063+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:27:09.564063+0000","last_undegraded":"2024-04-22T13:27:09.564063+0000","last_fullsized":"2024-04-22T13:27:09.564063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540227+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:27:09.540227+0000","last_peered":"2024-04-22T13:27:09.540227+0000","last_clean":"2024-04-22T13:27:09.540227+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:27:09.540227+0000","last_undegraded":"2024-04-22T13:27:09.540227+0000","last_fullsized":"2024-04-22T13:27:09.540227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563416+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:27:09.563416+0000","last_peered":"2024-04-22T13:27:09.563416+0000","last_clean":"2024-04-22T13:27:09.563416+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:27:09.563416+0000","last_undegraded":"2024-04-22T13:27:09.563416+0000","last_fullsized":"2024-04-22T13:27:09.563416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448852+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:27:09.448852+0000","last_peered":"2024-04-22T13:27:09.448852+0000","last_clean":"2024-04-22T13:27:09.448852+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:27:09.448852+0000","last_undegraded":"2024-04-22T13:27:09.448852+0000","last_fullsized":"2024-04-22T13:27:09.448852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564466+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:27:09.564466+0000","last_peered":"2024-04-22T13:27:09.564466+0000","last_clean":"2024-04-22T13:27:09.564466+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:27:09.564466+0000","last_undegraded":"2024-04-22T13:27:09.564466+0000","last_fullsized":"2024-04-22T13:27:09.564466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563562+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:27:09.563562+0000","last_peered":"2024-04-22T13:27:09.563562+0000","last_clean":"2024-04-22T13:27:09.563562+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:27:09.563562+0000","last_undegraded":"2024-04-22T13:27:09.563562+0000","last_fullsized":"2024-04-22T13:27:09.563562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563593+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:27:09.563593+0000","last_peered":"2024-04-22T13:27:09.563593+0000","last_clean":"2024-04-22T13:27:09.563593+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:27:09.563593+0000","last_undegraded":"2024-04-22T13:27:09.563593+0000","last_fullsized":"2024-04-22T13:27:09.563593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563776+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:27:09.563776+0000","last_peered":"2024-04-22T13:27:09.563776+0000","last_clean":"2024-04-22T13:27:09.563776+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:27:09.563776+0000","last_undegraded":"2024-04-22T13:27:09.563776+0000","last_fullsized":"2024-04-22T13:27:09.563776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562794+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:27:09.562794+0000","last_peered":"2024-04-22T13:27:09.562794+0000","last_clean":"2024-04-22T13:27:09.562794+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:27:09.562794+0000","last_undegraded":"2024-04-22T13:27:09.562794+0000","last_fullsized":"2024-04-22T13:27:09.562794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562713+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:27:09.562713+0000","last_peered":"2024-04-22T13:27:09.562713+0000","last_clean":"2024-04-22T13:27:09.562713+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:27:09.562713+0000","last_undegraded":"2024-04-22T13:27:09.562713+0000","last_fullsized":"2024-04-22T13:27:09.562713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563250+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.563250+0000","last_peered":"2024-04-22T13:27:09.563250+0000","last_clean":"2024-04-22T13:27:09.563250+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:27:09.563250+0000","last_undegraded":"2024-04-22T13:27:09.563250+0000","last_fullsized":"2024-04-22T13:27:09.563250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446872+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:27:09.446872+0000","last_peered":"2024-04-22T13:27:09.446872+0000","last_clean":"2024-04-22T13:27:09.446872+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:27:09.446872+0000","last_undegraded":"2024-04-22T13:27:09.446872+0000","last_fullsized":"2024-04-22T13:27:09.446872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563565+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:27:09.563565+0000","last_peered":"2024-04-22T13:27:09.563565+0000","last_clean":"2024-04-22T13:27:09.563565+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:27:09.563565+0000","last_undegraded":"2024-04-22T13:27:09.563565+0000","last_fullsized":"2024-04-22T13:27:09.563565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.409929+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:27:19.409929+0000","last_peered":"2024-04-22T13:27:19.409929+0000","last_clean":"2024-04-22T13:27:19.409929+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:27:19.409929+0000","last_undegraded":"2024-04-22T13:27:19.409929+0000","last_fullsized":"2024-04-22T13:27:19.409929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541086+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:27:09.541086+0000","last_peered":"2024-04-22T13:27:09.541086+0000","last_clean":"2024-04-22T13:27:09.541086+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:27:09.541086+0000","last_undegraded":"2024-04-22T13:27:09.541086+0000","last_fullsized":"2024-04-22T13:27:09.541086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541272+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:27:09.541272+0000","last_peered":"2024-04-22T13:27:09.541272+0000","last_clean":"2024-04-22T13:27:09.541272+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:27:09.541272+0000","last_undegraded":"2024-04-22T13:27:09.541272+0000","last_fullsized":"2024-04-22T13:27:09.541272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564850+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:27:09.564850+0000","last_peered":"2024-04-22T13:27:09.564850+0000","last_clean":"2024-04-22T13:27:09.564850+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:27:09.564850+0000","last_undegraded":"2024-04-22T13:27:09.564850+0000","last_fullsized":"2024-04-22T13:27:09.564850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'13","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563038+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:27:09.563038+0000","last_peered":"2024-04-22T13:27:09.563038+0000","last_clean":"2024-04-22T13:27:09.563038+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:27:09.563038+0000","last_undegraded":"2024-04-22T13:27:09.563038+0000","last_fullsized":"2024-04-22T13:27:09.563038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562953+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:27:09.562953+0000","last_peered":"2024-04-22T13:27:09.562953+0000","last_clean":"2024-04-22T13:27:09.562953+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:27:09.562953+0000","last_undegraded":"2024-04-22T13:27:09.562953+0000","last_fullsized":"2024-04-22T13:27:09.562953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.451042+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:27:09.451042+0000","last_peered":"2024-04-22T13:27:09.451042+0000","last_clean":"2024-04-22T13:27:09.451042+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:27:09.451042+0000","last_undegraded":"2024-04-22T13:27:09.451042+0000","last_fullsized":"2024-04-22T13:27:09.451042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547806+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:27:09.547806+0000","last_peered":"2024-04-22T13:27:09.547806+0000","last_clean":"2024-04-22T13:27:09.547806+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:27:09.547806+0000","last_undegraded":"2024-04-22T13:27:09.547806+0000","last_fullsized":"2024-04-22T13:27:09.547806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.411305+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:27:19.411305+0000","last_peered":"2024-04-22T13:27:19.411305+0000","last_clean":"2024-04-22T13:27:19.411305+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:27:19.411305+0000","last_undegraded":"2024-04-22T13:27:19.411305+0000","last_fullsized":"2024-04-22T13:27:19.411305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450920+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:27:09.450920+0000","last_peered":"2024-04-22T13:27:09.450920+0000","last_clean":"2024-04-22T13:27:09.450920+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:27:09.450920+0000","last_undegraded":"2024-04-22T13:27:09.450920+0000","last_fullsized":"2024-04-22T13:27:09.450920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448243+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:27:09.448243+0000","last_peered":"2024-04-22T13:27:09.448243+0000","last_clean":"2024-04-22T13:27:09.448243+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:27:09.448243+0000","last_undegraded":"2024-04-22T13:27:09.448243+0000","last_fullsized":"2024-04-22T13:27:09.448243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547784+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:27:09.547784+0000","last_peered":"2024-04-22T13:27:09.547784+0000","last_clean":"2024-04-22T13:27:09.547784+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:27:09.547784+0000","last_undegraded":"2024-04-22T13:27:09.547784+0000","last_fullsized":"2024-04-22T13:27:09.547784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562955+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:27:09.562955+0000","last_peered":"2024-04-22T13:27:09.562955+0000","last_clean":"2024-04-22T13:27:09.562955+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:27:09.562955+0000","last_undegraded":"2024-04-22T13:27:09.562955+0000","last_fullsized":"2024-04-22T13:27:09.562955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540293+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:27:09.540293+0000","last_peered":"2024-04-22T13:27:09.540293+0000","last_clean":"2024-04-22T13:27:09.540293+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:27:09.540293+0000","last_undegraded":"2024-04-22T13:27:09.540293+0000","last_fullsized":"2024-04-22T13:27:09.540293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547473+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:27:09.547473+0000","last_peered":"2024-04-22T13:27:09.547473+0000","last_clean":"2024-04-22T13:27:09.547473+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:27:09.547473+0000","last_undegraded":"2024-04-22T13:27:09.547473+0000","last_fullsized":"2024-04-22T13:27:09.547473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446526+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:27:09.446526+0000","last_peered":"2024-04-22T13:27:09.446526+0000","last_clean":"2024-04-22T13:27:09.446526+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:27:09.446526+0000","last_undegraded":"2024-04-22T13:27:09.446526+0000","last_fullsized":"2024-04-22T13:27:09.446526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":276,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:20.411992+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:27:20.411992+0000","last_peered":"2024-04-22T13:27:20.411992+0000","last_clean":"2024-04-22T13:27:20.411992+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:27:20.411992+0000","last_undegraded":"2024-04-22T13:27:20.411992+0000","last_fullsized":"2024-04-22T13:27:20.411992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":248,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563427+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:27:09.563427+0000","last_peered":"2024-04-22T13:27:09.563427+0000","last_clean":"2024-04-22T13:27:09.563427+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:27:09.563427+0000","last_undegraded":"2024-04-22T13:27:09.563427+0000","last_fullsized":"2024-04-22T13:27:09.563427+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564441+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:27:09.564441+0000","last_peered":"2024-04-22T13:27:09.564441+0000","last_clean":"2024-04-22T13:27:09.564441+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:27:09.564441+0000","last_undegraded":"2024-04-22T13:27:09.564441+0000","last_fullsized":"2024-04-22T13:27:09.564441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564869+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:27:09.564869+0000","last_peered":"2024-04-22T13:27:09.564869+0000","last_clean":"2024-04-22T13:27:09.564869+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:27:09.564869+0000","last_undegraded":"2024-04-22T13:27:09.564869+0000","last_fullsized":"2024-04-22T13:27:09.564869+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547887+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:27:09.547887+0000","last_peered":"2024-04-22T13:27:09.547887+0000","last_clean":"2024-04-22T13:27:09.547887+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:27:09.547887+0000","last_undegraded":"2024-04-22T13:27:09.547887+0000","last_fullsized":"2024-04-22T13:27:09.547887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562984+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:27:09.562984+0000","last_peered":"2024-04-22T13:27:09.562984+0000","last_clean":"2024-04-22T13:27:09.562984+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:27:09.562984+0000","last_undegraded":"2024-04-22T13:27:09.562984+0000","last_fullsized":"2024-04-22T13:27:09.562984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":319,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:20.417834+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:27:20.417834+0000","last_peered":"2024-04-22T13:27:20.417834+0000","last_clean":"2024-04-22T13:27:20.417834+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:27:20.417834+0000","last_undegraded":"2024-04-22T13:27:20.417834+0000","last_fullsized":"2024-04-22T13:27:20.417834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68543,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":198,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450855+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:27:09.450855+0000","last_peered":"2024-04-22T13:27:09.450855+0000","last_clean":"2024-04-22T13:27:09.450855+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:27:09.450855+0000","last_undegraded":"2024-04-22T13:27:09.450855+0000","last_fullsized":"2024-04-22T13:27:09.450855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563392+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:27:09.563392+0000","last_peered":"2024-04-22T13:27:09.563392+0000","last_clean":"2024-04-22T13:27:09.563392+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:27:09.563392+0000","last_undegraded":"2024-04-22T13:27:09.563392+0000","last_fullsized":"2024-04-22T13:27:09.563392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562910+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:27:09.562910+0000","last_peered":"2024-04-22T13:27:09.562910+0000","last_clean":"2024-04-22T13:27:09.562910+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:27:09.562910+0000","last_undegraded":"2024-04-22T13:27:09.562910+0000","last_fullsized":"2024-04-22T13:27:09.562910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547599+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:27:09.547599+0000","last_peered":"2024-04-22T13:27:09.547599+0000","last_clean":"2024-04-22T13:27:09.547599+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:27:09.547599+0000","last_undegraded":"2024-04-22T13:27:09.547599+0000","last_fullsized":"2024-04-22T13:27:09.547599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564267+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:27:09.564267+0000","last_peered":"2024-04-22T13:27:09.564267+0000","last_clean":"2024-04-22T13:27:09.564267+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:27:09.564267+0000","last_undegraded":"2024-04-22T13:27:09.564267+0000","last_fullsized":"2024-04-22T13:27:09.564267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.410628+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:27:19.410628+0000","last_peered":"2024-04-22T13:27:19.410628+0000","last_clean":"2024-04-22T13:27:19.410628+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:27:19.410628+0000","last_undegraded":"2024-04-22T13:27:19.410628+0000","last_fullsized":"2024-04-22T13:27:19.410628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563985+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:27:09.563985+0000","last_peered":"2024-04-22T13:27:09.563985+0000","last_clean":"2024-04-22T13:27:09.563985+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:27:09.563985+0000","last_undegraded":"2024-04-22T13:27:09.563985+0000","last_fullsized":"2024-04-22T13:27:09.563985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450657+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:27:09.450657+0000","last_peered":"2024-04-22T13:27:09.450657+0000","last_clean":"2024-04-22T13:27:09.450657+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:27:09.450657+0000","last_undegraded":"2024-04-22T13:27:09.450657+0000","last_fullsized":"2024-04-22T13:27:09.450657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547522+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:27:09.547522+0000","last_peered":"2024-04-22T13:27:09.547522+0000","last_clean":"2024-04-22T13:27:09.547522+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:27:09.547522+0000","last_undegraded":"2024-04-22T13:27:09.547522+0000","last_fullsized":"2024-04-22T13:27:09.547522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450885+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:27:09.450885+0000","last_peered":"2024-04-22T13:27:09.450885+0000","last_clean":"2024-04-22T13:27:09.450885+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:27:09.450885+0000","last_undegraded":"2024-04-22T13:27:09.450885+0000","last_fullsized":"2024-04-22T13:27:09.450885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563035+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:27:09.563035+0000","last_peered":"2024-04-22T13:27:09.563035+0000","last_clean":"2024-04-22T13:27:09.563035+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:27:09.563035+0000","last_undegraded":"2024-04-22T13:27:09.563035+0000","last_fullsized":"2024-04-22T13:27:09.563035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446773+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:27:09.446773+0000","last_peered":"2024-04-22T13:27:09.446773+0000","last_clean":"2024-04-22T13:27:09.446773+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:27:09.446773+0000","last_undegraded":"2024-04-22T13:27:09.446773+0000","last_fullsized":"2024-04-22T13:27:09.446773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448737+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:27:09.448737+0000","last_peered":"2024-04-22T13:27:09.448737+0000","last_clean":"2024-04-22T13:27:09.448737+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:27:09.448737+0000","last_undegraded":"2024-04-22T13:27:09.448737+0000","last_fullsized":"2024-04-22T13:27:09.448737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.410621+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:27:19.410621+0000","last_peered":"2024-04-22T13:27:19.410621+0000","last_clean":"2024-04-22T13:27:19.410621+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:27:19.410621+0000","last_undegraded":"2024-04-22T13:27:19.410621+0000","last_fullsized":"2024-04-22T13:27:19.410621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448555+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:27:09.448555+0000","last_peered":"2024-04-22T13:27:09.448555+0000","last_clean":"2024-04-22T13:27:09.448555+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:27:09.448555+0000","last_undegraded":"2024-04-22T13:27:09.448555+0000","last_fullsized":"2024-04-22T13:27:09.448555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446492+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:27:09.446492+0000","last_peered":"2024-04-22T13:27:09.446492+0000","last_clean":"2024-04-22T13:27:09.446492+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:27:09.446492+0000","last_undegraded":"2024-04-22T13:27:09.446492+0000","last_fullsized":"2024-04-22T13:27:09.446492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.409823+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:27:19.409823+0000","last_peered":"2024-04-22T13:27:19.409823+0000","last_clean":"2024-04-22T13:27:19.409823+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:27:19.409823+0000","last_undegraded":"2024-04-22T13:27:19.409823+0000","last_fullsized":"2024-04-22T13:27:19.409823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540620+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:27:09.540620+0000","last_peered":"2024-04-22T13:27:09.540620+0000","last_clean":"2024-04-22T13:27:09.540620+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:27:09.540620+0000","last_undegraded":"2024-04-22T13:27:09.540620+0000","last_fullsized":"2024-04-22T13:27:09.540620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563518+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:27:09.563518+0000","last_peered":"2024-04-22T13:27:09.563518+0000","last_clean":"2024-04-22T13:27:09.563518+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:27:09.563518+0000","last_undegraded":"2024-04-22T13:27:09.563518+0000","last_fullsized":"2024-04-22T13:27:09.563518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":403,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:11.559108+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:27:11.559108+0000","last_peered":"2024-04-22T13:27:11.559108+0000","last_clean":"2024-04-22T13:27:11.559108+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:27:11.559108+0000","last_undegraded":"2024-04-22T13:27:11.559108+0000","last_fullsized":"2024-04-22T13:27:11.559108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562824+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:27:09.562824+0000","last_peered":"2024-04-22T13:27:09.562824+0000","last_clean":"2024-04-22T13:27:09.562824+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:27:09.562824+0000","last_undegraded":"2024-04-22T13:27:09.562824+0000","last_fullsized":"2024-04-22T13:27:09.562824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564286+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:27:09.564286+0000","last_peered":"2024-04-22T13:27:09.564286+0000","last_clean":"2024-04-22T13:27:09.564286+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:27:09.564286+0000","last_undegraded":"2024-04-22T13:27:09.564286+0000","last_fullsized":"2024-04-22T13:27:09.564286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450990+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:27:09.450990+0000","last_peered":"2024-04-22T13:27:09.450990+0000","last_clean":"2024-04-22T13:27:09.450990+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:27:09.450990+0000","last_undegraded":"2024-04-22T13:27:09.450990+0000","last_fullsized":"2024-04-22T13:27:09.450990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563277+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:27:09.563277+0000","last_peered":"2024-04-22T13:27:09.563277+0000","last_clean":"2024-04-22T13:27:09.563277+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:27:09.563277+0000","last_undegraded":"2024-04-22T13:27:09.563277+0000","last_fullsized":"2024-04-22T13:27:09.563277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'103","reported_seq":217,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:18.206290+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:27:18.206290+0000","last_peered":"2024-04-22T13:27:18.206290+0000","last_clean":"2024-04-22T13:27:18.206290+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:27:18.206290+0000","last_undegraded":"2024-04-22T13:27:18.206290+0000","last_fullsized":"2024-04-22T13:27:18.206290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541266+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:27:09.541266+0000","last_peered":"2024-04-22T13:27:09.541266+0000","last_clean":"2024-04-22T13:27:09.541266+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:27:09.541266+0000","last_undegraded":"2024-04-22T13:27:09.541266+0000","last_fullsized":"2024-04-22T13:27:09.541266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563131+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:27:09.563131+0000","last_peered":"2024-04-22T13:27:09.563131+0000","last_clean":"2024-04-22T13:27:09.563131+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:27:09.563131+0000","last_undegraded":"2024-04-22T13:27:09.563131+0000","last_fullsized":"2024-04-22T13:27:09.563131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564847+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:27:09.564847+0000","last_peered":"2024-04-22T13:27:09.564847+0000","last_clean":"2024-04-22T13:27:09.564847+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:27:09.564847+0000","last_undegraded":"2024-04-22T13:27:09.564847+0000","last_fullsized":"2024-04-22T13:27:09.564847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562881+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:27:09.562881+0000","last_peered":"2024-04-22T13:27:09.562881+0000","last_clean":"2024-04-22T13:27:09.562881+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:27:09.562881+0000","last_undegraded":"2024-04-22T13:27:09.562881+0000","last_fullsized":"2024-04-22T13:27:09.562881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450691+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:27:09.450691+0000","last_peered":"2024-04-22T13:27:09.450691+0000","last_clean":"2024-04-22T13:27:09.450691+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:27:09.450691+0000","last_undegraded":"2024-04-22T13:27:09.450691+0000","last_fullsized":"2024-04-22T13:27:09.450691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540359+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:27:09.540359+0000","last_peered":"2024-04-22T13:27:09.540359+0000","last_clean":"2024-04-22T13:27:09.540359+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:27:09.540359+0000","last_undegraded":"2024-04-22T13:27:09.540359+0000","last_fullsized":"2024-04-22T13:27:09.540359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562679+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:27:09.562679+0000","last_peered":"2024-04-22T13:27:09.562679+0000","last_clean":"2024-04-22T13:27:09.562679+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:27:09.562679+0000","last_undegraded":"2024-04-22T13:27:09.562679+0000","last_fullsized":"2024-04-22T13:27:09.562679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562789+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:27:09.562789+0000","last_peered":"2024-04-22T13:27:09.562789+0000","last_clean":"2024-04-22T13:27:09.562789+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:27:09.562789+0000","last_undegraded":"2024-04-22T13:27:09.562789+0000","last_fullsized":"2024-04-22T13:27:09.562789+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540822+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:27:09.540822+0000","last_peered":"2024-04-22T13:27:09.540822+0000","last_clean":"2024-04-22T13:27:09.540822+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:27:09.540822+0000","last_undegraded":"2024-04-22T13:27:09.540822+0000","last_fullsized":"2024-04-22T13:27:09.540822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450996+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:27:09.450996+0000","last_peered":"2024-04-22T13:27:09.450996+0000","last_clean":"2024-04-22T13:27:09.450996+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:27:09.450996+0000","last_undegraded":"2024-04-22T13:27:09.450996+0000","last_fullsized":"2024-04-22T13:27:09.450996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564270+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:27:09.564270+0000","last_peered":"2024-04-22T13:27:09.564270+0000","last_clean":"2024-04-22T13:27:09.564270+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:27:09.564270+0000","last_undegraded":"2024-04-22T13:27:09.564270+0000","last_fullsized":"2024-04-22T13:27:09.564270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.410927+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:27:19.410927+0000","last_peered":"2024-04-22T13:27:19.410927+0000","last_clean":"2024-04-22T13:27:19.410927+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:27:19.410927+0000","last_undegraded":"2024-04-22T13:27:19.410927+0000","last_fullsized":"2024-04-22T13:27:19.410927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564284+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.564284+0000","last_peered":"2024-04-22T13:27:09.564284+0000","last_clean":"2024-04-22T13:27:09.564284+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:27:09.564284+0000","last_undegraded":"2024-04-22T13:27:09.564284+0000","last_fullsized":"2024-04-22T13:27:09.564284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'67","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:17.859417+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:27:17.859417+0000","last_peered":"2024-04-22T13:27:17.859417+0000","last_clean":"2024-04-22T13:27:17.859417+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:27:17.859417+0000","last_undegraded":"2024-04-22T13:27:17.859417+0000","last_fullsized":"2024-04-22T13:27:17.859417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564302+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:27:09.564302+0000","last_peered":"2024-04-22T13:27:09.564302+0000","last_clean":"2024-04-22T13:27:09.564302+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:27:09.564302+0000","last_undegraded":"2024-04-22T13:27:09.564302+0000","last_fullsized":"2024-04-22T13:27:09.564302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563457+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:27:09.563457+0000","last_peered":"2024-04-22T13:27:09.563457+0000","last_clean":"2024-04-22T13:27:09.563457+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:27:09.563457+0000","last_undegraded":"2024-04-22T13:27:09.563457+0000","last_fullsized":"2024-04-22T13:27:09.563457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564533+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:27:09.564533+0000","last_peered":"2024-04-22T13:27:09.564533+0000","last_clean":"2024-04-22T13:27:09.564533+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:27:09.564533+0000","last_undegraded":"2024-04-22T13:27:09.564533+0000","last_fullsized":"2024-04-22T13:27:09.564533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562817+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:27:09.562817+0000","last_peered":"2024-04-22T13:27:09.562817+0000","last_clean":"2024-04-22T13:27:09.562817+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:27:09.562817+0000","last_undegraded":"2024-04-22T13:27:09.562817+0000","last_fullsized":"2024-04-22T13:27:09.562817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563366+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:27:09.563366+0000","last_peered":"2024-04-22T13:27:09.563366+0000","last_clean":"2024-04-22T13:27:09.563366+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:27:09.563366+0000","last_undegraded":"2024-04-22T13:27:09.563366+0000","last_fullsized":"2024-04-22T13:27:09.563366+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563322+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:27:09.563322+0000","last_peered":"2024-04-22T13:27:09.563322+0000","last_clean":"2024-04-22T13:27:09.563322+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:27:09.563322+0000","last_undegraded":"2024-04-22T13:27:09.563322+0000","last_fullsized":"2024-04-22T13:27:09.563322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564345+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:27:09.564345+0000","last_peered":"2024-04-22T13:27:09.564345+0000","last_clean":"2024-04-22T13:27:09.564345+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:27:09.564345+0000","last_undegraded":"2024-04-22T13:27:09.564345+0000","last_fullsized":"2024-04-22T13:27:09.564345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563441+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:27:09.563441+0000","last_peered":"2024-04-22T13:27:09.563441+0000","last_clean":"2024-04-22T13:27:09.563441+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:27:09.563441+0000","last_undegraded":"2024-04-22T13:27:09.563441+0000","last_fullsized":"2024-04-22T13:27:09.563441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563022+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:27:09.563022+0000","last_peered":"2024-04-22T13:27:09.563022+0000","last_clean":"2024-04-22T13:27:09.563022+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:27:09.563022+0000","last_undegraded":"2024-04-22T13:27:09.563022+0000","last_fullsized":"2024-04-22T13:27:09.563022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563680+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:27:09.563680+0000","last_peered":"2024-04-22T13:27:09.563680+0000","last_clean":"2024-04-22T13:27:09.563680+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:27:09.563680+0000","last_undegraded":"2024-04-22T13:27:09.563680+0000","last_fullsized":"2024-04-22T13:27:09.563680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541195+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:27:09.541195+0000","last_peered":"2024-04-22T13:27:09.541195+0000","last_clean":"2024-04-22T13:27:09.541195+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:27:09.541195+0000","last_undegraded":"2024-04-22T13:27:09.541195+0000","last_fullsized":"2024-04-22T13:27:09.541195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562758+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:27:09.562758+0000","last_peered":"2024-04-22T13:27:09.562758+0000","last_clean":"2024-04-22T13:27:09.562758+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:27:09.562758+0000","last_undegraded":"2024-04-22T13:27:09.562758+0000","last_fullsized":"2024-04-22T13:27:09.562758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540273+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:27:09.540273+0000","last_peered":"2024-04-22T13:27:09.540273+0000","last_clean":"2024-04-22T13:27:09.540273+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:27:09.540273+0000","last_undegraded":"2024-04-22T13:27:09.540273+0000","last_fullsized":"2024-04-22T13:27:09.540273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562941+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:27:09.562941+0000","last_peered":"2024-04-22T13:27:09.562941+0000","last_clean":"2024-04-22T13:27:09.562941+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:27:09.562941+0000","last_undegraded":"2024-04-22T13:27:09.562941+0000","last_fullsized":"2024-04-22T13:27:09.562941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563605+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:27:09.563605+0000","last_peered":"2024-04-22T13:27:09.563605+0000","last_clean":"2024-04-22T13:27:09.563605+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:27:09.563605+0000","last_undegraded":"2024-04-22T13:27:09.563605+0000","last_fullsized":"2024-04-22T13:27:09.563605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562834+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:27:09.562834+0000","last_peered":"2024-04-22T13:27:09.562834+0000","last_clean":"2024-04-22T13:27:09.562834+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:27:09.562834+0000","last_undegraded":"2024-04-22T13:27:09.562834+0000","last_fullsized":"2024-04-22T13:27:09.562834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799208,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1635,"num_read_kb":1041,"num_write":503,"num_write_kb":2383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2397624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":382,"ondisk_log_size":382,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":485517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514538,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41620,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703532,"statfs":{"total":95995035648,"available":95952416768,"internally_reserved":0,"allocated":2052096,"data_stored":1472231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678064,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40892,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704260,"statfs":{"total":95995035648,"available":95953162240,"internally_reserved":0,"allocated":1429504,"data_stored":877573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939702,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36728,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708424,"statfs":{"total":95995035648,"available":95957426176,"internally_reserved":0,"allocated":1425408,"data_stored":877951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41088,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704064,"statfs":{"total":95995035648,"available":95952961536,"internally_reserved":0,"allocated":1630208,"data_stored":1078567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":554312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":486503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":555763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41336,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703816,"statfs":{"total":95995035648,"available":95952707584,"internally_reserved":0,"allocated":1761280,"data_stored":1205347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37384,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707768,"statfs":{"total":95995035648,"available":95956754432,"internally_reserved":0,"allocated":1974272,"data_stored":1405695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":398511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:27:21.488 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-22T13:27:21.488 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:27:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 -- 172.21.15.5:0/1917638483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80108060 msgr2=0x7fac80108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 --2- 172.21.15.5:0/1917638483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80108060 0x7fac80108440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009930 tx=0x7fac7c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 -- 172.21.15.5:0/1917638483 shutdown_connections 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 --2- 172.21.15.5:0/1917638483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac8006b000 0x7fac8010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 --2- 172.21.15.5:0/1917638483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8006a640 0x7fac8006aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 --2- 172.21.15.5:0/1917638483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac80108060 0x7fac80108440 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 -- 172.21.15.5:0/1917638483 >> 172.21.15.5:0/1917638483 conn(0x7fac80077c40 msgr2=0x7fac80078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 -- 172.21.15.5:0/1917638483 shutdown_connections 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.630+0000 7fac87f8f640 1 -- 172.21.15.5:0/1917638483 wait complete. 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 Processor -- start 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 -- start start 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac8006a640 0x7fac80198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 0x7fac80198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac80108060 0x7fac8019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac80112250 con 0x7fac8006b000 2024-04-22T13:27:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac801120d0 con 0x7fac8006a640 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac87f8f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac801123d0 con 0x7fac80108060 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac8552a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 0x7fac80198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac8552a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 0x7fac80198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46798/0 (socket says 172.21.15.5:46798) 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac8552a640 1 -- 172.21.15.5:0/573759993 learned_addr learned my addr 172.21.15.5:0/573759993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.631+0000 7fac85d2b640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac8006a640 0x7fac80198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac8552a640 1 -- 172.21.15.5:0/573759993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac80108060 msgr2=0x7fac8019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac8552a640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac80108060 0x7fac8019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac8552a640 1 -- 172.21.15.5:0/573759993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac8006a640 msgr2=0x7fac80198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac8552a640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac8006a640 0x7fac80198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac8552a640 1 -- 172.21.15.5:0/573759993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8019d6f0 con 0x7fac8006b000 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac8552a640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 0x7fac80198c90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fac7000b4a0 tx=0x7fac7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70014070 con 0x7fac8006b000 2024-04-22T13:27:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac70010ac0 con 0x7fac8006b000 2024-04-22T13:27:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8019d9e0 con 0x7fac8006b000 2024-04-22T13:27:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70017c30 con 0x7fac8006b000 2024-04-22T13:27:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.632+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a5270 con 0x7fac8006b000 2024-04-22T13:27:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.633+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac80103650 con 0x7fac8006b000 2024-04-22T13:27:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.634+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fac70017070 con 0x7fac8006b000 2024-04-22T13:27:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.635+0000 7fac76ffd640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac6c0a3ec0 0x7fac6c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.635+0000 7fac85d2b640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac6c0a3ec0 0x7fac6c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.635+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fac700cfff0 con 0x7fac8006b000 2024-04-22T13:27:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.635+0000 7fac85d2b640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac6c0a3ec0 0x7fac6c0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009900 tx=0x7fac7c005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.639+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac7009c580 con 0x7fac8006b000 2024-04-22T13:27:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.821+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac80062390 con 0x7fac6c0a3ec0 2024-04-22T13:27:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.828+0000 7fac76ffd640 1 -- 172.21.15.5:0/573759993 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257819 (secure 0 0 0) 0x7fac80062390 con 0x7fac6c0a3ec0 2024-04-22T13:27:21.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-22T13:27:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac6c0a3ec0 msgr2=0x7fac6c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac6c0a3ec0 0x7fac6c0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009900 tx=0x7fac7c005a20 comp rx=0 tx=0).stop 2024-04-22T13:27:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 msgr2=0x7fac80198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 0x7fac80198c90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fac7000b4a0 tx=0x7fac7000b960 comp rx=0 tx=0).stop 2024-04-22T13:27:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 shutdown_connections 2024-04-22T13:27:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac8006b000 0x7fac80198c90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fac6c0a3ec0 0x7fac6c0a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac80108060 0x7fac8019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 --2- 172.21.15.5:0/573759993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac8006a640 0x7fac80198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 >> 172.21.15.5:0/573759993 conn(0x7fac80077c40 msgr2=0x7fac800ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 shutdown_connections 2024-04-22T13:27:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.831+0000 7fac87f8f640 1 -- 172.21.15.5:0/573759993 wait complete. 2024-04-22T13:27:21.838 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-22T13:27:21.461967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389810,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1872,"num_read_kb":1371,"num_write":706,"num_write_kb":5323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1032,"ondisk_log_size":1032,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":423948,"kb_used_data":16176,"kb_used_omap":18,"kb_used_meta":407277,"kb_avail":1124517876,"statfs":{"total":1151940427776,"available":1151506305024,"internally_reserved":0,"allocated":16564224,"data_stored":9968250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":417051962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":461,"num_read_kb":501,"num_write":157,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.990056"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563135+0000","last_change":"2024-04-22T13:25:10.901247+0000","last_active":"2024-04-22T13:27:09.563135+0000","last_peered":"2024-04-22T13:27:09.563135+0000","last_clean":"2024-04-22T13:27:09.563135+0000","last_became_active":"2024-04-22T13:25:10.900887+0000","last_became_peered":"2024-04-22T13:25:10.900887+0000","last_unstale":"2024-04-22T13:27:09.563135+0000","last_undegraded":"2024-04-22T13:27:09.563135+0000","last_fullsized":"2024-04-22T13:27:09.563135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:16.236645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564703+0000","last_change":"2024-04-22T13:25:10.906347+0000","last_active":"2024-04-22T13:27:09.564703+0000","last_peered":"2024-04-22T13:27:09.564703+0000","last_clean":"2024-04-22T13:27:09.564703+0000","last_became_active":"2024-04-22T13:25:10.906018+0000","last_became_peered":"2024-04-22T13:25:10.906018+0000","last_unstale":"2024-04-22T13:27:09.564703+0000","last_undegraded":"2024-04-22T13:27:09.564703+0000","last_fullsized":"2024-04-22T13:27:09.564703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:45:14.471830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564676+0000","last_change":"2024-04-22T13:25:10.917007+0000","last_active":"2024-04-22T13:27:09.564676+0000","last_peered":"2024-04-22T13:27:09.564676+0000","last_clean":"2024-04-22T13:27:09.564676+0000","last_became_active":"2024-04-22T13:25:10.916558+0000","last_became_peered":"2024-04-22T13:25:10.916558+0000","last_unstale":"2024-04-22T13:27:09.564676+0000","last_undegraded":"2024-04-22T13:27:09.564676+0000","last_fullsized":"2024-04-22T13:27:09.564676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:51:39.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563774+0000","last_change":"2024-04-22T13:25:10.906049+0000","last_active":"2024-04-22T13:27:09.563774+0000","last_peered":"2024-04-22T13:27:09.563774+0000","last_clean":"2024-04-22T13:27:09.563774+0000","last_became_active":"2024-04-22T13:25:10.905075+0000","last_became_peered":"2024-04-22T13:25:10.905075+0000","last_unstale":"2024-04-22T13:27:09.563774+0000","last_undegraded":"2024-04-22T13:27:09.563774+0000","last_fullsized":"2024-04-22T13:27:09.563774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:33:33.358766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450924+0000","last_change":"2024-04-22T13:25:10.916567+0000","last_active":"2024-04-22T13:27:09.450924+0000","last_peered":"2024-04-22T13:27:09.450924+0000","last_clean":"2024-04-22T13:27:09.450924+0000","last_became_active":"2024-04-22T13:25:10.915413+0000","last_became_peered":"2024-04-22T13:25:10.915413+0000","last_unstale":"2024-04-22T13:27:09.450924+0000","last_undegraded":"2024-04-22T13:27:09.450924+0000","last_fullsized":"2024-04-22T13:27:09.450924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:11.919552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448296+0000","last_change":"2024-04-22T13:25:10.900378+0000","last_active":"2024-04-22T13:27:09.448296+0000","last_peered":"2024-04-22T13:27:09.448296+0000","last_clean":"2024-04-22T13:27:09.448296+0000","last_became_active":"2024-04-22T13:25:10.897192+0000","last_became_peered":"2024-04-22T13:25:10.897192+0000","last_unstale":"2024-04-22T13:27:09.448296+0000","last_undegraded":"2024-04-22T13:27:09.448296+0000","last_fullsized":"2024-04-22T13:27:09.448296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:01:38.950236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564063+0000","last_change":"2024-04-22T13:25:10.895786+0000","last_active":"2024-04-22T13:27:09.564063+0000","last_peered":"2024-04-22T13:27:09.564063+0000","last_clean":"2024-04-22T13:27:09.564063+0000","last_became_active":"2024-04-22T13:25:10.895397+0000","last_became_peered":"2024-04-22T13:25:10.895397+0000","last_unstale":"2024-04-22T13:27:09.564063+0000","last_undegraded":"2024-04-22T13:27:09.564063+0000","last_fullsized":"2024-04-22T13:27:09.564063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:54:36.576194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540227+0000","last_change":"2024-04-22T13:25:10.909278+0000","last_active":"2024-04-22T13:27:09.540227+0000","last_peered":"2024-04-22T13:27:09.540227+0000","last_clean":"2024-04-22T13:27:09.540227+0000","last_became_active":"2024-04-22T13:25:10.908471+0000","last_became_peered":"2024-04-22T13:25:10.908471+0000","last_unstale":"2024-04-22T13:27:09.540227+0000","last_undegraded":"2024-04-22T13:27:09.540227+0000","last_fullsized":"2024-04-22T13:27:09.540227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:12.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563416+0000","last_change":"2024-04-22T13:25:10.909396+0000","last_active":"2024-04-22T13:27:09.563416+0000","last_peered":"2024-04-22T13:27:09.563416+0000","last_clean":"2024-04-22T13:27:09.563416+0000","last_became_active":"2024-04-22T13:25:10.909041+0000","last_became_peered":"2024-04-22T13:25:10.909041+0000","last_unstale":"2024-04-22T13:27:09.563416+0000","last_undegraded":"2024-04-22T13:27:09.563416+0000","last_fullsized":"2024-04-22T13:27:09.563416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:13.635853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448852+0000","last_change":"2024-04-22T13:25:10.903631+0000","last_active":"2024-04-22T13:27:09.448852+0000","last_peered":"2024-04-22T13:27:09.448852+0000","last_clean":"2024-04-22T13:27:09.448852+0000","last_became_active":"2024-04-22T13:25:10.903270+0000","last_became_peered":"2024-04-22T13:25:10.903270+0000","last_unstale":"2024-04-22T13:27:09.448852+0000","last_undegraded":"2024-04-22T13:27:09.448852+0000","last_fullsized":"2024-04-22T13:27:09.448852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:01.535409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564466+0000","last_change":"2024-04-22T13:25:10.904004+0000","last_active":"2024-04-22T13:27:09.564466+0000","last_peered":"2024-04-22T13:27:09.564466+0000","last_clean":"2024-04-22T13:27:09.564466+0000","last_became_active":"2024-04-22T13:25:10.903671+0000","last_became_peered":"2024-04-22T13:25:10.903671+0000","last_unstale":"2024-04-22T13:27:09.564466+0000","last_undegraded":"2024-04-22T13:27:09.564466+0000","last_fullsized":"2024-04-22T13:27:09.564466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:10:55.688219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563562+0000","last_change":"2024-04-22T13:25:10.906599+0000","last_active":"2024-04-22T13:27:09.563562+0000","last_peered":"2024-04-22T13:27:09.563562+0000","last_clean":"2024-04-22T13:27:09.563562+0000","last_became_active":"2024-04-22T13:25:10.905668+0000","last_became_peered":"2024-04-22T13:25:10.905668+0000","last_unstale":"2024-04-22T13:27:09.563562+0000","last_undegraded":"2024-04-22T13:27:09.563562+0000","last_fullsized":"2024-04-22T13:27:09.563562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:32:36.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563593+0000","last_change":"2024-04-22T13:25:10.904368+0000","last_active":"2024-04-22T13:27:09.563593+0000","last_peered":"2024-04-22T13:27:09.563593+0000","last_clean":"2024-04-22T13:27:09.563593+0000","last_became_active":"2024-04-22T13:25:10.904023+0000","last_became_peered":"2024-04-22T13:25:10.904023+0000","last_unstale":"2024-04-22T13:27:09.563593+0000","last_undegraded":"2024-04-22T13:27:09.563593+0000","last_fullsized":"2024-04-22T13:27:09.563593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:45:55.964394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563776+0000","last_change":"2024-04-22T13:25:10.907046+0000","last_active":"2024-04-22T13:27:09.563776+0000","last_peered":"2024-04-22T13:27:09.563776+0000","last_clean":"2024-04-22T13:27:09.563776+0000","last_became_active":"2024-04-22T13:25:10.906682+0000","last_became_peered":"2024-04-22T13:25:10.906682+0000","last_unstale":"2024-04-22T13:27:09.563776+0000","last_undegraded":"2024-04-22T13:27:09.563776+0000","last_fullsized":"2024-04-22T13:27:09.563776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:01.384711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562794+0000","last_change":"2024-04-22T13:25:10.912502+0000","last_active":"2024-04-22T13:27:09.562794+0000","last_peered":"2024-04-22T13:27:09.562794+0000","last_clean":"2024-04-22T13:27:09.562794+0000","last_became_active":"2024-04-22T13:25:10.911898+0000","last_became_peered":"2024-04-22T13:25:10.911898+0000","last_unstale":"2024-04-22T13:27:09.562794+0000","last_undegraded":"2024-04-22T13:27:09.562794+0000","last_fullsized":"2024-04-22T13:27:09.562794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:21.657432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562713+0000","last_change":"2024-04-22T13:25:10.898995+0000","last_active":"2024-04-22T13:27:09.562713+0000","last_peered":"2024-04-22T13:27:09.562713+0000","last_clean":"2024-04-22T13:27:09.562713+0000","last_became_active":"2024-04-22T13:25:10.898101+0000","last_became_peered":"2024-04-22T13:25:10.898101+0000","last_unstale":"2024-04-22T13:27:09.562713+0000","last_undegraded":"2024-04-22T13:27:09.562713+0000","last_fullsized":"2024-04-22T13:27:09.562713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:11.074516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563250+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.563250+0000","last_peered":"2024-04-22T13:27:09.563250+0000","last_clean":"2024-04-22T13:27:09.563250+0000","last_became_active":"2024-04-22T13:25:10.899894+0000","last_became_peered":"2024-04-22T13:25:10.899894+0000","last_unstale":"2024-04-22T13:27:09.563250+0000","last_undegraded":"2024-04-22T13:27:09.563250+0000","last_fullsized":"2024-04-22T13:27:09.563250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:56:03.028481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446872+0000","last_change":"2024-04-22T13:25:10.892514+0000","last_active":"2024-04-22T13:27:09.446872+0000","last_peered":"2024-04-22T13:27:09.446872+0000","last_clean":"2024-04-22T13:27:09.446872+0000","last_became_active":"2024-04-22T13:25:10.892181+0000","last_became_peered":"2024-04-22T13:25:10.892181+0000","last_unstale":"2024-04-22T13:27:09.446872+0000","last_undegraded":"2024-04-22T13:27:09.446872+0000","last_fullsized":"2024-04-22T13:27:09.446872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:32.049869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563565+0000","last_change":"2024-04-22T13:25:10.900123+0000","last_active":"2024-04-22T13:27:09.563565+0000","last_peered":"2024-04-22T13:27:09.563565+0000","last_clean":"2024-04-22T13:27:09.563565+0000","last_became_active":"2024-04-22T13:25:10.899754+0000","last_became_peered":"2024-04-22T13:25:10.899754+0000","last_unstale":"2024-04-22T13:27:09.563565+0000","last_undegraded":"2024-04-22T13:27:09.563565+0000","last_fullsized":"2024-04-22T13:27:09.563565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:56.237089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.409929+0000","last_change":"2024-04-22T13:25:09.878064+0000","last_active":"2024-04-22T13:27:19.409929+0000","last_peered":"2024-04-22T13:27:19.409929+0000","last_clean":"2024-04-22T13:27:19.409929+0000","last_became_active":"2024-04-22T13:25:09.877732+0000","last_became_peered":"2024-04-22T13:25:09.877732+0000","last_unstale":"2024-04-22T13:27:19.409929+0000","last_undegraded":"2024-04-22T13:27:19.409929+0000","last_fullsized":"2024-04-22T13:27:19.409929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:26.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541086+0000","last_change":"2024-04-22T13:25:10.907926+0000","last_active":"2024-04-22T13:27:09.541086+0000","last_peered":"2024-04-22T13:27:09.541086+0000","last_clean":"2024-04-22T13:27:09.541086+0000","last_became_active":"2024-04-22T13:25:10.907536+0000","last_became_peered":"2024-04-22T13:25:10.907536+0000","last_unstale":"2024-04-22T13:27:09.541086+0000","last_undegraded":"2024-04-22T13:27:09.541086+0000","last_fullsized":"2024-04-22T13:27:09.541086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:37:09.267893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541272+0000","last_change":"2024-04-22T13:25:09.882239+0000","last_active":"2024-04-22T13:27:09.541272+0000","last_peered":"2024-04-22T13:27:09.541272+0000","last_clean":"2024-04-22T13:27:09.541272+0000","last_became_active":"2024-04-22T13:25:09.881915+0000","last_became_peered":"2024-04-22T13:25:09.881915+0000","last_unstale":"2024-04-22T13:27:09.541272+0000","last_undegraded":"2024-04-22T13:27:09.541272+0000","last_fullsized":"2024-04-22T13:27:09.541272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:29:00.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564850+0000","last_change":"2024-04-22T13:25:10.916943+0000","last_active":"2024-04-22T13:27:09.564850+0000","last_peered":"2024-04-22T13:27:09.564850+0000","last_clean":"2024-04-22T13:27:09.564850+0000","last_became_active":"2024-04-22T13:25:10.916595+0000","last_became_peered":"2024-04-22T13:25:10.916595+0000","last_unstale":"2024-04-22T13:27:09.564850+0000","last_undegraded":"2024-04-22T13:27:09.564850+0000","last_fullsized":"2024-04-22T13:27:09.564850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:02.725585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'13","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563038+0000","last_change":"2024-04-22T13:26:15.473351+0000","last_active":"2024-04-22T13:27:09.563038+0000","last_peered":"2024-04-22T13:27:09.563038+0000","last_clean":"2024-04-22T13:27:09.563038+0000","last_became_active":"2024-04-22T13:26:15.461181+0000","last_became_peered":"2024-04-22T13:26:15.461181+0000","last_unstale":"2024-04-22T13:27:09.563038+0000","last_undegraded":"2024-04-22T13:27:09.563038+0000","last_fullsized":"2024-04-22T13:27:09.563038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:55.152918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562953+0000","last_change":"2024-04-22T13:25:10.890357+0000","last_active":"2024-04-22T13:27:09.562953+0000","last_peered":"2024-04-22T13:27:09.562953+0000","last_clean":"2024-04-22T13:27:09.562953+0000","last_became_active":"2024-04-22T13:25:10.889781+0000","last_became_peered":"2024-04-22T13:25:10.889781+0000","last_unstale":"2024-04-22T13:27:09.562953+0000","last_undegraded":"2024-04-22T13:27:09.562953+0000","last_fullsized":"2024-04-22T13:27:09.562953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:31.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.451042+0000","last_change":"2024-04-22T13:25:09.873481+0000","last_active":"2024-04-22T13:27:09.451042+0000","last_peered":"2024-04-22T13:27:09.451042+0000","last_clean":"2024-04-22T13:27:09.451042+0000","last_became_active":"2024-04-22T13:25:09.872979+0000","last_became_peered":"2024-04-22T13:25:09.872979+0000","last_unstale":"2024-04-22T13:27:09.451042+0000","last_undegraded":"2024-04-22T13:27:09.451042+0000","last_fullsized":"2024-04-22T13:27:09.451042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:21.477901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547806+0000","last_change":"2024-04-22T13:25:10.906068+0000","last_active":"2024-04-22T13:27:09.547806+0000","last_peered":"2024-04-22T13:27:09.547806+0000","last_clean":"2024-04-22T13:27:09.547806+0000","last_became_active":"2024-04-22T13:25:10.905743+0000","last_became_peered":"2024-04-22T13:25:10.905743+0000","last_unstale":"2024-04-22T13:27:09.547806+0000","last_undegraded":"2024-04-22T13:27:09.547806+0000","last_fullsized":"2024-04-22T13:27:09.547806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:21.126705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.411305+0000","last_change":"2024-04-22T13:25:09.873560+0000","last_active":"2024-04-22T13:27:19.411305+0000","last_peered":"2024-04-22T13:27:19.411305+0000","last_clean":"2024-04-22T13:27:19.411305+0000","last_became_active":"2024-04-22T13:25:09.872688+0000","last_became_peered":"2024-04-22T13:25:09.872688+0000","last_unstale":"2024-04-22T13:27:19.411305+0000","last_undegraded":"2024-04-22T13:27:19.411305+0000","last_fullsized":"2024-04-22T13:27:19.411305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:56:23.723918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450920+0000","last_change":"2024-04-22T13:25:10.900463+0000","last_active":"2024-04-22T13:27:09.450920+0000","last_peered":"2024-04-22T13:27:09.450920+0000","last_clean":"2024-04-22T13:27:09.450920+0000","last_became_active":"2024-04-22T13:25:10.900099+0000","last_became_peered":"2024-04-22T13:25:10.900099+0000","last_unstale":"2024-04-22T13:27:09.450920+0000","last_undegraded":"2024-04-22T13:27:09.450920+0000","last_fullsized":"2024-04-22T13:27:09.450920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:35.637886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448243+0000","last_change":"2024-04-22T13:25:09.873457+0000","last_active":"2024-04-22T13:27:09.448243+0000","last_peered":"2024-04-22T13:27:09.448243+0000","last_clean":"2024-04-22T13:27:09.448243+0000","last_became_active":"2024-04-22T13:25:09.873133+0000","last_became_peered":"2024-04-22T13:25:09.873133+0000","last_unstale":"2024-04-22T13:27:09.448243+0000","last_undegraded":"2024-04-22T13:27:09.448243+0000","last_fullsized":"2024-04-22T13:27:09.448243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:35:54.215669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547784+0000","last_change":"2024-04-22T13:25:10.904721+0000","last_active":"2024-04-22T13:27:09.547784+0000","last_peered":"2024-04-22T13:27:09.547784+0000","last_clean":"2024-04-22T13:27:09.547784+0000","last_became_active":"2024-04-22T13:25:10.904344+0000","last_became_peered":"2024-04-22T13:25:10.904344+0000","last_unstale":"2024-04-22T13:27:09.547784+0000","last_undegraded":"2024-04-22T13:27:09.547784+0000","last_fullsized":"2024-04-22T13:27:09.547784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:13:41.386509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562955+0000","last_change":"2024-04-22T13:25:09.872863+0000","last_active":"2024-04-22T13:27:09.562955+0000","last_peered":"2024-04-22T13:27:09.562955+0000","last_clean":"2024-04-22T13:27:09.562955+0000","last_became_active":"2024-04-22T13:25:09.872463+0000","last_became_peered":"2024-04-22T13:25:09.872463+0000","last_unstale":"2024-04-22T13:27:09.562955+0000","last_undegraded":"2024-04-22T13:27:09.562955+0000","last_fullsized":"2024-04-22T13:27:09.562955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:37.214136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540293+0000","last_change":"2024-04-22T13:25:10.909617+0000","last_active":"2024-04-22T13:27:09.540293+0000","last_peered":"2024-04-22T13:27:09.540293+0000","last_clean":"2024-04-22T13:27:09.540293+0000","last_became_active":"2024-04-22T13:25:10.908942+0000","last_became_peered":"2024-04-22T13:25:10.908942+0000","last_unstale":"2024-04-22T13:27:09.540293+0000","last_undegraded":"2024-04-22T13:27:09.540293+0000","last_fullsized":"2024-04-22T13:27:09.540293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:41.619206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547473+0000","last_change":"2024-04-22T13:25:09.874803+0000","last_active":"2024-04-22T13:27:09.547473+0000","last_peered":"2024-04-22T13:27:09.547473+0000","last_clean":"2024-04-22T13:27:09.547473+0000","last_became_active":"2024-04-22T13:25:09.874367+0000","last_became_peered":"2024-04-22T13:25:09.874367+0000","last_unstale":"2024-04-22T13:27:09.547473+0000","last_undegraded":"2024-04-22T13:27:09.547473+0000","last_fullsized":"2024-04-22T13:27:09.547473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:51:23.552096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446526+0000","last_change":"2024-04-22T13:25:10.901590+0000","last_active":"2024-04-22T13:27:09.446526+0000","last_peered":"2024-04-22T13:27:09.446526+0000","last_clean":"2024-04-22T13:27:09.446526+0000","last_became_active":"2024-04-22T13:25:10.901273+0000","last_became_peered":"2024-04-22T13:25:10.901273+0000","last_unstale":"2024-04-22T13:27:09.446526+0000","last_undegraded":"2024-04-22T13:27:09.446526+0000","last_fullsized":"2024-04-22T13:27:09.446526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:13.219176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":276,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:20.411992+0000","last_change":"2024-04-22T13:25:09.872444+0000","last_active":"2024-04-22T13:27:20.411992+0000","last_peered":"2024-04-22T13:27:20.411992+0000","last_clean":"2024-04-22T13:27:20.411992+0000","last_became_active":"2024-04-22T13:25:09.872103+0000","last_became_peered":"2024-04-22T13:25:09.872103+0000","last_unstale":"2024-04-22T13:27:20.411992+0000","last_undegraded":"2024-04-22T13:27:20.411992+0000","last_fullsized":"2024-04-22T13:27:20.411992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:07.582228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":248,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563427+0000","last_change":"2024-04-22T13:25:10.896124+0000","last_active":"2024-04-22T13:27:09.563427+0000","last_peered":"2024-04-22T13:27:09.563427+0000","last_clean":"2024-04-22T13:27:09.563427+0000","last_became_active":"2024-04-22T13:25:10.895641+0000","last_became_peered":"2024-04-22T13:25:10.895641+0000","last_unstale":"2024-04-22T13:27:09.563427+0000","last_undegraded":"2024-04-22T13:27:09.563427+0000","last_fullsized":"2024-04-22T13:27:09.563427+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:12.561886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564441+0000","last_change":"2024-04-22T13:25:09.874090+0000","last_active":"2024-04-22T13:27:09.564441+0000","last_peered":"2024-04-22T13:27:09.564441+0000","last_clean":"2024-04-22T13:27:09.564441+0000","last_became_active":"2024-04-22T13:25:09.873624+0000","last_became_peered":"2024-04-22T13:25:09.873624+0000","last_unstale":"2024-04-22T13:27:09.564441+0000","last_undegraded":"2024-04-22T13:27:09.564441+0000","last_fullsized":"2024-04-22T13:27:09.564441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:44:03.072139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564869+0000","last_change":"2024-04-22T13:26:15.516279+0000","last_active":"2024-04-22T13:27:09.564869+0000","last_peered":"2024-04-22T13:27:09.564869+0000","last_clean":"2024-04-22T13:27:09.564869+0000","last_became_active":"2024-04-22T13:26:15.515877+0000","last_became_peered":"2024-04-22T13:26:15.515877+0000","last_unstale":"2024-04-22T13:27:09.564869+0000","last_undegraded":"2024-04-22T13:27:09.564869+0000","last_fullsized":"2024-04-22T13:27:09.564869+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:56.508950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547887+0000","last_change":"2024-04-22T13:25:09.882626+0000","last_active":"2024-04-22T13:27:09.547887+0000","last_peered":"2024-04-22T13:27:09.547887+0000","last_clean":"2024-04-22T13:27:09.547887+0000","last_became_active":"2024-04-22T13:25:09.882315+0000","last_became_peered":"2024-04-22T13:25:09.882315+0000","last_unstale":"2024-04-22T13:27:09.547887+0000","last_undegraded":"2024-04-22T13:27:09.547887+0000","last_fullsized":"2024-04-22T13:27:09.547887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:02:41.375103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562984+0000","last_change":"2024-04-22T13:25:10.908195+0000","last_active":"2024-04-22T13:27:09.562984+0000","last_peered":"2024-04-22T13:27:09.562984+0000","last_clean":"2024-04-22T13:27:09.562984+0000","last_became_active":"2024-04-22T13:25:10.907755+0000","last_became_peered":"2024-04-22T13:25:10.907755+0000","last_unstale":"2024-04-22T13:27:09.562984+0000","last_undegraded":"2024-04-22T13:27:09.562984+0000","last_fullsized":"2024-04-22T13:27:09.562984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:35.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":319,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:20.417834+0000","last_change":"2024-04-22T13:25:09.874233+0000","last_active":"2024-04-22T13:27:20.417834+0000","last_peered":"2024-04-22T13:27:20.417834+0000","last_clean":"2024-04-22T13:27:20.417834+0000","last_became_active":"2024-04-22T13:25:09.873877+0000","last_became_peered":"2024-04-22T13:25:09.873877+0000","last_unstale":"2024-04-22T13:27:20.417834+0000","last_undegraded":"2024-04-22T13:27:20.417834+0000","last_fullsized":"2024-04-22T13:27:20.417834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:24.444212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68543,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":198,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450855+0000","last_change":"2024-04-22T13:25:10.910638+0000","last_active":"2024-04-22T13:27:09.450855+0000","last_peered":"2024-04-22T13:27:09.450855+0000","last_clean":"2024-04-22T13:27:09.450855+0000","last_became_active":"2024-04-22T13:25:10.910334+0000","last_became_peered":"2024-04-22T13:25:10.910334+0000","last_unstale":"2024-04-22T13:27:09.450855+0000","last_undegraded":"2024-04-22T13:27:09.450855+0000","last_fullsized":"2024-04-22T13:27:09.450855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:56.058700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563392+0000","last_change":"2024-04-22T13:25:09.878564+0000","last_active":"2024-04-22T13:27:09.563392+0000","last_peered":"2024-04-22T13:27:09.563392+0000","last_clean":"2024-04-22T13:27:09.563392+0000","last_became_active":"2024-04-22T13:25:09.878160+0000","last_became_peered":"2024-04-22T13:25:09.878160+0000","last_unstale":"2024-04-22T13:27:09.563392+0000","last_undegraded":"2024-04-22T13:27:09.563392+0000","last_fullsized":"2024-04-22T13:27:09.563392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:28.968213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562910+0000","last_change":"2024-04-22T13:25:10.900683+0000","last_active":"2024-04-22T13:27:09.562910+0000","last_peered":"2024-04-22T13:27:09.562910+0000","last_clean":"2024-04-22T13:27:09.562910+0000","last_became_active":"2024-04-22T13:25:10.900310+0000","last_became_peered":"2024-04-22T13:25:10.900310+0000","last_unstale":"2024-04-22T13:27:09.562910+0000","last_undegraded":"2024-04-22T13:27:09.562910+0000","last_fullsized":"2024-04-22T13:27:09.562910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:22.387538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547599+0000","last_change":"2024-04-22T13:25:09.872239+0000","last_active":"2024-04-22T13:27:09.547599+0000","last_peered":"2024-04-22T13:27:09.547599+0000","last_clean":"2024-04-22T13:27:09.547599+0000","last_became_active":"2024-04-22T13:25:09.871942+0000","last_became_peered":"2024-04-22T13:25:09.871942+0000","last_unstale":"2024-04-22T13:27:09.547599+0000","last_undegraded":"2024-04-22T13:27:09.547599+0000","last_fullsized":"2024-04-22T13:27:09.547599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:31:29.253980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564267+0000","last_change":"2024-04-22T13:25:10.905069+0000","last_active":"2024-04-22T13:27:09.564267+0000","last_peered":"2024-04-22T13:27:09.564267+0000","last_clean":"2024-04-22T13:27:09.564267+0000","last_became_active":"2024-04-22T13:25:10.904665+0000","last_became_peered":"2024-04-22T13:25:10.904665+0000","last_unstale":"2024-04-22T13:27:09.564267+0000","last_undegraded":"2024-04-22T13:27:09.564267+0000","last_fullsized":"2024-04-22T13:27:09.564267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:23.067817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.410628+0000","last_change":"2024-04-22T13:25:09.872829+0000","last_active":"2024-04-22T13:27:19.410628+0000","last_peered":"2024-04-22T13:27:19.410628+0000","last_clean":"2024-04-22T13:27:19.410628+0000","last_became_active":"2024-04-22T13:25:09.872482+0000","last_became_peered":"2024-04-22T13:25:09.872482+0000","last_unstale":"2024-04-22T13:27:19.410628+0000","last_undegraded":"2024-04-22T13:27:19.410628+0000","last_fullsized":"2024-04-22T13:27:19.410628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:07.192644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563985+0000","last_change":"2024-04-22T13:25:10.907517+0000","last_active":"2024-04-22T13:27:09.563985+0000","last_peered":"2024-04-22T13:27:09.563985+0000","last_clean":"2024-04-22T13:27:09.563985+0000","last_became_active":"2024-04-22T13:25:10.907163+0000","last_became_peered":"2024-04-22T13:25:10.907163+0000","last_unstale":"2024-04-22T13:27:09.563985+0000","last_undegraded":"2024-04-22T13:27:09.563985+0000","last_fullsized":"2024-04-22T13:27:09.563985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:27.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450657+0000","last_change":"2024-04-22T13:25:09.869740+0000","last_active":"2024-04-22T13:27:09.450657+0000","last_peered":"2024-04-22T13:27:09.450657+0000","last_clean":"2024-04-22T13:27:09.450657+0000","last_became_active":"2024-04-22T13:25:09.869359+0000","last_became_peered":"2024-04-22T13:25:09.869359+0000","last_unstale":"2024-04-22T13:27:09.450657+0000","last_undegraded":"2024-04-22T13:27:09.450657+0000","last_fullsized":"2024-04-22T13:27:09.450657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:21:33.456280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.547522+0000","last_change":"2024-04-22T13:25:10.905288+0000","last_active":"2024-04-22T13:27:09.547522+0000","last_peered":"2024-04-22T13:27:09.547522+0000","last_clean":"2024-04-22T13:27:09.547522+0000","last_became_active":"2024-04-22T13:25:10.904890+0000","last_became_peered":"2024-04-22T13:25:10.904890+0000","last_unstale":"2024-04-22T13:27:09.547522+0000","last_undegraded":"2024-04-22T13:27:09.547522+0000","last_fullsized":"2024-04-22T13:27:09.547522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:15.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450885+0000","last_change":"2024-04-22T13:25:09.871396+0000","last_active":"2024-04-22T13:27:09.450885+0000","last_peered":"2024-04-22T13:27:09.450885+0000","last_clean":"2024-04-22T13:27:09.450885+0000","last_became_active":"2024-04-22T13:25:09.870564+0000","last_became_peered":"2024-04-22T13:25:09.870564+0000","last_unstale":"2024-04-22T13:27:09.450885+0000","last_undegraded":"2024-04-22T13:27:09.450885+0000","last_fullsized":"2024-04-22T13:27:09.450885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:55:50.773349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563035+0000","last_change":"2024-04-22T13:25:10.896858+0000","last_active":"2024-04-22T13:27:09.563035+0000","last_peered":"2024-04-22T13:27:09.563035+0000","last_clean":"2024-04-22T13:27:09.563035+0000","last_became_active":"2024-04-22T13:25:10.896445+0000","last_became_peered":"2024-04-22T13:25:10.896445+0000","last_unstale":"2024-04-22T13:27:09.563035+0000","last_undegraded":"2024-04-22T13:27:09.563035+0000","last_fullsized":"2024-04-22T13:27:09.563035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:38.588104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446773+0000","last_change":"2024-04-22T13:25:09.883126+0000","last_active":"2024-04-22T13:27:09.446773+0000","last_peered":"2024-04-22T13:27:09.446773+0000","last_clean":"2024-04-22T13:27:09.446773+0000","last_became_active":"2024-04-22T13:25:09.882829+0000","last_became_peered":"2024-04-22T13:25:09.882829+0000","last_unstale":"2024-04-22T13:27:09.446773+0000","last_undegraded":"2024-04-22T13:27:09.446773+0000","last_fullsized":"2024-04-22T13:27:09.446773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:00:17.781003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448737+0000","last_change":"2024-04-22T13:25:10.901805+0000","last_active":"2024-04-22T13:27:09.448737+0000","last_peered":"2024-04-22T13:27:09.448737+0000","last_clean":"2024-04-22T13:27:09.448737+0000","last_became_active":"2024-04-22T13:25:10.901416+0000","last_became_peered":"2024-04-22T13:25:10.901416+0000","last_unstale":"2024-04-22T13:27:09.448737+0000","last_undegraded":"2024-04-22T13:27:09.448737+0000","last_fullsized":"2024-04-22T13:27:09.448737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:41:32.517625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.410621+0000","last_change":"2024-04-22T13:25:09.872758+0000","last_active":"2024-04-22T13:27:19.410621+0000","last_peered":"2024-04-22T13:27:19.410621+0000","last_clean":"2024-04-22T13:27:19.410621+0000","last_became_active":"2024-04-22T13:25:09.872457+0000","last_became_peered":"2024-04-22T13:25:09.872457+0000","last_unstale":"2024-04-22T13:27:19.410621+0000","last_undegraded":"2024-04-22T13:27:19.410621+0000","last_fullsized":"2024-04-22T13:27:19.410621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:17:14.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.448555+0000","last_change":"2024-04-22T13:25:10.902418+0000","last_active":"2024-04-22T13:27:09.448555+0000","last_peered":"2024-04-22T13:27:09.448555+0000","last_clean":"2024-04-22T13:27:09.448555+0000","last_became_active":"2024-04-22T13:25:10.902058+0000","last_became_peered":"2024-04-22T13:25:10.902058+0000","last_unstale":"2024-04-22T13:27:09.448555+0000","last_undegraded":"2024-04-22T13:27:09.448555+0000","last_fullsized":"2024-04-22T13:27:09.448555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:42:26.739515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.446492+0000","last_change":"2024-04-22T13:26:15.457159+0000","last_active":"2024-04-22T13:27:09.446492+0000","last_peered":"2024-04-22T13:27:09.446492+0000","last_clean":"2024-04-22T13:27:09.446492+0000","last_became_active":"2024-04-22T13:26:15.456739+0000","last_became_peered":"2024-04-22T13:26:15.456739+0000","last_unstale":"2024-04-22T13:27:09.446492+0000","last_undegraded":"2024-04-22T13:27:09.446492+0000","last_fullsized":"2024-04-22T13:27:09.446492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:56.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.409823+0000","last_change":"2024-04-22T13:25:09.875856+0000","last_active":"2024-04-22T13:27:19.409823+0000","last_peered":"2024-04-22T13:27:19.409823+0000","last_clean":"2024-04-22T13:27:19.409823+0000","last_became_active":"2024-04-22T13:25:09.875496+0000","last_became_peered":"2024-04-22T13:25:09.875496+0000","last_unstale":"2024-04-22T13:27:19.409823+0000","last_undegraded":"2024-04-22T13:27:19.409823+0000","last_fullsized":"2024-04-22T13:27:19.409823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:57:30.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540620+0000","last_change":"2024-04-22T13:25:10.917393+0000","last_active":"2024-04-22T13:27:09.540620+0000","last_peered":"2024-04-22T13:27:09.540620+0000","last_clean":"2024-04-22T13:27:09.540620+0000","last_became_active":"2024-04-22T13:25:10.917004+0000","last_became_peered":"2024-04-22T13:25:10.917004+0000","last_unstale":"2024-04-22T13:27:09.540620+0000","last_undegraded":"2024-04-22T13:27:09.540620+0000","last_fullsized":"2024-04-22T13:27:09.540620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:48.187121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563518+0000","last_change":"2024-04-22T13:25:09.875073+0000","last_active":"2024-04-22T13:27:09.563518+0000","last_peered":"2024-04-22T13:27:09.563518+0000","last_clean":"2024-04-22T13:27:09.563518+0000","last_became_active":"2024-04-22T13:25:09.874744+0000","last_became_peered":"2024-04-22T13:25:09.874744+0000","last_unstale":"2024-04-22T13:27:09.563518+0000","last_undegraded":"2024-04-22T13:27:09.563518+0000","last_fullsized":"2024-04-22T13:27:09.563518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:42:04.496646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":403,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:11.559108+0000","last_change":"2024-04-22T13:24:02.320802+0000","last_active":"2024-04-22T13:27:11.559108+0000","last_peered":"2024-04-22T13:27:11.559108+0000","last_clean":"2024-04-22T13:27:11.559108+0000","last_became_active":"2024-04-22T13:24:02.320311+0000","last_became_peered":"2024-04-22T13:24:02.320311+0000","last_unstale":"2024-04-22T13:27:11.559108+0000","last_undegraded":"2024-04-22T13:27:11.559108+0000","last_fullsized":"2024-04-22T13:27:11.559108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:20:42.350711+0000","last_clean_scrub_stamp":"2024-04-22T13:20:42.350711+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:07.106513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562824+0000","last_change":"2024-04-22T13:25:10.896526+0000","last_active":"2024-04-22T13:27:09.562824+0000","last_peered":"2024-04-22T13:27:09.562824+0000","last_clean":"2024-04-22T13:27:09.562824+0000","last_became_active":"2024-04-22T13:25:10.896003+0000","last_became_peered":"2024-04-22T13:25:10.896003+0000","last_unstale":"2024-04-22T13:27:09.562824+0000","last_undegraded":"2024-04-22T13:27:09.562824+0000","last_fullsized":"2024-04-22T13:27:09.562824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:22:53.648824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564286+0000","last_change":"2024-04-22T13:25:09.871355+0000","last_active":"2024-04-22T13:27:09.564286+0000","last_peered":"2024-04-22T13:27:09.564286+0000","last_clean":"2024-04-22T13:27:09.564286+0000","last_became_active":"2024-04-22T13:25:09.871043+0000","last_became_peered":"2024-04-22T13:25:09.871043+0000","last_unstale":"2024-04-22T13:27:09.564286+0000","last_undegraded":"2024-04-22T13:27:09.564286+0000","last_fullsized":"2024-04-22T13:27:09.564286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:29:36.623724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450990+0000","last_change":"2024-04-22T13:25:09.873519+0000","last_active":"2024-04-22T13:27:09.450990+0000","last_peered":"2024-04-22T13:27:09.450990+0000","last_clean":"2024-04-22T13:27:09.450990+0000","last_became_active":"2024-04-22T13:25:09.872733+0000","last_became_peered":"2024-04-22T13:25:09.872733+0000","last_unstale":"2024-04-22T13:27:09.450990+0000","last_undegraded":"2024-04-22T13:27:09.450990+0000","last_fullsized":"2024-04-22T13:27:09.450990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:06.532938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563277+0000","last_change":"2024-04-22T13:25:10.895177+0000","last_active":"2024-04-22T13:27:09.563277+0000","last_peered":"2024-04-22T13:27:09.563277+0000","last_clean":"2024-04-22T13:27:09.563277+0000","last_became_active":"2024-04-22T13:25:10.894459+0000","last_became_peered":"2024-04-22T13:25:10.894459+0000","last_unstale":"2024-04-22T13:27:09.563277+0000","last_undegraded":"2024-04-22T13:27:09.563277+0000","last_fullsized":"2024-04-22T13:27:09.563277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:52.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'103","reported_seq":217,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:18.206290+0000","last_change":"2024-04-22T13:25:09.877660+0000","last_active":"2024-04-22T13:27:18.206290+0000","last_peered":"2024-04-22T13:27:18.206290+0000","last_clean":"2024-04-22T13:27:18.206290+0000","last_became_active":"2024-04-22T13:25:09.877290+0000","last_became_peered":"2024-04-22T13:25:09.877290+0000","last_unstale":"2024-04-22T13:27:18.206290+0000","last_undegraded":"2024-04-22T13:27:18.206290+0000","last_fullsized":"2024-04-22T13:27:18.206290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:32.350048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541266+0000","last_change":"2024-04-22T13:25:10.906588+0000","last_active":"2024-04-22T13:27:09.541266+0000","last_peered":"2024-04-22T13:27:09.541266+0000","last_clean":"2024-04-22T13:27:09.541266+0000","last_became_active":"2024-04-22T13:25:10.906208+0000","last_became_peered":"2024-04-22T13:25:10.906208+0000","last_unstale":"2024-04-22T13:27:09.541266+0000","last_undegraded":"2024-04-22T13:27:09.541266+0000","last_fullsized":"2024-04-22T13:27:09.541266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:05.335623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563131+0000","last_change":"2024-04-22T13:25:09.876157+0000","last_active":"2024-04-22T13:27:09.563131+0000","last_peered":"2024-04-22T13:27:09.563131+0000","last_clean":"2024-04-22T13:27:09.563131+0000","last_became_active":"2024-04-22T13:25:09.875847+0000","last_became_peered":"2024-04-22T13:25:09.875847+0000","last_unstale":"2024-04-22T13:27:09.563131+0000","last_undegraded":"2024-04-22T13:27:09.563131+0000","last_fullsized":"2024-04-22T13:27:09.563131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:19:09.793118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564847+0000","last_change":"2024-04-22T13:25:10.913558+0000","last_active":"2024-04-22T13:27:09.564847+0000","last_peered":"2024-04-22T13:27:09.564847+0000","last_clean":"2024-04-22T13:27:09.564847+0000","last_became_active":"2024-04-22T13:25:10.912598+0000","last_became_peered":"2024-04-22T13:25:10.912598+0000","last_unstale":"2024-04-22T13:27:09.564847+0000","last_undegraded":"2024-04-22T13:27:09.564847+0000","last_fullsized":"2024-04-22T13:27:09.564847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:09.488309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562881+0000","last_change":"2024-04-22T13:25:09.882728+0000","last_active":"2024-04-22T13:27:09.562881+0000","last_peered":"2024-04-22T13:27:09.562881+0000","last_clean":"2024-04-22T13:27:09.562881+0000","last_became_active":"2024-04-22T13:25:09.882420+0000","last_became_peered":"2024-04-22T13:25:09.882420+0000","last_unstale":"2024-04-22T13:27:09.562881+0000","last_undegraded":"2024-04-22T13:27:09.562881+0000","last_fullsized":"2024-04-22T13:27:09.562881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:31:47.352806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450691+0000","last_change":"2024-04-22T13:25:10.914866+0000","last_active":"2024-04-22T13:27:09.450691+0000","last_peered":"2024-04-22T13:27:09.450691+0000","last_clean":"2024-04-22T13:27:09.450691+0000","last_became_active":"2024-04-22T13:25:10.914196+0000","last_became_peered":"2024-04-22T13:25:10.914196+0000","last_unstale":"2024-04-22T13:27:09.450691+0000","last_undegraded":"2024-04-22T13:27:09.450691+0000","last_fullsized":"2024-04-22T13:27:09.450691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:00.319510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540359+0000","last_change":"2024-04-22T13:25:09.873231+0000","last_active":"2024-04-22T13:27:09.540359+0000","last_peered":"2024-04-22T13:27:09.540359+0000","last_clean":"2024-04-22T13:27:09.540359+0000","last_became_active":"2024-04-22T13:25:09.872910+0000","last_became_peered":"2024-04-22T13:25:09.872910+0000","last_unstale":"2024-04-22T13:27:09.540359+0000","last_undegraded":"2024-04-22T13:27:09.540359+0000","last_fullsized":"2024-04-22T13:27:09.540359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:23:32.050141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562679+0000","last_change":"2024-04-22T13:25:10.897771+0000","last_active":"2024-04-22T13:27:09.562679+0000","last_peered":"2024-04-22T13:27:09.562679+0000","last_clean":"2024-04-22T13:27:09.562679+0000","last_became_active":"2024-04-22T13:25:10.896844+0000","last_became_peered":"2024-04-22T13:25:10.896844+0000","last_unstale":"2024-04-22T13:27:09.562679+0000","last_undegraded":"2024-04-22T13:27:09.562679+0000","last_fullsized":"2024-04-22T13:27:09.562679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:37:49.090637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562789+0000","last_change":"2024-04-22T13:26:15.477094+0000","last_active":"2024-04-22T13:27:09.562789+0000","last_peered":"2024-04-22T13:27:09.562789+0000","last_clean":"2024-04-22T13:27:09.562789+0000","last_became_active":"2024-04-22T13:26:15.462486+0000","last_became_peered":"2024-04-22T13:26:15.462486+0000","last_unstale":"2024-04-22T13:27:09.562789+0000","last_undegraded":"2024-04-22T13:27:09.562789+0000","last_fullsized":"2024-04-22T13:27:09.562789+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:04.484665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540822+0000","last_change":"2024-04-22T13:25:10.910692+0000","last_active":"2024-04-22T13:27:09.540822+0000","last_peered":"2024-04-22T13:27:09.540822+0000","last_clean":"2024-04-22T13:27:09.540822+0000","last_became_active":"2024-04-22T13:25:10.910095+0000","last_became_peered":"2024-04-22T13:25:10.910095+0000","last_unstale":"2024-04-22T13:27:09.540822+0000","last_undegraded":"2024-04-22T13:27:09.540822+0000","last_fullsized":"2024-04-22T13:27:09.540822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:43.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.450996+0000","last_change":"2024-04-22T13:25:09.870159+0000","last_active":"2024-04-22T13:27:09.450996+0000","last_peered":"2024-04-22T13:27:09.450996+0000","last_clean":"2024-04-22T13:27:09.450996+0000","last_became_active":"2024-04-22T13:25:09.869650+0000","last_became_peered":"2024-04-22T13:25:09.869650+0000","last_unstale":"2024-04-22T13:27:09.450996+0000","last_undegraded":"2024-04-22T13:27:09.450996+0000","last_fullsized":"2024-04-22T13:27:09.450996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:04.730441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564270+0000","last_change":"2024-04-22T13:25:10.894418+0000","last_active":"2024-04-22T13:27:09.564270+0000","last_peered":"2024-04-22T13:27:09.564270+0000","last_clean":"2024-04-22T13:27:09.564270+0000","last_became_active":"2024-04-22T13:25:10.894075+0000","last_became_peered":"2024-04-22T13:25:10.894075+0000","last_unstale":"2024-04-22T13:27:09.564270+0000","last_undegraded":"2024-04-22T13:27:09.564270+0000","last_fullsized":"2024-04-22T13:27:09.564270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:06.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:19.410927+0000","last_change":"2024-04-22T13:25:09.882155+0000","last_active":"2024-04-22T13:27:19.410927+0000","last_peered":"2024-04-22T13:27:19.410927+0000","last_clean":"2024-04-22T13:27:19.410927+0000","last_became_active":"2024-04-22T13:25:09.881750+0000","last_became_peered":"2024-04-22T13:25:09.881750+0000","last_unstale":"2024-04-22T13:27:19.410927+0000","last_undegraded":"2024-04-22T13:27:19.410927+0000","last_fullsized":"2024-04-22T13:27:19.410927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:49.899545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564284+0000","last_change":"2024-04-22T13:25:10.900244+0000","last_active":"2024-04-22T13:27:09.564284+0000","last_peered":"2024-04-22T13:27:09.564284+0000","last_clean":"2024-04-22T13:27:09.564284+0000","last_became_active":"2024-04-22T13:25:10.899161+0000","last_became_peered":"2024-04-22T13:25:10.899161+0000","last_unstale":"2024-04-22T13:27:09.564284+0000","last_undegraded":"2024-04-22T13:27:09.564284+0000","last_fullsized":"2024-04-22T13:27:09.564284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:28:40.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'67","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:17.859417+0000","last_change":"2024-04-22T13:25:09.880535+0000","last_active":"2024-04-22T13:27:17.859417+0000","last_peered":"2024-04-22T13:27:17.859417+0000","last_clean":"2024-04-22T13:27:17.859417+0000","last_became_active":"2024-04-22T13:25:09.880216+0000","last_became_peered":"2024-04-22T13:25:09.880216+0000","last_unstale":"2024-04-22T13:27:17.859417+0000","last_undegraded":"2024-04-22T13:27:17.859417+0000","last_fullsized":"2024-04-22T13:27:17.859417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:22.800108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564302+0000","last_change":"2024-04-22T13:25:10.899215+0000","last_active":"2024-04-22T13:27:09.564302+0000","last_peered":"2024-04-22T13:27:09.564302+0000","last_clean":"2024-04-22T13:27:09.564302+0000","last_became_active":"2024-04-22T13:25:10.898809+0000","last_became_peered":"2024-04-22T13:25:10.898809+0000","last_unstale":"2024-04-22T13:27:09.564302+0000","last_undegraded":"2024-04-22T13:27:09.564302+0000","last_fullsized":"2024-04-22T13:27:09.564302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:43.733055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563457+0000","last_change":"2024-04-22T13:25:09.870803+0000","last_active":"2024-04-22T13:27:09.563457+0000","last_peered":"2024-04-22T13:27:09.563457+0000","last_clean":"2024-04-22T13:27:09.563457+0000","last_became_active":"2024-04-22T13:25:09.870352+0000","last_became_peered":"2024-04-22T13:25:09.870352+0000","last_unstale":"2024-04-22T13:27:09.563457+0000","last_undegraded":"2024-04-22T13:27:09.563457+0000","last_fullsized":"2024-04-22T13:27:09.563457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:08.852070+0000","last_clean_scrub_stamp":"2024-04-22T13:25:08.852070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:42.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564533+0000","last_change":"2024-04-22T13:25:10.902154+0000","last_active":"2024-04-22T13:27:09.564533+0000","last_peered":"2024-04-22T13:27:09.564533+0000","last_clean":"2024-04-22T13:27:09.564533+0000","last_became_active":"2024-04-22T13:25:10.901773+0000","last_became_peered":"2024-04-22T13:25:10.901773+0000","last_unstale":"2024-04-22T13:27:09.564533+0000","last_undegraded":"2024-04-22T13:27:09.564533+0000","last_fullsized":"2024-04-22T13:27:09.564533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:39.184514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562817+0000","last_change":"2024-04-22T13:25:10.898837+0000","last_active":"2024-04-22T13:27:09.562817+0000","last_peered":"2024-04-22T13:27:09.562817+0000","last_clean":"2024-04-22T13:27:09.562817+0000","last_became_active":"2024-04-22T13:25:10.898401+0000","last_became_peered":"2024-04-22T13:25:10.898401+0000","last_unstale":"2024-04-22T13:27:09.562817+0000","last_undegraded":"2024-04-22T13:27:09.562817+0000","last_fullsized":"2024-04-22T13:27:09.562817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:40:47.344228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563366+0000","last_change":"2024-04-22T13:26:15.515905+0000","last_active":"2024-04-22T13:27:09.563366+0000","last_peered":"2024-04-22T13:27:09.563366+0000","last_clean":"2024-04-22T13:27:09.563366+0000","last_became_active":"2024-04-22T13:26:15.515499+0000","last_became_peered":"2024-04-22T13:26:15.515499+0000","last_unstale":"2024-04-22T13:27:09.563366+0000","last_undegraded":"2024-04-22T13:27:09.563366+0000","last_fullsized":"2024-04-22T13:27:09.563366+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:46:43.337290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563322+0000","last_change":"2024-04-22T13:26:15.460203+0000","last_active":"2024-04-22T13:27:09.563322+0000","last_peered":"2024-04-22T13:27:09.563322+0000","last_clean":"2024-04-22T13:27:09.563322+0000","last_became_active":"2024-04-22T13:26:15.459661+0000","last_became_peered":"2024-04-22T13:26:15.459661+0000","last_unstale":"2024-04-22T13:27:09.563322+0000","last_undegraded":"2024-04-22T13:27:09.563322+0000","last_fullsized":"2024-04-22T13:27:09.563322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:57.206490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.564345+0000","last_change":"2024-04-22T13:25:10.905805+0000","last_active":"2024-04-22T13:27:09.564345+0000","last_peered":"2024-04-22T13:27:09.564345+0000","last_clean":"2024-04-22T13:27:09.564345+0000","last_became_active":"2024-04-22T13:25:10.904865+0000","last_became_peered":"2024-04-22T13:25:10.904865+0000","last_unstale":"2024-04-22T13:27:09.564345+0000","last_undegraded":"2024-04-22T13:27:09.564345+0000","last_fullsized":"2024-04-22T13:27:09.564345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:15:11.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563441+0000","last_change":"2024-04-22T13:26:15.459853+0000","last_active":"2024-04-22T13:27:09.563441+0000","last_peered":"2024-04-22T13:27:09.563441+0000","last_clean":"2024-04-22T13:27:09.563441+0000","last_became_active":"2024-04-22T13:26:15.459441+0000","last_became_peered":"2024-04-22T13:26:15.459441+0000","last_unstale":"2024-04-22T13:27:09.563441+0000","last_undegraded":"2024-04-22T13:27:09.563441+0000","last_fullsized":"2024-04-22T13:27:09.563441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:32.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563022+0000","last_change":"2024-04-22T13:25:10.902723+0000","last_active":"2024-04-22T13:27:09.563022+0000","last_peered":"2024-04-22T13:27:09.563022+0000","last_clean":"2024-04-22T13:27:09.563022+0000","last_became_active":"2024-04-22T13:25:10.902370+0000","last_became_peered":"2024-04-22T13:25:10.902370+0000","last_unstale":"2024-04-22T13:27:09.563022+0000","last_undegraded":"2024-04-22T13:27:09.563022+0000","last_fullsized":"2024-04-22T13:27:09.563022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:20.932748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563680+0000","last_change":"2024-04-22T13:25:10.903664+0000","last_active":"2024-04-22T13:27:09.563680+0000","last_peered":"2024-04-22T13:27:09.563680+0000","last_clean":"2024-04-22T13:27:09.563680+0000","last_became_active":"2024-04-22T13:25:10.903302+0000","last_became_peered":"2024-04-22T13:25:10.903302+0000","last_unstale":"2024-04-22T13:27:09.563680+0000","last_undegraded":"2024-04-22T13:27:09.563680+0000","last_fullsized":"2024-04-22T13:27:09.563680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:55:01.816540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.541195+0000","last_change":"2024-04-22T13:25:10.898223+0000","last_active":"2024-04-22T13:27:09.541195+0000","last_peered":"2024-04-22T13:27:09.541195+0000","last_clean":"2024-04-22T13:27:09.541195+0000","last_became_active":"2024-04-22T13:25:10.897145+0000","last_became_peered":"2024-04-22T13:25:10.897145+0000","last_unstale":"2024-04-22T13:27:09.541195+0000","last_undegraded":"2024-04-22T13:27:09.541195+0000","last_fullsized":"2024-04-22T13:27:09.541195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:55.442049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562758+0000","last_change":"2024-04-22T13:26:15.461911+0000","last_active":"2024-04-22T13:27:09.562758+0000","last_peered":"2024-04-22T13:27:09.562758+0000","last_clean":"2024-04-22T13:27:09.562758+0000","last_became_active":"2024-04-22T13:26:15.460868+0000","last_became_peered":"2024-04-22T13:26:15.460868+0000","last_unstale":"2024-04-22T13:27:09.562758+0000","last_undegraded":"2024-04-22T13:27:09.562758+0000","last_fullsized":"2024-04-22T13:27:09.562758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:30.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.540273+0000","last_change":"2024-04-22T13:25:10.896333+0000","last_active":"2024-04-22T13:27:09.540273+0000","last_peered":"2024-04-22T13:27:09.540273+0000","last_clean":"2024-04-22T13:27:09.540273+0000","last_became_active":"2024-04-22T13:25:10.896001+0000","last_became_peered":"2024-04-22T13:25:10.896001+0000","last_unstale":"2024-04-22T13:27:09.540273+0000","last_undegraded":"2024-04-22T13:27:09.540273+0000","last_fullsized":"2024-04-22T13:27:09.540273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:34.041411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562941+0000","last_change":"2024-04-22T13:25:10.905201+0000","last_active":"2024-04-22T13:27:09.562941+0000","last_peered":"2024-04-22T13:27:09.562941+0000","last_clean":"2024-04-22T13:27:09.562941+0000","last_became_active":"2024-04-22T13:25:10.904864+0000","last_became_peered":"2024-04-22T13:25:10.904864+0000","last_unstale":"2024-04-22T13:27:09.562941+0000","last_undegraded":"2024-04-22T13:27:09.562941+0000","last_fullsized":"2024-04-22T13:27:09.562941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:38.583578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.563605+0000","last_change":"2024-04-22T13:25:10.892478+0000","last_active":"2024-04-22T13:27:09.563605+0000","last_peered":"2024-04-22T13:27:09.563605+0000","last_clean":"2024-04-22T13:27:09.563605+0000","last_became_active":"2024-04-22T13:25:10.891771+0000","last_became_peered":"2024-04-22T13:25:10.891771+0000","last_unstale":"2024-04-22T13:27:09.563605+0000","last_undegraded":"2024-04-22T13:27:09.563605+0000","last_fullsized":"2024-04-22T13:27:09.563605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:06.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-22T13:27:09.562834+0000","last_change":"2024-04-22T13:25:10.901864+0000","last_active":"2024-04-22T13:27:09.562834+0000","last_peered":"2024-04-22T13:27:09.562834+0000","last_clean":"2024-04-22T13:27:09.562834+0000","last_became_active":"2024-04-22T13:25:10.901536+0000","last_became_peered":"2024-04-22T13:25:10.901536+0000","last_unstale":"2024-04-22T13:27:09.562834+0000","last_undegraded":"2024-04-22T13:27:09.562834+0000","last_fullsized":"2024-04-22T13:27:09.562834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:25:09.856868+0000","last_clean_scrub_stamp":"2024-04-22T13:25:09.856868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:57:25.876620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799208,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1635,"num_read_kb":1041,"num_write":503,"num_write_kb":2383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2397624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":382,"ondisk_log_size":382,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":485517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514538,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41620,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703532,"statfs":{"total":95995035648,"available":95952416768,"internally_reserved":0,"allocated":2052096,"data_stored":1472231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678064,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40892,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704260,"statfs":{"total":95995035648,"available":95953162240,"internally_reserved":0,"allocated":1429504,"data_stored":877573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939702,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36728,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708424,"statfs":{"total":95995035648,"available":95957426176,"internally_reserved":0,"allocated":1425408,"data_stored":877951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41088,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704064,"statfs":{"total":95995035648,"available":95952961536,"internally_reserved":0,"allocated":1630208,"data_stored":1078567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":554312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":486503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":555763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41336,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703816,"statfs":{"total":95995035648,"available":95952707584,"internally_reserved":0,"allocated":1761280,"data_stored":1205347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37384,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707768,"statfs":{"total":95995035648,"available":95956754432,"internally_reserved":0,"allocated":1974272,"data_stored":1405695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":398511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:27:21.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.984+0000 7f2d403bd640 1 -- 172.21.15.5:0/1516243089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 msgr2=0x7f2d3806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.984+0000 7f2d403bd640 1 --2- 172.21.15.5:0/1516243089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d3806aac0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009930 tx=0x7f2d2802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- 172.21.15.5:0/1516243089 shutdown_connections 2024-04-22T13:27:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 --2- 172.21.15.5:0/1516243089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d3806aac0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 --2- 172.21.15.5:0/1516243089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d3806b000 0x7f2d3810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 --2- 172.21.15.5:0/1516243089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38108060 0x7f2d38108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- 172.21.15.5:0/1516243089 >> 172.21.15.5:0/1516243089 conn(0x7f2d38077c40 msgr2=0x7f2d38078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- 172.21.15.5:0/1516243089 shutdown_connections 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- 172.21.15.5:0/1516243089 wait complete. 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 Processor -- start 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- start start 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d381986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d3806b000 0x7f2d38198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38108060 0x7f2d3819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381121a0 con 0x7f2d3806a640 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38112020 con 0x7f2d3806b000 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.985+0000 7f2d403bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38112320 con 0x7f2d38108060 2024-04-22T13:27:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d381986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d381986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46812/0 (socket says 172.21.15.5:46812) 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 -- 172.21.15.5:0/2601382086 learned_addr learned my addr 172.21.15.5:0/2601382086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3d958640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d3806b000 0x7f2d38198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e95a640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38108060 0x7f2d3819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 -- 172.21.15.5:0/2601382086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38108060 msgr2=0x7f2d3819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38108060 0x7f2d3819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 -- 172.21.15.5:0/2601382086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d3806b000 msgr2=0x7f2d38198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d3806b000 0x7f2d38198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 -- 172.21.15.5:0/2601382086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3819d690 con 0x7f2d3806a640 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d3e159640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d381986a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400d770 tx=0x7f2d3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34017070 con 0x7f2d3806a640 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d340054d0 con 0x7f2d3806a640 2024-04-22T13:27:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3819d980 con 0x7f2d3806a640 2024-04-22T13:27:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34004c30 con 0x7f2d3806a640 2024-04-22T13:27:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.986+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d381a5210 con 0x7f2d3806a640 2024-04-22T13:27:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.987+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d38103650 con 0x7f2d3806a640 2024-04-22T13:27:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.988+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f2d34004dd0 con 0x7f2d3806a640 2024-04-22T13:27:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.989+0000 7f2d2f7fe640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d240a3ec0 0x7f2d240a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.989+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f2d340d0850 con 0x7f2d3806a640 2024-04-22T13:27:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.989+0000 7f2d3d958640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d240a3ec0 0x7f2d240a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.990+0000 7f2d3d958640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d240a3ec0 0x7f2d240a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005ab0 tx=0x7f2d28005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:21.993+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d3409c700 con 0x7f2d3806a640 2024-04-22T13:27:22.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.217+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d38069250 con 0x7f2d3806a640 2024-04-22T13:27:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.218+0000 7f2d2f7fe640 1 -- 172.21.15.5:0/2601382086 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f2d340a0a00 con 0x7f2d3806a640 2024-04-22T13:27:22.214 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:22.214 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d240a3ec0 msgr2=0x7f2d240a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d240a3ec0 0x7f2d240a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005ab0 tx=0x7f2d28005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 msgr2=0x7f2d381986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d381986a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400d770 tx=0x7f2d3400dc30 comp rx=0 tx=0).stop 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 shutdown_connections 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d240a3ec0 0x7f2d240a6380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38108060 0x7f2d3819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d3806b000 0x7f2d38198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 --2- 172.21.15.5:0/2601382086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3806a640 0x7f2d381986a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 >> 172.21.15.5:0/2601382086 conn(0x7f2d38077c40 msgr2=0x7f2d380ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 shutdown_connections 2024-04-22T13:27:22.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.220+0000 7f2d403bd640 1 -- 172.21.15.5:0/2601382086 wait complete. 2024-04-22T13:27:22.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/2741429162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806a640 msgr2=0x7f6f0806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/2741429162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806a640 0x7f6f0806aac0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009930 tx=0x7f6ef002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/2741429162 shutdown_connections 2024-04-22T13:27:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/2741429162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806a640 0x7f6f0806aac0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/2741429162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0806b000 0x7f6f0810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/2741429162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f08108060 0x7f6f08108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/2741429162 >> 172.21.15.5:0/2741429162 conn(0x7f6f08077c40 msgr2=0x7f6f08078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/2741429162 shutdown_connections 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.335+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/2741429162 wait complete. 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 Processor -- start 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 -- start start 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0806a640 0x7f6f08198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 0x7f6f08198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f08108060 0x7f6f0819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08112180 con 0x7f6f0806b000 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08112000 con 0x7f6f0806a640 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0d0fd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08112300 con 0x7f6f08108060 2024-04-22T13:27:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 0x7f6f08198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 0x7f6f08198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46830/0 (socket says 172.21.15.5:46830) 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 -- 172.21.15.5:0/441197633 learned_addr learned my addr 172.21.15.5:0/441197633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0759e640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f08108060 0x7f6f0819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f06d9d640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0806a640 0x7f6f08198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 -- 172.21.15.5:0/441197633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f08108060 msgr2=0x7f6f0819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f08108060 0x7f6f0819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 -- 172.21.15.5:0/441197633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0806a640 msgr2=0x7f6f08198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.336+0000 7f6f0659c640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0806a640 0x7f6f08198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6f0659c640 1 -- 172.21.15.5:0/441197633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0819d710 con 0x7f6f0806b000 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6f0659c640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 0x7f6f08198c60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009230 tx=0x7f6ef002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0043020 con 0x7f6f0806b000 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0819d9a0 con 0x7f6f0806b000 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef002caf0 con 0x7f6f0806b000 2024-04-22T13:27:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0003af0 con 0x7f6f0806b000 2024-04-22T13:27:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.337+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081a5290 con 0x7f6f0806b000 2024-04-22T13:27:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.338+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f08103650 con 0x7f6f0806b000 2024-04-22T13:27:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.339+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f6ef0046020 con 0x7f6f0806b000 2024-04-22T13:27:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.340+0000 7f6efffff640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ef40a3e40 0x7f6ef40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.340+0000 7f6f06d9d640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ef40a3e40 0x7f6ef40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.340+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6ef00f2dd0 con 0x7f6f0806b000 2024-04-22T13:27:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.340+0000 7f6f06d9d640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ef40a3e40 0x7f6ef40a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800bb60 tx=0x7f6ef800b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.343+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ef004d020 con 0x7f6f0806b000 2024-04-22T13:27:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.531+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f0810a4d0 con 0x7f6f0806b000 2024-04-22T13:27:22.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.533+0000 7f6efffff640 1 -- 172.21.15.5:0/441197633 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7f6ef00dde20 con 0x7f6f0806b000 2024-04-22T13:27:22.529 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:22.529 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ef40a3e40 msgr2=0x7f6ef40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ef40a3e40 0x7f6ef40a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800bb60 tx=0x7f6ef800b390 comp rx=0 tx=0).stop 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 msgr2=0x7f6f08198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 0x7f6f08198c60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009230 tx=0x7f6ef002c710 comp rx=0 tx=0).stop 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 shutdown_connections 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0806b000 0x7f6f08198c60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ef40a3e40 0x7f6ef40a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f08108060 0x7f6f0819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 --2- 172.21.15.5:0/441197633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f0806a640 0x7f6f08198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 >> 172.21.15.5:0/441197633 conn(0x7f6f08077c40 msgr2=0x7f6f080ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 shutdown_connections 2024-04-22T13:27:22.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.535+0000 7f6f0d0fd640 1 -- 172.21.15.5:0/441197633 wait complete. 2024-04-22T13:27:22.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:27:22.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 -- 172.21.15.5:0/2795186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670010c6d0 msgr2=0x7f670010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 --2- 172.21.15.5:0/2795186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670010c6d0 0x7f670010eb60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009ca0 tx=0x7f66f802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 -- 172.21.15.5:0/2795186216 shutdown_connections 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 --2- 172.21.15.5:0/2795186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670010c6d0 0x7f670010eb60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 --2- 172.21.15.5:0/2795186216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6700103b70 0x7f670010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 --2- 172.21.15.5:0/2795186216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67001031c0 0x7f67001035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 -- 172.21.15.5:0/2795186216 >> 172.21.15.5:0/2795186216 conn(0x7f6700077f70 msgr2=0x7f6700078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:22.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 -- 172.21.15.5:0/2795186216 shutdown_connections 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 -- 172.21.15.5:0/2795186216 wait complete. 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 Processor -- start 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.652+0000 7f6705c06640 1 -- start start 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f6705c06640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67001031c0 0x7f670019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f6705c06640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f670010c6d0 0x7f670019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f6705c06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 0x7f6700198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f6705c06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67001123f0 con 0x7f6700198750 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f6705c06640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6700112270 con 0x7f67001031c0 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f6705c06640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6700112570 con 0x7f670010c6d0 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 0x7f6700198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 0x7f6700198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46848/0 (socket says 172.21.15.5:46848) 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 -- 172.21.15.5:0/672966516 learned_addr learned my addr 172.21.15.5:0/672966516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:22.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66ff7fe640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67001031c0 0x7f670019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66feffd640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f670010c6d0 0x7f670019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 -- 172.21.15.5:0/672966516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f670010c6d0 msgr2=0x7f670019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f670010c6d0 0x7f670019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 -- 172.21.15.5:0/672966516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67001031c0 msgr2=0x7f670019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67001031c0 0x7f670019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 -- 172.21.15.5:0/672966516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6700199350 con 0x7f6700198750 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66ff7fe640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67001031c0 0x7f670019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66feffd640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f670010c6d0 0x7f670019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fffff640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 0x7f6700198bd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005e30 tx=0x7f66f8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8041020 con 0x7f6700198750 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.653+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f8032970 con 0x7f6700198750 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.654+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8039040 con 0x7f6700198750 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.654+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67001995e0 con 0x7f6700198750 2024-04-22T13:27:22.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.654+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67001a5770 con 0x7f6700198750 2024-04-22T13:27:22.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.655+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670006ae40 con 0x7f6700198750 2024-04-22T13:27:22.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.656+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f66f8044020 con 0x7f6700198750 2024-04-22T13:27:22.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.656+0000 7f66fcff9640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66e80a3ec0 0x7f66e80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.656+0000 7f66ff7fe640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66e80a3ec0 0x7f66e80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.656+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f66f80f2bc0 con 0x7f6700198750 2024-04-22T13:27:22.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.657+0000 7f66ff7fe640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66e80a3ec0 0x7f66e80a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009930 tx=0x7f66f0008040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:22.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.660+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66f804b020 con 0x7f6700198750 2024-04-22T13:27:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.847+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6700107680 con 0x7f6700198750 2024-04-22T13:27:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.847+0000 7f66fcff9640 1 -- 172.21.15.5:0/672966516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f66f8045e40 con 0x7f6700198750 2024-04-22T13:27:22.844 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66e80a3ec0 msgr2=0x7f66e80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66e80a3ec0 0x7f66e80a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009930 tx=0x7f66f0008040 comp rx=0 tx=0).stop 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 msgr2=0x7f6700198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 0x7f6700198bd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005e30 tx=0x7f66f8034670 comp rx=0 tx=0).stop 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 shutdown_connections 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6700198750 0x7f6700198bd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66e80a3ec0 0x7f66e80a6380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f670010c6d0 0x7f670019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 --2- 172.21.15.5:0/672966516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67001031c0 0x7f670019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 >> 172.21.15.5:0/672966516 conn(0x7f6700077f70 msgr2=0x7f67000fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 shutdown_connections 2024-04-22T13:27:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.850+0000 7f6705c06640 1 -- 172.21.15.5:0/672966516 wait complete. 2024-04-22T13:27:22.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:27:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 -- 172.21.15.5:0/3792949163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102b30 msgr2=0x7f4960106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 --2- 172.21.15.5:0/3792949163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102b30 0x7f4960106b70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4950009900 tx=0x7f495002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 -- 172.21.15.5:0/3792949163 shutdown_connections 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 --2- 172.21.15.5:0/3792949163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102b30 0x7f4960106b70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 --2- 172.21.15.5:0/3792949163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4960107250 0x7f496010ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 --2- 172.21.15.5:0/3792949163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960102180 0x7f4960102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 -- 172.21.15.5:0/3792949163 >> 172.21.15.5:0/3792949163 conn(0x7f49600fda00 msgr2=0x7f49600ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 -- 172.21.15.5:0/3792949163 shutdown_connections 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 -- 172.21.15.5:0/3792949163 wait complete. 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 Processor -- start 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.966+0000 7f49683c7640 1 -- start start 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 0x7f496019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960107250 0x7f496019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49601985b0 0x7f4960198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4960112190 con 0x7f4960102180 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4960112010 con 0x7f49601985b0 2024-04-22T13:27:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4960112310 con 0x7f4960107250 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 0x7f496019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 0x7f496019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46864/0 (socket says 172.21.15.5:46864) 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 -- 172.21.15.5:0/482756421 learned_addr learned my addr 172.21.15.5:0/482756421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4965962640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960107250 0x7f496019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966964640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49601985b0 0x7f4960198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 -- 172.21.15.5:0/482756421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960107250 msgr2=0x7f496019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960107250 0x7f496019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 -- 172.21.15.5:0/482756421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49601985b0 msgr2=0x7f4960198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49601985b0 0x7f4960198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 -- 172.21.15.5:0/482756421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49601990d0 con 0x7f4960102180 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966964640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49601985b0 0x7f4960198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f4966163640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 0x7f496019e400 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d770 tx=0x7f495c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c017070 con 0x7f4960102180 2024-04-22T13:27:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49601993c0 con 0x7f4960102180 2024-04-22T13:27:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f495c0054d0 con 0x7f4960102180 2024-04-22T13:27:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.967+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c004c30 con 0x7f4960102180 2024-04-22T13:27:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.968+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49601a55c0 con 0x7f4960102180 2024-04-22T13:27:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.968+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4960069250 con 0x7f4960102180 2024-04-22T13:27:22.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.970+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f495c004dd0 con 0x7f4960102180 2024-04-22T13:27:22.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.970+0000 7f49577fe640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f494c0a3e40 0x7f494c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.971+0000 7f4965962640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f494c0a3e40 0x7f494c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.971+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f495c0d0990 con 0x7f4960102180 2024-04-22T13:27:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.971+0000 7f4965962640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f494c0a3e40 0x7f494c0a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4950009230 tx=0x7f4950005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:22.974+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f495c09ce10 con 0x7f4960102180 2024-04-22T13:27:23.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.160+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4960102b30 con 0x7f4960102180 2024-04-22T13:27:23.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.160+0000 7f49577fe640 1 -- 172.21.15.5:0/482756421 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f495c09c710 con 0x7f4960102180 2024-04-22T13:27:23.157 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:27:23.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f494c0a3e40 msgr2=0x7f494c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f494c0a3e40 0x7f494c0a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4950009230 tx=0x7f4950005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 msgr2=0x7f496019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 0x7f496019e400 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d770 tx=0x7f495c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 shutdown_connections 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f494c0a3e40 0x7f494c0a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49601985b0 0x7f4960198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960107250 0x7f496019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 --2- 172.21.15.5:0/482756421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4960102180 0x7f496019e400 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 >> 172.21.15.5:0/482756421 conn(0x7f49600fda00 msgr2=0x7f49600fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 shutdown_connections 2024-04-22T13:27:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.162+0000 7f49683c7640 1 -- 172.21.15.5:0/482756421 wait complete. 2024-04-22T13:27:23.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 56 op/s 2024-04-22T13:27:23.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: from='client.15660 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2601382086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:23.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/441197633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:23.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:22 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/672966516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:23.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 56 op/s 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: from='client.15660 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2601382086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/441197633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:23.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:22 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/672966516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 -- 172.21.15.5:0/1246589588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108103170 msgr2=0x7fa108103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 --2- 172.21.15.5:0/1246589588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108103170 0x7fa108103550 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009900 tx=0x7fa0fc02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 -- 172.21.15.5:0/1246589588 shutdown_connections 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 --2- 172.21.15.5:0/1246589588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10810c540 0x7fa10810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 --2- 172.21.15.5:0/1246589588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103a90 0x7fa10810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 --2- 172.21.15.5:0/1246589588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108103170 0x7fa108103550 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 -- 172.21.15.5:0/1246589588 >> 172.21.15.5:0/1246589588 conn(0x7fa108077fd0 msgr2=0x7fa1080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 -- 172.21.15.5:0/1246589588 shutdown_connections 2024-04-22T13:27:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 -- 172.21.15.5:0/1246589588 wait complete. 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 Processor -- start 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.280+0000 7fa10eff4640 1 -- start start 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103170 0x7fa1081986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa108103a90 0x7fa108198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 0x7fa10819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081121a0 con 0x7fa10810c540 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112020 con 0x7fa108103170 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112320 con 0x7fa108103a90 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 0x7fa10819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 0x7fa10819cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46884/0 (socket says 172.21.15.5:46884) 2024-04-22T13:27:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 -- 172.21.15.5:0/689412766 learned_addr learned my addr 172.21.15.5:0/689412766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa107fff640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa108103a90 0x7fa108198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10cd90640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103170 0x7fa1081986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 -- 172.21.15.5:0/689412766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa108103a90 msgr2=0x7fa108198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa108103a90 0x7fa108198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 -- 172.21.15.5:0/689412766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103170 msgr2=0x7fa1081986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103170 0x7fa1081986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 -- 172.21.15.5:0/689412766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819d6b0 con 0x7fa10810c540 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10cd90640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103170 0x7fa1081986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10d591640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 0x7fa10819cf80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa10000eaa0 tx=0x7fa10000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100004d20 con 0x7fa10810c540 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819d9a0 con 0x7fa10810c540 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa100005020 con 0x7fa10810c540 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.281+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000116d0 con 0x7fa10810c540 2024-04-22T13:27:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.282+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a5230 con 0x7fa10810c540 2024-04-22T13:27:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.282+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa108069250 con 0x7fa10810c540 2024-04-22T13:27:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.284+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fa10000caf0 con 0x7fa10810c540 2024-04-22T13:27:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.284+0000 7fa105ffb640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa0f80a3e40 0x7fa0f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.284+0000 7fa10cd90640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa0f80a3e40 0x7fa0f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.285+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa1000d0ba0 con 0x7fa10810c540 2024-04-22T13:27:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.285+0000 7fa10cd90640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa0f80a3e40 0x7fa0f80a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005ab0 tx=0x7fa0fc005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 56 op/s 2024-04-22T13:27:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: from='client.15660 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:27:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2601382086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/441197633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:23.285 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:22 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/672966516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:27:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.289+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa100016030 con 0x7fa10810c540 2024-04-22T13:27:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.475+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa108107690 con 0x7fa10810c540 2024-04-22T13:27:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.475+0000 7fa105ffb640 1 -- 172.21.15.5:0/689412766 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fa10009d020 con 0x7fa10810c540 2024-04-22T13:27:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:27:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa0f80a3e40 msgr2=0x7fa0f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa0f80a3e40 0x7fa0f80a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005ab0 tx=0x7fa0fc005a20 comp rx=0 tx=0).stop 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 msgr2=0x7fa10819cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 0x7fa10819cf80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa10000eaa0 tx=0x7fa10000eea0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 shutdown_connections 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa10810c540 0x7fa10819cf80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa0f80a3e40 0x7fa0f80a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa108103a90 0x7fa108198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 --2- 172.21.15.5:0/689412766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa108103170 0x7fa1081986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 >> 172.21.15.5:0/689412766 conn(0x7fa108077fd0 msgr2=0x7fa1080fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 shutdown_connections 2024-04-22T13:27:23.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.477+0000 7fa10eff4640 1 -- 172.21.15.5:0/689412766 wait complete. 2024-04-22T13:27:23.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 -- 172.21.15.5:0/212696563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae81010a0 msgr2=0x7f9ae8101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/212696563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae81010a0 0x7f9ae8101500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009900 tx=0x7f9adc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 -- 172.21.15.5:0/212696563 shutdown_connections 2024-04-22T13:27:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/212696563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae81010a0 0x7f9ae8101500 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/212696563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8101a40 0x7f9ae810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/212696563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 -- 172.21.15.5:0/212696563 >> 172.21.15.5:0/212696563 conn(0x7f9ae8077c40 msgr2=0x7f9ae8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 -- 172.21.15.5:0/212696563 shutdown_connections 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 -- 172.21.15.5:0/212696563 wait complete. 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 Processor -- start 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.591+0000 7f9aecc8c640 1 -- start start 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9aecc8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81010a0 0x7f9ae8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9aecc8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ae8101a40 0x7f9ae8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9aecc8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9aecc8c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8111180 con 0x7f9ae8105dd0 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9aecc8c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8111000 con 0x7f9ae8101a40 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9aecc8c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8111300 con 0x7f9ae81010a0 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46910/0 (socket says 172.21.15.5:46910) 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 -- 172.21.15.5:0/3133904417 learned_addr learned my addr 172.21.15.5:0/3133904417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae5d9b640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ae8101a40 0x7f9ae8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae659c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81010a0 0x7f9ae8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 -- 172.21.15.5:0/3133904417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81010a0 msgr2=0x7f9ae8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81010a0 0x7f9ae8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 -- 172.21.15.5:0/3133904417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ae8101a40 msgr2=0x7f9ae8199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ae8101a40 0x7f9ae8199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 -- 172.21.15.5:0/3133904417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae81a4be0 con 0x7f9ae8105dd0 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae659c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81010a0 0x7f9ae8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ae6d9d640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81a46a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000cad0 tx=0x7f9ae000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.592+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000ae20 con 0x7f9ae8105dd0 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.593+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae0005020 con 0x7f9ae8105dd0 2024-04-22T13:27:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.593+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae81a4db0 con 0x7f9ae8105dd0 2024-04-22T13:27:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.593+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000f6b0 con 0x7f9ae8105dd0 2024-04-22T13:27:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.593+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81a52d0 con 0x7f9ae8105dd0 2024-04-22T13:27:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.594+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae8102540 con 0x7f9ae8105dd0 2024-04-22T13:27:23.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.595+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f9ae00051d0 con 0x7f9ae8105dd0 2024-04-22T13:27:23.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.595+0000 7f9ad77fe640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ad00a3ec0 0x7f9ad00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.596+0000 7f9ae659c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ad00a3ec0 0x7f9ad00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.596+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9ae00d0d90 con 0x7f9ae8105dd0 2024-04-22T13:27:23.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.596+0000 7f9ae659c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ad00a3ec0 0x7f9ad00a6380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009ab0 tx=0x7f9ad80094b0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.599+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ae0014030 con 0x7f9ae8105dd0 2024-04-22T13:27:23.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.823+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ae8069250 con 0x7f9ae8105dd0 2024-04-22T13:27:23.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.824+0000 7f9ad77fe640 1 -- 172.21.15.5:0/3133904417 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f9ae009d290 con 0x7f9ae8105dd0 2024-04-22T13:27:23.820 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:23.820 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ad00a3ec0 msgr2=0x7f9ad00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ad00a3ec0 0x7f9ad00a6380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009ab0 tx=0x7f9ad80094b0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 msgr2=0x7f9ae81a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81a46a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000cad0 tx=0x7f9ae000cf90 comp rx=0 tx=0).stop 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 shutdown_connections 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ae8105dd0 0x7f9ae81a46a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ad00a3ec0 0x7f9ad00a6380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ae8101a40 0x7f9ae8199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 --2- 172.21.15.5:0/3133904417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81010a0 0x7f9ae8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 >> 172.21.15.5:0/3133904417 conn(0x7f9ae8077c40 msgr2=0x7f9ae80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 shutdown_connections 2024-04-22T13:27:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.826+0000 7f9aecc8c640 1 -- 172.21.15.5:0/3133904417 wait complete. 2024-04-22T13:27:23.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:27:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 -- 172.21.15.5:0/1533607098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c0102b50 msgr2=0x7f42c0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 --2- 172.21.15.5:0/1533607098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c0102b50 0x7f42c0106b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009900 tx=0x7f42b402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 -- 172.21.15.5:0/1533607098 shutdown_connections 2024-04-22T13:27:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 --2- 172.21.15.5:0/1533607098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c0102b50 0x7f42c0106b90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 --2- 172.21.15.5:0/1533607098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c0107270 0x7f42c010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 --2- 172.21.15.5:0/1533607098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c01021a0 0x7f42c0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 -- 172.21.15.5:0/1533607098 >> 172.21.15.5:0/1533607098 conn(0x7f42c00793e0 msgr2=0x7f42c00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 -- 172.21.15.5:0/1533607098 shutdown_connections 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.940+0000 7f42c642e640 1 -- 172.21.15.5:0/1533607098 wait complete. 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 Processor -- start 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 -- start start 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c01021a0 0x7f42c0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c0107270 0x7f42c0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 0x7f42c019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01121b0 con 0x7f42c019cff0 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0112030 con 0x7f42c01021a0 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c642e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0112330 con 0x7f42c0107270 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 0x7f42c019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 0x7f42c019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46930/0 (socket says 172.21.15.5:46930) 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 -- 172.21.15.5:0/3922175716 learned_addr learned my addr 172.21.15.5:0/3922175716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42bf7fe640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c0107270 0x7f42c0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42bffff640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c01021a0 0x7f42c0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 -- 172.21.15.5:0/3922175716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c0107270 msgr2=0x7f42c0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c0107270 0x7f42c0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 -- 172.21.15.5:0/3922175716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c01021a0 msgr2=0x7f42c0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c01021a0 0x7f42c0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.941+0000 7f42c49cb640 1 -- 172.21.15.5:0/3922175716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c019dae0 con 0x7f42c019cff0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.942+0000 7f42c49cb640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 0x7f42c019d4a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f42b800cad0 tx=0x7f42b800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.942+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b800ae20 con 0x7f42c019cff0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.942+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b8005020 con 0x7f42c019cff0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.942+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c019ddd0 con 0x7f42c019cff0 2024-04-22T13:27:23.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.942+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b800f6b0 con 0x7f42c019cff0 2024-04-22T13:27:23.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.942+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c01a5540 con 0x7f42c019cff0 2024-04-22T13:27:23.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.943+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c0069250 con 0x7f42c019cff0 2024-04-22T13:27:23.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.944+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f42b80051d0 con 0x7f42c019cff0 2024-04-22T13:27:23.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.945+0000 7f42bd7fa640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42a80a3e40 0x7f42a80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:23.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.945+0000 7f42bffff640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42a80a3e40 0x7f42a80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:23.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.945+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f42b80d0a90 con 0x7f42c019cff0 2024-04-22T13:27:23.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.945+0000 7f42bffff640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42a80a3e40 0x7f42a80a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42b0005fd0 tx=0x7f42b000b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:23.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:23.949+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42b8014030 con 0x7f42c019cff0 2024-04-22T13:27:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.137+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42c0102b50 con 0x7f42c019cff0 2024-04-22T13:27:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.138+0000 7f42bd7fa640 1 -- 172.21.15.5:0/3922175716 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22921 (secure 0 0 0) 0x7f42b809c860 con 0x7f42c019cff0 2024-04-22T13:27:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:24.135 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:27:09.442034+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:27:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42a80a3e40 msgr2=0x7f42a80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42a80a3e40 0x7f42a80a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42b0005fd0 tx=0x7f42b000b390 comp rx=0 tx=0).stop 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 msgr2=0x7f42c019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 0x7f42c019d4a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f42b800cad0 tx=0x7f42b800cf90 comp rx=0 tx=0).stop 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 shutdown_connections 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c019cff0 0x7f42c019d4a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42a80a3e40 0x7f42a80a6300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c0107270 0x7f42c0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 --2- 172.21.15.5:0/3922175716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42c01021a0 0x7f42c0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 >> 172.21.15.5:0/3922175716 conn(0x7f42c00793e0 msgr2=0x7f42c00fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 shutdown_connections 2024-04-22T13:27:24.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.140+0000 7f42c642e640 1 -- 172.21.15.5:0/3922175716 wait complete. 2024-04-22T13:27:24.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/482756421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:24.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/689412766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:24.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:23 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3133904417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:24.148 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 918426 2024-04-22T13:27:24.148 DEBUG:tasks.quiescer:Yielding 2024-04-22T13:27:24.148 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T13:27:24.163 INFO:tasks.workunit:Pulling workunits from ref d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:27:24.163 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T13:27:24.163 INFO:tasks.workunit:timeout=3h 2024-04-22T13:27:24.164 INFO:tasks.workunit:cleanup=True 2024-04-22T13:27:24.164 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:24.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-22 13:25:18.059187054 +0000 2024-04-22T13:27:24.180 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-22 13:25:18.059187054 +0000 2024-04-22T13:27:24.181 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-22 13:27:07.046800289 +0000 2024-04-22T13:27:24.181 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-22 13:25:18.059187054 +0000 2024-04-22T13:27:24.181 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T13:27:24.181 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T13:27:24.256 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:27:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.271+0000 7fa6d3767640 1 -- 172.21.15.5:0/731013952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc069f10 msgr2=0x7fa6cc106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.271+0000 7fa6d3767640 1 --2- 172.21.15.5:0/731013952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc069f10 0x7fa6cc106be0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009900 tx=0x7fa6bc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.271+0000 7fa6d3767640 1 -- 172.21.15.5:0/731013952 shutdown_connections 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.271+0000 7fa6d3767640 1 --2- 172.21.15.5:0/731013952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc069f10 0x7fa6cc106be0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.271+0000 7fa6d3767640 1 --2- 172.21.15.5:0/731013952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6cc1072c0 0x7fa6cc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.271+0000 7fa6d3767640 1 --2- 172.21.15.5:0/731013952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc069560 0x7fa6cc069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- 172.21.15.5:0/731013952 >> 172.21.15.5:0/731013952 conn(0x7fa6cc0771d0 msgr2=0x7fa6cc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- 172.21.15.5:0/731013952 shutdown_connections 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- 172.21.15.5:0/731013952 wait complete. 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 Processor -- start 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- start start 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6cc069560 0x7fa6cc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc1072c0 0x7fa6cc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 0x7fa6cc19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc112180 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc112000 con 0x7fa6cc069560 2024-04-22T13:27:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.272+0000 7fa6d3767640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc112300 con 0x7fa6cc1072c0 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 0x7fa6cc19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 0x7fa6cc19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46950/0 (socket says 172.21.15.5:46950) 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 -- 172.21.15.5:0/469899295 learned_addr learned my addr 172.21.15.5:0/469899295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1503640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6cc069560 0x7fa6cc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d0d02640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc1072c0 0x7fa6cc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 -- 172.21.15.5:0/469899295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc1072c0 msgr2=0x7fa6cc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc1072c0 0x7fa6cc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 -- 172.21.15.5:0/469899295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6cc069560 msgr2=0x7fa6cc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6cc069560 0x7fa6cc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 -- 172.21.15.5:0/469899295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc19da90 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d1d04640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 0x7fa6cc19d450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400cad0 tx=0x7fa6c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400ae20 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc19dd80 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c4005020 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400f6b0 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.273+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc1a5540 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.274+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc068e00 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:23 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/482756421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:24.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:23 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/689412766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:24.273 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:23 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3133904417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.276+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fa6c40051d0 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.276+0000 7fa6c27fc640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6b80a3e40 0x7fa6b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.276+0000 7fa6d1503640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6b80a3e40 0x7fa6b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.276+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa6c40d0e10 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.277+0000 7fa6d1503640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6b80a3e40 0x7fa6b80a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005fd0 tx=0x7fa6c800b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.280+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6c4014030 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.283 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T13:27:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:23 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/482756421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:27:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:23 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/689412766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:27:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:23 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3133904417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:24.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.506+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6cc069f10 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.507+0000 7fa6c27fc640 1 -- 172.21.15.5:0/469899295 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7fa6c409cc40 con 0x7fa6cc19cfa0 2024-04-22T13:27:24.503 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:24.503 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6b80a3e40 msgr2=0x7fa6b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6b80a3e40 0x7fa6b80a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005fd0 tx=0x7fa6c800b390 comp rx=0 tx=0).stop 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 msgr2=0x7fa6cc19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 0x7fa6cc19d450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400cad0 tx=0x7fa6c400cf90 comp rx=0 tx=0).stop 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 shutdown_connections 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6cc19cfa0 0x7fa6cc19d450 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6b80a3e40 0x7fa6b80a6300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc1072c0 0x7fa6cc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 --2- 172.21.15.5:0/469899295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6cc069560 0x7fa6cc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 >> 172.21.15.5:0/469899295 conn(0x7fa6cc0771d0 msgr2=0x7fa6cc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 shutdown_connections 2024-04-22T13:27:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.510+0000 7fa6d3767640 1 -- 172.21.15.5:0/469899295 wait complete. 2024-04-22T13:27:24.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T13:25:10.226899+0000', 'modified': '2024-04-22T13:27:07.748442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24512, 'mds_2': 24496, 'mds_3': 24485, 'mds_4': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14550': {'gid': 14550, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/1096407808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 1096407808}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 1096407808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/373669901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 373669901}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 373669901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/4083506255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 4083506255}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 4083506255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6833/1584198825', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 1584198825}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 1584198825}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/2603097758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2603097758}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2603097758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.107:6837/207360580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 207360580}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 207360580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.49:6835/827214764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 827214764}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 827214764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.107:6839/3107882391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3107882391}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3107882391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6837/164218196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 164218196}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 164218196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6839/1393810832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 1393810832}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 1393810832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24512, 24557, 24496, 24485, 24469]} 2024-04-22T13:27:24.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T13:27:24.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T13:27:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 -- 172.21.15.5:0/1659396300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 msgr2=0x7efe981012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 --2- 172.21.15.5:0/1659396300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981012b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe80009900 tx=0x7efe8002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 -- 172.21.15.5:0/1659396300 shutdown_connections 2024-04-22T13:27:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 --2- 172.21.15.5:0/1659396300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 0x7efe9810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 --2- 172.21.15.5:0/1659396300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe98101880 0x7efe981058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 --2- 172.21.15.5:0/1659396300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981012b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 -- 172.21.15.5:0/1659396300 >> 172.21.15.5:0/1659396300 conn(0x7efe98077a00 msgr2=0x7efe980fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 -- 172.21.15.5:0/1659396300 shutdown_connections 2024-04-22T13:27:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 -- 172.21.15.5:0/1659396300 wait complete. 2024-04-22T13:27:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.668+0000 7efe9da01640 1 Processor -- start 2024-04-22T13:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 -- start start 2024-04-22T13:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 0x7efe981a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe9819a610 0x7efe9819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe98110f00 con 0x7efe98100ed0 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efe98110d80 con 0x7efe9819a610 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe9da01640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe98111080 con 0x7efe98105fa0 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46966/0 (socket says 172.21.15.5:46966) 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 -- 172.21.15.5:0/1238381199 learned_addr learned my addr 172.21.15.5:0/1238381199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe977fe640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe9819a610 0x7efe9819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe967fc640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 0x7efe981a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 -- 172.21.15.5:0/1238381199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 msgr2=0x7efe981a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 0x7efe981a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 -- 172.21.15.5:0/1238381199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe9819a610 msgr2=0x7efe9819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe9819a610 0x7efe9819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe96ffd640 1 -- 172.21.15.5:0/1238381199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9819b130 con 0x7efe98100ed0 2024-04-22T13:27:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe967fc640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 0x7efe981a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.669+0000 7efe977fe640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe9819a610 0x7efe9819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.670+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981a0460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efe80005940 tx=0x7efe8002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.670+0000 7efe9c9ff640 1 -- 172.21.15.5:0/1238381199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80043020 con 0x7efe98100ed0 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.670+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9819b3c0 con 0x7efe98100ed0 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.670+0000 7efe9c9ff640 1 -- 172.21.15.5:0/1238381199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe8002cbe0 con 0x7efe98100ed0 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.670+0000 7efe9c9ff640 1 -- 172.21.15.5:0/1238381199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80003bf0 con 0x7efe98100ed0 2024-04-22T13:27:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.670+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a7660 con 0x7efe98100ed0 2024-04-22T13:27:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.672+0000 7efe9c9ff640 1 -- 172.21.15.5:0/1238381199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7efe80046020 con 0x7efe98100ed0 2024-04-22T13:27:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.673+0000 7efe9c9ff640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efe840a3e40 0x7efe840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.673+0000 7efe967fc640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efe840a3e40 0x7efe840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.673+0000 7efe9c9ff640 1 -- 172.21.15.5:0/1238381199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7efe800f2cc0 con 0x7efe98100ed0 2024-04-22T13:27:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.673+0000 7efe967fc640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efe840a3e40 0x7efe840a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe9810dfc0 tx=0x7efe88005d30 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.673+0000 7efea05ea740 1 Processor -- start 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 -- start start 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc87528d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc875318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc87534310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc8748cb10 con 0x55dc87529340 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc8742a240 con 0x55dc87528990 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efea05ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc87488ad0 con 0x55dc87531ec0 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc875318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc875318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46972/0 (socket says 172.21.15.5:46972) 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 -- 172.21.15.5:0/1550726888 learned_addr learned my addr 172.21.15.5:0/1550726888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe977fe640 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc87534310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc87528d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 -- 172.21.15.5:0/1550726888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 msgr2=0x55dc87534310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc87534310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 -- 172.21.15.5:0/1550726888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 msgr2=0x55dc87528d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc87528d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.674+0000 7efe967fc640 1 -- 172.21.15.5:0/1550726888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8719f2d0 con 0x55dc87529340 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.675+0000 7efe967fc640 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc875318f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe8800bfd0 tx=0x7efe88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf8bcc0a87fbf59 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.675+0000 7efe8dffb640 1 -- 172.21.15.5:0/1550726888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88020070 con 0x55dc87529340 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.675+0000 7efe8dffb640 1 -- 172.21.15.5:0/1550726888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe8801cb00 con 0x55dc87529340 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.675+0000 7efe8dffb640 1 -- 172.21.15.5:0/1550726888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8801bc10 con 0x55dc87529340 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.675+0000 7efea05ea740 1 -- 172.21.15.5:0/1550726888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 msgr2=0x55dc875318f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.675+0000 7efea05ea740 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc875318f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe8800bfd0 tx=0x7efe88010000 comp rx=0 tx=0).stop 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- 172.21.15.5:0/1550726888 shutdown_connections 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc875318f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc87534310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 --2- 172.21.15.5:0/1550726888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc87528d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- 172.21.15.5:0/1550726888 >> 172.21.15.5:0/1550726888 conn(0x55dc87522c40 msgr2=0x55dc87523020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- 172.21.15.5:0/1550726888 shutdown_connections 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- 172.21.15.5:0/1550726888 wait complete. 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- start start 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 Processor -- start 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc8752ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87529340 0x55dc8752f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87531ec0 0x55dc8752f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc87534e30 con 0x55dc87528990 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc87415190 con 0x55dc87531ec0 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc87534fb0 con 0x55dc87529340 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.676+0000 7efea05ea740 10 client.0 _ll_register_callbacks cb 0x55dc874108b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc8752ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc8752ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46974/0 (socket says 172.21.15.5:46974) 2024-04-22T13:27:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 -- 172.21.15.5:0/2717139543 learned_addr learned my addr 172.21.15.5:0/2717139543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe977fe640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87531ec0 0x55dc8752f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe967fc640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87529340 0x55dc8752f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 -- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87529340 msgr2=0x55dc8752f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87529340 0x55dc8752f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 -- 172.21.15.5:0/2717139543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87531ec0 msgr2=0x55dc8752f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87531ec0 0x55dc8752f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 -- 172.21.15.5:0/2717139543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc872696b0 con 0x55dc87528990 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe977fe640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87531ec0 0x55dc8752f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe96ffd640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc8752ef50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efe800096c0 tx=0x7efe800be580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa10d4f5c615ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe76ffd640 1 -- 172.21.15.5:0/2717139543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800beb50 con 0x55dc87528990 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe76ffd640 1 -- 172.21.15.5:0/2717139543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7efe800df0b0 con 0x55dc87528990 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe8801cb00 con 0x55dc87528990 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efe76ffd640 1 -- 172.21.15.5:0/2717139543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe800c2f00 con 0x55dc87528990 2024-04-22T13:27:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efea05ea740 10 client.15726 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.677+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc875411c0 con 0x55dc87528990 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efe76ffd640 1 -- 172.21.15.5:0/2717139543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe8801cb00 con 0x55dc87528990 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efe76ffd640 1 -- 172.21.15.5:0/2717139543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe800f05a0 con 0x55dc87528990 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efea05ea740 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efea05ea740 10 client.15726 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efea05ea740 20 client.15726 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efea05ea740 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875437b0 0x55dc87544c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efea05ea740 4 client.15726 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8749a530 con 0x55dc875437b0 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.678+0000 7efe967fc640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875437b0 0x55dc87544c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.679+0000 7efe76ffd640 10 client.15726 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.679+0000 7efe967fc640 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875437b0 0x55dc87544c20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bf55bac82d7dee server_cookie=26636940f7995a77 in_seq=0 out_seq=0 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.679+0000 7efe76ffd640 1 -- 172.21.15.5:0/2717139543 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc8749a530 con 0x55dc875437b0 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.679+0000 7efe76ffd640 10 client.15726 handle_command_reply: tid=0 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.679+0000 7efea05ea740 1 client.15726 shutdown 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 msgr2=0x55dc8752ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc8752ef50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efe800096c0 tx=0x7efe800be580 comp rx=0 tx=0).stop 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 shutdown_connections 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87531ec0 0x55dc8752f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87529340 0x55dc8752f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875437b0 0x55dc87544c20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 --2- 172.21.15.5:0/2717139543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc8752ef50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efe76ffd640 0 client.15726 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 >> 172.21.15.5:0/2717139543 conn(0x55dc87522c40 msgr2=0x55dc87524db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 shutdown_connections 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 1 -- 172.21.15.5:0/2717139543 wait complete. 2024-04-22T13:27:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.680+0000 7efea05ea740 20 client.15726 trim_cache size 0 max 16384 2024-04-22T13:27:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.698+0000 7efea05ea740 1 Processor -- start 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.698+0000 7efea05ea740 1 -- start start 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efea05ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc87531680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efea05ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87529340 0x55dc87529770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efea05ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8751d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efea05ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc87542f60 con 0x55dc87528990 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efea05ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc87543630 con 0x55dc87529340 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efea05ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc87543d00 con 0x55dc87531ec0 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc87531680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc87531680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46988/0 (socket says 172.21.15.5:46988) 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 -- 172.21.15.5:0/1177128325 learned_addr learned my addr 172.21.15.5:0/1177128325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe977fe640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8751d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe967fc640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87529340 0x55dc87529770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 -- 172.21.15.5:0/1177128325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 msgr2=0x55dc8751d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8751d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 -- 172.21.15.5:0/1177128325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87529340 msgr2=0x55dc87529770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87529340 0x55dc87529770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 -- 172.21.15.5:0/1177128325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc87529cb0 con 0x55dc87528990 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe977fe640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8751d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe967fc640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87529340 0x55dc87529770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe96ffd640 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc87531680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efe800bb4d0 tx=0x7efe800bff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460b733f1b49dba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.699+0000 7efe8d7fa640 1 -- 172.21.15.5:0/1177128325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80088070 con 0x55dc87528990 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efe8d7fa640 1 -- 172.21.15.5:0/1177128325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe800c77c0 con 0x55dc87528990 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efe8d7fa640 1 -- 172.21.15.5:0/1177128325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800f0740 con 0x55dc87528990 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 -- 172.21.15.5:0/1177128325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 msgr2=0x55dc87531680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc87531680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efe800bb4d0 tx=0x7efe800bff40 comp rx=0 tx=0).stop 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 -- 172.21.15.5:0/1177128325 shutdown_connections 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8751d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87529340 0x55dc87529770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 --2- 172.21.15.5:0/1177128325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87528990 0x55dc87531680 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 -- 172.21.15.5:0/1177128325 >> 172.21.15.5:0/1177128325 conn(0x55dc87522c40 msgr2=0x55dc8752e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 -- 172.21.15.5:0/1177128325 shutdown_connections 2024-04-22T13:27:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.700+0000 7efea05ea740 1 -- 172.21.15.5:0/1177128325 wait complete. 2024-04-22T13:27:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 -- start start 2024-04-22T13:27:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 Processor -- start 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc8752b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc8752b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8752bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc87544aa0 con 0x55dc87529340 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc875443d0 con 0x55dc87528990 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc87545170 con 0x55dc87531ec0 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efea05ea740 10 client.0 _ll_register_callbacks cb 0x7efe800be920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efe967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc8752b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efe967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc8752b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46994/0 (socket says 172.21.15.5:46994) 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efe967fc640 1 -- 172.21.15.5:0/4171368780 learned_addr learned my addr 172.21.15.5:0/4171368780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efe977fe640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8752bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.701+0000 7efe96ffd640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc8752b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe967fc640 1 -- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 msgr2=0x55dc8752bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe967fc640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8752bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe967fc640 1 -- 172.21.15.5:0/4171368780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 msgr2=0x55dc8752b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe967fc640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc8752b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe967fc640 1 -- 172.21.15.5:0/4171368780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8754c150 con 0x55dc87529340 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe96ffd640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc8752b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe967fc640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc8752b840 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efe88012040 tx=0x7efe88005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be853ed741b296e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe76ffd640 1 -- 172.21.15.5:0/4171368780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88020040 con 0x55dc87529340 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe76ffd640 1 -- 172.21.15.5:0/4171368780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7efe8801ba40 con 0x55dc87529340 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe800c77c0 con 0x55dc87529340 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe76ffd640 1 -- 172.21.15.5:0/4171368780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe8801bc40 con 0x55dc87529340 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efea05ea740 10 client.15732 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc8754d360 con 0x55dc87529340 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.702+0000 7efe76ffd640 1 -- 172.21.15.5:0/4171368780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe800c77c0 con 0x55dc87529340 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efe76ffd640 1 -- 172.21.15.5:0/4171368780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe880337a0 con 0x55dc87529340 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efea05ea740 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efea05ea740 10 client.15732 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efea05ea740 20 client.15732 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efea05ea740 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875ad9c0 0x55dc875ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efea05ea740 4 client.15732 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "status"}) -- 0x55dc874943a0 con 0x55dc875ad9c0 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efe96ffd640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875ad9c0 0x55dc875ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efe96ffd640 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875ad9c0 0x55dc875ac6d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9f64804c80e2c63 server_cookie=302c35d98c569215 in_seq=0 out_seq=0 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.703+0000 7efe76ffd640 10 client.15732 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efe76ffd640 1 -- 172.21.15.5:0/4171368780 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dc874943a0 con 0x55dc875ad9c0 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efe76ffd640 10 client.15732 handle_command_reply: tid=0 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 client.15732 shutdown 2024-04-22T13:27:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 msgr2=0x55dc8752b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc8752b840 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efe88012040 tx=0x7efe88005ee0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 shutdown_connections 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55dc875ad9c0 0x55dc875ac6d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc87529340 0x55dc8752b840 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc87531ec0 0x55dc8752bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 --2- 172.21.15.5:0/4171368780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc87528990 0x55dc8752b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 >> 172.21.15.5:0/4171368780 conn(0x55dc87522c40 msgr2=0x55dc87523020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efe76ffd640 0 client.15732 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 shutdown_connections 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 1 -- 172.21.15.5:0/4171368780 wait complete. 2024-04-22T13:27:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.704+0000 7efea05ea740 20 client.15732 trim_cache size 0 max 16384 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24469, 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 127.767097972, 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 92, 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 91, 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 152.95116783099999 2024-04-22T13:27:24.707 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efe840a3e40 msgr2=0x7efe840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efe840a3e40 0x7efe840a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe9810dfc0 tx=0x7efe88005d30 comp rx=0 tx=0).stop 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 msgr2=0x7efe981a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981a0460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efe80005940 tx=0x7efe8002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 shutdown_connections 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efe840a3e40 0x7efe840a6300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe9819a610 0x7efe9819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe98105fa0 0x7efe981a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 --2- 172.21.15.5:0/1238381199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe98100ed0 0x7efe981a0460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 >> 172.21.15.5:0/1238381199 conn(0x7efe98077a00 msgr2=0x7efe98069270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 shutdown_connections 2024-04-22T13:27:24.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.705+0000 7efe9da01640 1 -- 172.21.15.5:0/1238381199 wait complete. 2024-04-22T13:27:24.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 -- 172.21.15.5:0/508940882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f010c6d0 msgr2=0x7f88f010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/508940882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f010c6d0 0x7f88f010eb60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009ca0 tx=0x7f88e802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 -- 172.21.15.5:0/508940882 shutdown_connections 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/508940882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f010c6d0 0x7f88f010eb60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/508940882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88f0103b70 0x7f88f010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/508940882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f01031c0 0x7f88f01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 -- 172.21.15.5:0/508940882 >> 172.21.15.5:0/508940882 conn(0x7f88f0077f70 msgr2=0x7f88f0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 -- 172.21.15.5:0/508940882 shutdown_connections 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.824+0000 7f88f7cc8640 1 -- 172.21.15.5:0/508940882 wait complete. 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 Processor -- start 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 -- start start 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 0x7f88f0100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88f010c6d0 0x7f88f0101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f0101890 0x7f88f01a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01123a0 con 0x7f88f01031c0 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0112220 con 0x7f88f0101890 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f7cc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0112520 con 0x7f88f010c6d0 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f5a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 0x7f88f0100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f5a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 0x7f88f0100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47008/0 (socket says 172.21.15.5:47008) 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.825+0000 7f88f5a64640 1 -- 172.21.15.5:0/1166856457 learned_addr learned my addr 172.21.15.5:0/1166856457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f6265640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f0101890 0x7f88f01a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5263640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88f010c6d0 0x7f88f0101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5a64640 1 -- 172.21.15.5:0/1166856457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88f010c6d0 msgr2=0x7f88f0101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5a64640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88f010c6d0 0x7f88f0101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5a64640 1 -- 172.21.15.5:0/1166856457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f0101890 msgr2=0x7f88f01a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5a64640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f0101890 0x7f88f01a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5a64640 1 -- 172.21.15.5:0/1166856457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f01a7190 con 0x7f88f01031c0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f5a64640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 0x7f88f0100e10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d950 tx=0x7f88ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88e6ffd640 1 -- 172.21.15.5:0/1166856457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec017070 con 0x7f88f01031c0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88e6ffd640 1 -- 172.21.15.5:0/1166856457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ec009c30 con 0x7f88f01031c0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f01a7480 con 0x7f88f01031c0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88e6ffd640 1 -- 172.21.15.5:0/1166856457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00f620 con 0x7f88f01031c0 2024-04-22T13:27:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.826+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f01a79d0 con 0x7f88f01031c0 2024-04-22T13:27:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.828+0000 7f88e6ffd640 1 -- 172.21.15.5:0/1166856457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f88ec010460 con 0x7f88f01031c0 2024-04-22T13:27:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 Processor -- start 2024-04-22T13:27:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 -- start start 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554df4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df5160 0x563554dfd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 0x563554e00130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563554d58b10 con 0x563554dfdce0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563554cf6240 con 0x563554df47b0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88fa997740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563554d54ad0 con 0x563554df5160 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88e6ffd640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88dc0a3e70 0x7f88dc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 0x563554e00130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 0x563554e00130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47010/0 (socket says 172.21.15.5:47010) 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88f5263640 1 -- 172.21.15.5:0/888034097 learned_addr learned my addr 172.21.15.5:0/888034097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88f6265640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88dc0a3e70 0x7f88dc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88f5a64640 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df5160 0x563554dfd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.829+0000 7f88f6265640 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554df4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88e6ffd640 1 -- 172.21.15.5:0/1166856457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f88ec0d0690 con 0x7f88f01031c0 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f5263640 1 -- 172.21.15.5:0/888034097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df5160 msgr2=0x563554dfd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f5263640 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df5160 0x563554dfd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f5263640 1 -- 172.21.15.5:0/888034097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 msgr2=0x563554df4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f5263640 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554df4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f5263640 1 -- 172.21.15.5:0/888034097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563554a6b2d0 con 0x563554dfdce0 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f6265640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88dc0a3e70 0x7f88dc0a6330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f88e8004af0 tx=0x7f88e8009960 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88f5263640 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 0x563554e00130 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009b30 tx=0x7f88e00113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12251f473dbcf56f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88dbfff640 1 -- 172.21.15.5:0/888034097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0011e10 con 0x563554dfdce0 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88dbfff640 1 -- 172.21.15.5:0/888034097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e00054d0 con 0x563554dfdce0 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.830+0000 7f88dbfff640 1 -- 172.21.15.5:0/888034097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00124b0 con 0x563554dfdce0 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 -- 172.21.15.5:0/888034097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 msgr2=0x563554e00130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 0x563554e00130 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009b30 tx=0x7f88e00113e0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 -- 172.21.15.5:0/888034097 shutdown_connections 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554dfdce0 0x563554e00130 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df5160 0x563554dfd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 --2- 172.21.15.5:0/888034097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554df4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 -- 172.21.15.5:0/888034097 >> 172.21.15.5:0/888034097 conn(0x563554deea60 msgr2=0x563554deee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 -- 172.21.15.5:0/888034097 shutdown_connections 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 -- 172.21.15.5:0/888034097 wait complete. 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.831+0000 7f88fa997740 1 -- start start 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 Processor -- start 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554dfabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554dfdce0 0x563554dfb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563554e00c50 con 0x563554df5160 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563554ce1190 con 0x563554df47b0 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563554e00dd0 con 0x563554dfdce0 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88fa997740 10 client.0 _ll_register_callbacks cb 0x563554cdc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47016/0 (socket says 172.21.15.5:47016) 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 -- 172.21.15.5:0/3591026134 learned_addr learned my addr 172.21.15.5:0/3591026134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f6265640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554dfdce0 0x563554dfb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5a64640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554dfabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 -- 172.21.15.5:0/3591026134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554dfdce0 msgr2=0x563554dfb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554dfdce0 0x563554dfb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 -- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 msgr2=0x563554dfabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554dfabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.832+0000 7f88f5263640 1 -- 172.21.15.5:0/3591026134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563554b356b0 con 0x563554df5160 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88f5263640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f88e0004b90 tx=0x7f88e0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94cd7d1c2aa5880 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88d97fa640 1 -- 172.21.15.5:0/3591026134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e001f040 con 0x563554df5160 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88d97fa640 1 -- 172.21.15.5:0/3591026134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f88e0004f70 con 0x563554df5160 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e00054d0 con 0x563554df5160 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88d97fa640 1 -- 172.21.15.5:0/3591026134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88e0032050 con 0x563554df5160 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563554e0ce50 con 0x563554df5160 2024-04-22T13:27:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88d97fa640 1 -- 172.21.15.5:0/3591026134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e00054d0 con 0x563554df5160 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88d97fa640 1 -- 172.21.15.5:0/3591026134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f88e0012ca0 con 0x563554df5160 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 10 client.15744 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 20 client.15744 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e0f2e0 0x563554e10750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 4 client.15744 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.833+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563554d66530 con 0x563554e0f2e0 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.834+0000 7f88f5a64640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e0f2e0 0x563554e10750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.834+0000 7f88f5a64640 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e0f2e0 0x563554e10750 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3011419d399102fc server_cookie=79920b63e00d3bda in_seq=0 out_seq=0 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.834+0000 7f88d97fa640 10 client.15744 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88d97fa640 1 -- 172.21.15.5:0/3591026134 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563554d66530 con 0x563554e0f2e0 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88d97fa640 10 client.15744 handle_command_reply: tid=0 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 client.15744 shutdown 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 msgr2=0x563554dfb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f88e0004b90 tx=0x7f88e0004970 comp rx=0 tx=0).stop 2024-04-22T13:27:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 shutdown_connections 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e0f2e0 0x563554e10750 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb130 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88d97fa640 0 client.15744 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554dfdce0 0x563554dfb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 --2- 172.21.15.5:0/3591026134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554df47b0 0x563554dfabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 >> 172.21.15.5:0/3591026134 conn(0x563554deea60 msgr2=0x563554df0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 shutdown_connections 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 1 -- 172.21.15.5:0/3591026134 wait complete. 2024-04-22T13:27:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.835+0000 7f88fa997740 20 client.15744 trim_cache size 0 max 16384 2024-04-22T13:27:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 Processor -- start 2024-04-22T13:27:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 -- start start 2024-04-22T13:27:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554dfd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554df7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563554e0ea90 con 0x563554df5160 2024-04-22T13:27:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563554e0f160 con 0x563554dfdce0 2024-04-22T13:27:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88fa997740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563554e0f830 con 0x563554df47b0 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47018/0 (socket says 172.21.15.5:47018) 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 -- 172.21.15.5:0/776973337 learned_addr learned my addr 172.21.15.5:0/776973337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5a64640 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554dfd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f6265640 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554df7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 -- 172.21.15.5:0/776973337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 msgr2=0x563554dfd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554dfd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 -- 172.21.15.5:0/776973337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 msgr2=0x563554df7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554df7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5263640 1 -- 172.21.15.5:0/776973337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563554df4f00 con 0x563554df5160 2024-04-22T13:27:24.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.854+0000 7f88f5a64640 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554dfd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88f5263640 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb490 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f88e0024600 tx=0x7f88e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f5038921a42797 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88db7fe640 1 -- 172.21.15.5:0/776973337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00049e0 con 0x563554df5160 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88db7fe640 1 -- 172.21.15.5:0/776973337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e0029070 con 0x563554df5160 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88db7fe640 1 -- 172.21.15.5:0/776973337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0004280 con 0x563554df5160 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 -- 172.21.15.5:0/776973337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 msgr2=0x563554dfb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb490 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f88e0024600 tx=0x7f88e0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 -- 172.21.15.5:0/776973337 shutdown_connections 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554dfb490 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554df7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 --2- 172.21.15.5:0/776973337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554dfd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 -- 172.21.15.5:0/776973337 >> 172.21.15.5:0/776973337 conn(0x563554deea60 msgr2=0x563554de9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 -- 172.21.15.5:0/776973337 shutdown_connections 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.855+0000 7f88fa997740 1 -- 172.21.15.5:0/776973337 wait complete. 2024-04-22T13:27:24.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 -- start start 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 Processor -- start 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554e12810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554e12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554e16fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563554e105d0 con 0x563554df5160 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563554e0ff00 con 0x563554dfdce0 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563554e10ca0 con 0x563554df47b0 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88fa997740 10 client.0 _ll_register_callbacks cb 0x7f88e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554e12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88f5263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554e12d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47032/0 (socket says 172.21.15.5:47032) 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88f5263640 1 -- 172.21.15.5:0/1852867201 learned_addr learned my addr 172.21.15.5:0/1852867201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88f5a64640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554e12810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.856+0000 7f88f6265640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554e16fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f5263640 1 -- 172.21.15.5:0/1852867201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 msgr2=0x563554e12810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f5263640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554e12810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f5263640 1 -- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 msgr2=0x563554e16fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f5263640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554e16fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f5263640 1 -- 172.21.15.5:0/1852867201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563554e17510 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f6265640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554e16fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88f5263640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554e12d50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f88e0005d80 tx=0x7f88e0024770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6c1e0faaa663a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88d97fa640 1 -- 172.21.15.5:0/1852867201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0044060 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88d97fa640 1 -- 172.21.15.5:0/1852867201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f88e0024ce0 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e0029070 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88d97fa640 1 -- 172.21.15.5:0/1852867201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88e0009980 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88fa997740 10 client.15750 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563554e759f0 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.857+0000 7f88d97fa640 1 -- 172.21.15.5:0/1852867201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e0029070 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88d97fa640 1 -- 172.21.15.5:0/1852867201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f88e003f590 con 0x563554df5160 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88fa997740 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:24.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88fa997740 10 client.15750 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88fa997740 20 client.15750 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88fa997740 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e7a160 0x563554e78e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88fa997740 4 client.15750 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "status"}) -- 0x563554d603a0 con 0x563554e7a160 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.858+0000 7f88f5a64640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e7a160 0x563554e78e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88f5a64640 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e7a160 0x563554e78e70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b9628bd50432a6a server_cookie=1e9603659b1a8e83 in_seq=0 out_seq=0 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88d97fa640 10 client.15750 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88d97fa640 1 -- 172.21.15.5:0/1852867201 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563554d603a0 con 0x563554e7a160 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88d97fa640 10 client.15750 handle_command_reply: tid=0 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 client.15750 shutdown 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 msgr2=0x563554e12d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554e12d50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f88e0005d80 tx=0x7f88e0024770 comp rx=0 tx=0).stop 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 shutdown_connections 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x563554e7a160 0x563554e78e70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563554df5160 0x563554e12d50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563554dfdce0 0x563554e16fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88d97fa640 0 client.15750 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 --2- 172.21.15.5:0/1852867201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563554df47b0 0x563554e12810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 >> 172.21.15.5:0/1852867201 conn(0x563554deea60 msgr2=0x563554ded170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 shutdown_connections 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 1 -- 172.21.15.5:0/1852867201 wait complete. 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.859+0000 7f88fa997740 20 client.15750 trim_cache size 0 max 16384 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:24.862 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24485, 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 128.93225054300001, 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 92, 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 91, 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 158.43992144500001 2024-04-22T13:27:24.863 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88dc0a3e70 msgr2=0x7f88dc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88dc0a3e70 0x7f88dc0a6330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f88e8004af0 tx=0x7f88e8009960 comp rx=0 tx=0).stop 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 msgr2=0x7f88f0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 0x7f88f0100e10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d950 tx=0x7f88ec00de10 comp rx=0 tx=0).stop 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 shutdown_connections 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88dc0a3e70 0x7f88dc0a6330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88f0101890 0x7f88f01a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88f010c6d0 0x7f88f0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 --2- 172.21.15.5:0/1166856457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88f01031c0 0x7f88f0100e10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 >> 172.21.15.5:0/1166856457 conn(0x7f88f0077f70 msgr2=0x7f88f00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 shutdown_connections 2024-04-22T13:27:24.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.860+0000 7f88f7cc8640 1 -- 172.21.15.5:0/1166856457 wait complete. 2024-04-22T13:27:24.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T13:27:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 -- 172.21.15.5:0/4090370805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 msgr2=0x7f0f8010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 --2- 172.21.15.5:0/4090370805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f8010eef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009cd0 tx=0x7f0f7802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 -- 172.21.15.5:0/4090370805 shutdown_connections 2024-04-22T13:27:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 --2- 172.21.15.5:0/4090370805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f8010eef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 --2- 172.21.15.5:0/4090370805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80103b50 0x7f0f80103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 --2- 172.21.15.5:0/4090370805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 0x7f0f80102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 -- 172.21.15.5:0/4090370805 >> 172.21.15.5:0/4090370805 conn(0x7f0f800fda10 msgr2=0x7f0f800ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 -- 172.21.15.5:0/4090370805 shutdown_connections 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.974+0000 7f0f88206640 1 -- 172.21.15.5:0/4090370805 wait complete. 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 Processor -- start 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 -- start start 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 0x7f0f8019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80103b50 0x7f0f8019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f801985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80112290 con 0x7f0f801044f0 2024-04-22T13:27:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80112110 con 0x7f0f80102270 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f88206640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80112410 con 0x7f0f80103b50 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f801985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f801985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47058/0 (socket says 172.21.15.5:47058) 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 -- 172.21.15.5:0/1546070496 learned_addr learned my addr 172.21.15.5:0/1546070496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80103b50 0x7f0f8019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 0x7f0f8019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 -- 172.21.15.5:0/1546070496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80103b50 msgr2=0x7f0f8019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80103b50 0x7f0f8019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 -- 172.21.15.5:0/1546070496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 msgr2=0x7f0f8019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 0x7f0f8019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f867a3640 1 -- 172.21.15.5:0/1546070496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f80198d70 con 0x7f0f801044f0 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.975+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 0x7f0f8019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.976+0000 7f0f867a3640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f801985f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0f780098d0 tx=0x7f0f78032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.976+0000 7f0f76ffd640 1 -- 172.21.15.5:0/1546070496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78041020 con 0x7f0f801044f0 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.976+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f80199000 con 0x7f0f801044f0 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.976+0000 7f0f76ffd640 1 -- 172.21.15.5:0/1546070496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f78032970 con 0x7f0f801044f0 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.976+0000 7f0f76ffd640 1 -- 172.21.15.5:0/1546070496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78039040 con 0x7f0f801044f0 2024-04-22T13:27:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.976+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f801a52b0 con 0x7f0f801044f0 2024-04-22T13:27:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.978+0000 7f0f76ffd640 1 -- 172.21.15.5:0/1546070496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f0f78044020 con 0x7f0f801044f0 2024-04-22T13:27:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.978+0000 7f0f8adf9740 1 Processor -- start 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 -- start start 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 0x562517e3cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e45710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e45ce0 0x562517e48130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562517da0b10 con 0x562517e3c7b0 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562517d3e240 con 0x562517e3d160 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562517d9cad0 con 0x562517e45ce0 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 0x562517e3cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 0x562517e3cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47066/0 (socket says 172.21.15.5:47066) 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 -- 172.21.15.5:0/239221755 learned_addr learned my addr 172.21.15.5:0/239221755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f76ffd640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f6c0a3e40 0x7f0f6c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e45710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 -- 172.21.15.5:0/239221755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e45ce0 msgr2=0x562517e48130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f867a3640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f6c0a3e40 0x7f0f6c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e45ce0 0x562517e48130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 -- 172.21.15.5:0/239221755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 msgr2=0x562517e45710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e45710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 -- 172.21.15.5:0/239221755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562517ab32d0 con 0x562517e3c7b0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e45710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f76ffd640 1 -- 172.21.15.5:0/1546070496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0f780f2700 con 0x7f0f801044f0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.979+0000 7f0f857a1640 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 0x562517e3cb90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009d10 tx=0x7f0f70013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eafcb80b40f55c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f6bfff640 1 -- 172.21.15.5:0/239221755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70004050 con 0x562517e3c7b0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f867a3640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f6c0a3e40 0x7f0f6c0a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f780efe70 tx=0x7f0f780a9040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f6bfff640 1 -- 172.21.15.5:0/239221755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f70017030 con 0x562517e3c7b0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f6bfff640 1 -- 172.21.15.5:0/239221755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70010430 con 0x562517e3c7b0 2024-04-22T13:27:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 -- 172.21.15.5:0/239221755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 msgr2=0x562517e3cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 0x562517e3cb90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009d10 tx=0x7f0f70013830 comp rx=0 tx=0).stop 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 -- 172.21.15.5:0/239221755 shutdown_connections 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e45ce0 0x562517e48130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e45710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/239221755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3c7b0 0x562517e3cb90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 -- 172.21.15.5:0/239221755 >> 172.21.15.5:0/239221755 conn(0x562517e36a60 msgr2=0x562517e36e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.980+0000 7f0f8adf9740 1 -- 172.21.15.5:0/239221755 shutdown_connections 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 -- 172.21.15.5:0/239221755 wait complete. 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 -- start start 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 Processor -- start 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e43750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562517e48c50 con 0x562517e45ce0 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562517d29190 con 0x562517e3d160 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562517e48dd0 con 0x562517e3c7b0 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f8adf9740 10 client.0 _ll_register_callbacks cb 0x562517d248b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f857a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e43750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f857a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e43750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47074/0 (socket says 172.21.15.5:47074) 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f857a1640 1 -- 172.21.15.5:0/3140455187 learned_addr learned my addr 172.21.15.5:0/3140455187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.981+0000 7f0f857a1640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f857a1640 1 -- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 msgr2=0x562517e42cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f857a1640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e42cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f857a1640 1 -- 172.21.15.5:0/3140455187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 msgr2=0x562517e43210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f857a1640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e43210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f857a1640 1 -- 172.21.15.5:0/3140455187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562517b7d6b0 con 0x562517e45ce0 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f857a1640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e43750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0f70003dd0 tx=0x7f0f70003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5959e561d45ed694 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f697fa640 1 -- 172.21.15.5:0/3140455187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70003ef0 con 0x562517e45ce0 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f697fa640 1 -- 172.21.15.5:0/3140455187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0f70004090 con 0x562517e45ce0 2024-04-22T13:27:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f70017030 con 0x562517e45ce0 2024-04-22T13:27:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f697fa640 1 -- 172.21.15.5:0/3140455187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f70004290 con 0x562517e45ce0 2024-04-22T13:27:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f8adf9740 10 client.15762 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562517e54f10 con 0x562517e45ce0 2024-04-22T13:27:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.982+0000 7f0f697fa640 1 -- 172.21.15.5:0/3140455187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f70017030 con 0x562517e45ce0 2024-04-22T13:27:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f697fa640 1 -- 172.21.15.5:0/3140455187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f7002d4e0 con 0x562517e45ce0 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f8adf9740 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f8adf9740 10 client.15762 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f8adf9740 20 client.15762 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517e57180 0x562517e585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f8adf9740 4 client.15762 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562517dae530 con 0x562517e57180 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517e57180 0x562517e585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517e57180 0x562517e585f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a0bd03032f472e5 server_cookie=79a0f6e086de0366 in_seq=0 out_seq=0 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.983+0000 7f0f697fa640 10 client.15762 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.984+0000 7f0f697fa640 1 -- 172.21.15.5:0/3140455187 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562517dae530 con 0x562517e57180 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.984+0000 7f0f697fa640 10 client.15762 handle_command_reply: tid=0 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.984+0000 7f0f8adf9740 1 client.15762 shutdown 2024-04-22T13:27:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.984+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 msgr2=0x562517e43750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.984+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e43750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0f70003dd0 tx=0x7f0f70003b40 comp rx=0 tx=0).stop 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.984+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 shutdown_connections 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517e57180 0x562517e585f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e43750 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e43210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/3140455187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f697fa640 0 client.15762 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 >> 172.21.15.5:0/3140455187 conn(0x562517e36a60 msgr2=0x562517e38bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 shutdown_connections 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 1 -- 172.21.15.5:0/3140455187 wait complete. 2024-04-22T13:27:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:24.985+0000 7f0f8adf9740 20 client.15762 trim_cache size 0 max 16384 2024-04-22T13:27:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 Processor -- start 2024-04-22T13:27:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 -- start start 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 0x562517e36e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e45ce0 0x562517e31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562517e56930 con 0x562517e3d160 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562517e57000 con 0x562517e45ce0 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562517e576d0 con 0x562517e3c7b0 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 0x562517e36e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 0x562517e36e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47082/0 (socket says 172.21.15.5:47082) 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 -- 172.21.15.5:0/1861943301 learned_addr learned my addr 172.21.15.5:0/1861943301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e45ce0 0x562517e31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 -- 172.21.15.5:0/1861943301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 msgr2=0x562517e45400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e45400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 -- 172.21.15.5:0/1861943301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e45ce0 msgr2=0x562517e31800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e45ce0 0x562517e31800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 -- 172.21.15.5:0/1861943301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562517e37630 con 0x562517e3d160 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.003+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 0x562517e36e40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x562517e3c280 tx=0x7f0f7c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ecebcd7d26fd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f6b7fe640 1 -- 172.21.15.5:0/1861943301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00ed20 con 0x562517e3d160 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f6b7fe640 1 -- 172.21.15.5:0/1861943301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c01c8e0 con 0x562517e3d160 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f6b7fe640 1 -- 172.21.15.5:0/1861943301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00fd50 con 0x562517e3d160 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1861943301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 msgr2=0x562517e36e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 0x562517e36e40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x562517e3c280 tx=0x7f0f7c00cf10 comp rx=0 tx=0).stop 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1861943301 shutdown_connections 2024-04-22T13:27:25.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e3d160 0x562517e36e40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e45ce0 0x562517e31800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1861943301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e45400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1861943301 >> 172.21.15.5:0/1861943301 conn(0x562517e36a60 msgr2=0x562517e3d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1861943301 shutdown_connections 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.004+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1861943301 wait complete. 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 -- start start 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 Processor -- start 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e40030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562517e58470 con 0x562517e45ce0 2024-04-22T13:27:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562517e57da0 con 0x562517e3d160 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562517e58b40 con 0x562517e3c7b0 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f8adf9740 10 client.0 _ll_register_callbacks cb 0x7f0f70009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f857a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f857a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e40570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47088/0 (socket says 172.21.15.5:47088) 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f857a1640 1 -- 172.21.15.5:0/1227826950 learned_addr learned my addr 172.21.15.5:0/1227826950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e40030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.005+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f857a1640 1 -- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 msgr2=0x562517e3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e3faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f857a1640 1 -- 172.21.15.5:0/1227826950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 msgr2=0x562517e40030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e40030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f857a1640 1 -- 172.21.15.5:0/1227826950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562517e5ea40 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f857a1640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e40570 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009250 tx=0x7f0f70003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76428cef3d5d0d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f697fa640 1 -- 172.21.15.5:0/1227826950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7003a030 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f697fa640 1 -- 172.21.15.5:0/1227826950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0f70009780 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7c01c8e0 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f697fa640 1 -- 172.21.15.5:0/1227826950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f70047050 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f8adf9740 10 client.15768 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562517ebd340 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f697fa640 1 -- 172.21.15.5:0/1227826950 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c01c8e0 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f697fa640 1 -- 172.21.15.5:0/1227826950 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f7002d530 con 0x562517e45ce0 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f8adf9740 10 client.15768 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f8adf9740 10 client.15768 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.006+0000 7f0f8adf9740 20 client.15768 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.007+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517ec1e10 0x562517ec0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.007+0000 7f0f8adf9740 4 client.15768 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.007+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "status"}) -- 0x562517da83a0 con 0x562517ec1e10 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.007+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517ec1e10 0x562517ec0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.007+0000 7f0f697fa640 10 client.15768 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.007+0000 7f0f85fa2640 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517ec1e10 0x562517ec0b20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf42fb863015abd2 server_cookie=b77e0bf00a8e0cc8 in_seq=0 out_seq=0 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f697fa640 1 -- 172.21.15.5:0/1227826950 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562517da83a0 con 0x562517ec1e10 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f697fa640 10 client.15768 handle_command_reply: tid=0 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 client.15768 shutdown 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 msgr2=0x562517e40570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e40570 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009250 tx=0x7f0f70003b40 comp rx=0 tx=0).stop 2024-04-22T13:27:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 shutdown_connections 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562517ec1e10 0x562517ec0b20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562517e45ce0 0x562517e40570 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562517e3d160 0x562517e40030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f697fa640 0 client.15768 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 --2- 172.21.15.5:0/1227826950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562517e3c7b0 0x562517e3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 >> 172.21.15.5:0/1227826950 conn(0x562517e36a60 msgr2=0x562517e36e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 shutdown_connections 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 1 -- 172.21.15.5:0/1227826950 wait complete. 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f8adf9740 20 client.15768 trim_cache size 0 max 16384 2024-04-22T13:27:25.011 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24496, 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 130.08786478100001, 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 92, 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 91, 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 147.78492276200001 2024-04-22T13:27:25.012 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:25.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f6c0a3e40 msgr2=0x7f0f6c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f6c0a3e40 0x7f0f6c0a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f780efe70 tx=0x7f0f780a9040 comp rx=0 tx=0).stop 2024-04-22T13:27:25.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 msgr2=0x7f0f801985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f801985f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0f780098d0 tx=0x7f0f78032420 comp rx=0 tx=0).stop 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 shutdown_connections 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801044f0 0x7f0f801985f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f6c0a3e40 0x7f0f6c0a6300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80103b50 0x7f0f8019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.008+0000 7f0f88206640 1 --2- 172.21.15.5:0/1546070496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102270 0x7f0f8019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.009+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 >> 172.21.15.5:0/1546070496 conn(0x7f0f800fda10 msgr2=0x7f0f800fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.009+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 shutdown_connections 2024-04-22T13:27:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.009+0000 7f0f88206640 1 -- 172.21.15.5:0/1546070496 wait complete. 2024-04-22T13:27:25.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-22T13:27:25.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 -- 172.21.15.5:0/1388361284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988103400 msgr2=0x7f898810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 --2- 172.21.15.5:0/1388361284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988103400 0x7f898810de00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8980009cd0 tx=0x7f898002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 -- 172.21.15.5:0/1388361284 shutdown_connections 2024-04-22T13:27:25.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 --2- 172.21.15.5:0/1388361284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988103400 0x7f898810de00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 --2- 172.21.15.5:0/1388361284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988102a60 0x7f8988102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 --2- 172.21.15.5:0/1388361284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8988101180 0x7f8988101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 -- 172.21.15.5:0/1388361284 >> 172.21.15.5:0/1388361284 conn(0x7f8988077f70 msgr2=0x7f8988078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 -- 172.21.15.5:0/1388361284 shutdown_connections 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.123+0000 7f8990093640 1 -- 172.21.15.5:0/1388361284 wait complete. 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 Processor -- start 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 -- start start 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8988101180 0x7f898819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 0x7f898819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988103400 0x7f8988198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8988111200 con 0x7f8988102a60 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8988111080 con 0x7f8988103400 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f8990093640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8988111380 con 0x7f8988101180 2024-04-22T13:27:25.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 0x7f898819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 0x7f898819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47106/0 (socket says 172.21.15.5:47106) 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 -- 172.21.15.5:0/114061583 learned_addr learned my addr 172.21.15.5:0/114061583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898e630640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988103400 0x7f8988198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898de2f640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8988101180 0x7f898819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 -- 172.21.15.5:0/114061583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8988101180 msgr2=0x7f898819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8988101180 0x7f898819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 -- 172.21.15.5:0/114061583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988103400 msgr2=0x7f8988198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988103400 0x7f8988198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 -- 172.21.15.5:0/114061583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8988198f20 con 0x7f8988102a60 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898e630640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988103400 0x7f8988198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.124+0000 7f898d62e640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 0x7f898819eae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f897800b4a0 tx=0x7f897800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.125+0000 7f897effd640 1 -- 172.21.15.5:0/114061583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978014070 con 0x7f8988102a60 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.125+0000 7f897effd640 1 -- 172.21.15.5:0/114061583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8978010ac0 con 0x7f8988102a60 2024-04-22T13:27:25.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.125+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8988199210 con 0x7f8988102a60 2024-04-22T13:27:25.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.125+0000 7f897effd640 1 -- 172.21.15.5:0/114061583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978017c30 con 0x7f8988102a60 2024-04-22T13:27:25.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.125+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89881a5520 con 0x7f8988102a60 2024-04-22T13:27:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.127+0000 7f897effd640 1 -- 172.21.15.5:0/114061583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f8978017070 con 0x7f8988102a60 2024-04-22T13:27:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.127+0000 7f8992d67740 1 Processor -- start 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.127+0000 7f8992d67740 1 -- start start 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8992d67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb07b0 0x561c1fcb0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f897effd640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f89740a3ec0 0x7f89740a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8992d67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fcb9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8992d67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb9ce0 0x561c1fcbc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8992d67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c1fc14b10 con 0x561c1fcb1160 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8992d67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c1fbb2240 con 0x561c1fcb07b0 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8992d67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c1fc10ad0 con 0x561c1fcb9ce0 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fcb9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fcb9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47118/0 (socket says 172.21.15.5:47118) 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898de2f640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f89740a3ec0 0x7f89740a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 -- 172.21.15.5:0/2679745888 learned_addr learned my addr 172.21.15.5:0/2679745888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898e630640 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb9ce0 0x561c1fcbc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f897effd640 1 -- 172.21.15.5:0/114061583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f89780cff40 con 0x7f8988102a60 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898e630640 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb07b0 0x561c1fcb0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 -- 172.21.15.5:0/2679745888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb9ce0 msgr2=0x561c1fcbc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb9ce0 0x561c1fcbc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 -- 172.21.15.5:0/2679745888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb07b0 msgr2=0x561c1fcb0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb07b0 0x561c1fcb0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 -- 172.21.15.5:0/2679745888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1f9272d0 con 0x561c1fcb1160 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898e630640 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb07b0 0x561c1fcb0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898d62e640 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fcb9710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f897809bb80 tx=0x7f897800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e625fba1f6d82d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f898de2f640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f89740a3ec0 0x7f89740a6380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f898400bbe0 tx=0x7f898400b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8973fff640 1 -- 172.21.15.5:0/2679745888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897809f070 con 0x561c1fcb1160 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.128+0000 7f8973fff640 1 -- 172.21.15.5:0/2679745888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8978017360 con 0x561c1fcb1160 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8973fff640 1 -- 172.21.15.5:0/2679745888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897801e040 con 0x561c1fcb1160 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 -- 172.21.15.5:0/2679745888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 msgr2=0x561c1fcb9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fcb9710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f897809bb80 tx=0x7f897800bf90 comp rx=0 tx=0).stop 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 -- 172.21.15.5:0/2679745888 shutdown_connections 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fcb9710 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb9ce0 0x561c1fcbc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 --2- 172.21.15.5:0/2679745888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb07b0 0x561c1fcb0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 -- 172.21.15.5:0/2679745888 >> 172.21.15.5:0/2679745888 conn(0x561c1fcaaa60 msgr2=0x561c1fcaae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 -- 172.21.15.5:0/2679745888 shutdown_connections 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.129+0000 7f8992d67740 1 -- 172.21.15.5:0/2679745888 wait complete. 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 -- start start 2024-04-22T13:27:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 Processor -- start 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 0x561c1fcb6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb1160 0x561c1fcb71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c1fcbcbf0 con 0x561c1fcb07b0 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c1fb9d190 con 0x561c1fcb9ce0 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c1fcbcd70 con 0x561c1fcb1160 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f8992d67740 10 client.0 _ll_register_callbacks cb 0x561c1fb988b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f898de2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 0x561c1fcb6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f898de2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 0x561c1fcb6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47120/0 (socket says 172.21.15.5:47120) 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f898de2f640 1 -- 172.21.15.5:0/2401055349 learned_addr learned my addr 172.21.15.5:0/2401055349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.130+0000 7f898e630640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898d62e640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb1160 0x561c1fcb71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898de2f640 1 -- 172.21.15.5:0/2401055349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb1160 msgr2=0x561c1fcb71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898de2f640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb1160 0x561c1fcb71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898de2f640 1 -- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 msgr2=0x561c1fcb7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898de2f640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898de2f640 1 -- 172.21.15.5:0/2401055349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1f9f16b0 con 0x561c1fcb07b0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f898de2f640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 0x561c1fcb6cb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f89840058c0 tx=0x7f8984004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876c83b3acc5c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f89717fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f89717fa640 1 -- 172.21.15.5:0/2401055349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984005220 con 0x561c1fcb07b0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f89717fa640 1 -- 172.21.15.5:0/2401055349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f89840053c0 con 0x561c1fcb07b0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c1fc22530 con 0x561c1fcb07b0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f89717fa640 1 -- 172.21.15.5:0/2401055349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89840055c0 con 0x561c1fcb07b0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f8992d67740 10 client.15780 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c1fcc8ec0 con 0x561c1fcb07b0 2024-04-22T13:27:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.131+0000 7f89717fa640 1 -- 172.21.15.5:0/2401055349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561c1fc22530 con 0x561c1fcb07b0 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f89717fa640 1 -- 172.21.15.5:0/2401055349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f898401ebe0 con 0x561c1fcb07b0 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f8992d67740 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f8992d67740 10 client.15780 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f8992d67740 20 client.15780 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f8992d67740 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fccb510 0x561c1fccc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f8992d67740 4 client.15780 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1fc1c3a0 con 0x561c1fccb510 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f898d62e640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fccb510 0x561c1fccc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f89717fa640 10 client.15780 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.132+0000 7f898d62e640 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fccb510 0x561c1fccc980 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c684af4f482a08 server_cookie=33a839fa044c688d in_seq=0 out_seq=0 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f89717fa640 1 -- 172.21.15.5:0/2401055349 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c1fc1c3a0 con 0x561c1fccb510 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f89717fa640 10 client.15780 handle_command_reply: tid=0 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 client.15780 shutdown 2024-04-22T13:27:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 msgr2=0x561c1fcb6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 0x561c1fcb6cb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f89840058c0 tx=0x7f8984004730 comp rx=0 tx=0).stop 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 shutdown_connections 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fccb510 0x561c1fccc980 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f89717fa640 0 client.15780 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb1160 0x561c1fcb71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 --2- 172.21.15.5:0/2401055349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb07b0 0x561c1fcb6cb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 >> 172.21.15.5:0/2401055349 conn(0x561c1fcaaa60 msgr2=0x561c1fcacbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.133+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 shutdown_connections 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.134+0000 7f8992d67740 1 -- 172.21.15.5:0/2401055349 wait complete. 2024-04-22T13:27:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.134+0000 7f8992d67740 20 client.15780 trim_cache size 0 max 16384 2024-04-22T13:27:25.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 Processor -- start 2024-04-22T13:27:25.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 -- start start 2024-04-22T13:27:25.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fccc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c1fccacc0 con 0x561c1fcb1160 2024-04-22T13:27:25.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c1fccb390 con 0x561c1fcb9ce0 2024-04-22T13:27:25.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f8992d67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c1fccba60 con 0x561c1fcb07b0 2024-04-22T13:27:25.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f898d62e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fccc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f898d62e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fccc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47134/0 (socket says 172.21.15.5:47134) 2024-04-22T13:27:25.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.152+0000 7f898d62e640 1 -- 172.21.15.5:0/1072813688 learned_addr learned my addr 172.21.15.5:0/1072813688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898d62e640 1 -- 172.21.15.5:0/1072813688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 msgr2=0x561c1fcb93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898de2f640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898e630640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898d62e640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcb93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898d62e640 1 -- 172.21.15.5:0/1072813688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 msgr2=0x561c1fcb7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898d62e640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898d62e640 1 -- 172.21.15.5:0/1072813688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1fcb5200 con 0x561c1fcb1160 2024-04-22T13:27:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898de2f640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcb93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898e630640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f898d62e640 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fccc2b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f897809b490 tx=0x7f89780baf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765d93c099a9f8cd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f89737fe640 1 -- 172.21.15.5:0/1072813688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897809f040 con 0x561c1fcb1160 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f89737fe640 1 -- 172.21.15.5:0/1072813688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8978085170 con 0x561c1fcb1160 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.153+0000 7f89737fe640 1 -- 172.21.15.5:0/1072813688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897801e040 con 0x561c1fcb1160 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 -- 172.21.15.5:0/1072813688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 msgr2=0x561c1fccc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fccc2b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f897809b490 tx=0x7f89780baf90 comp rx=0 tx=0).stop 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 -- 172.21.15.5:0/1072813688 shutdown_connections 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb1160 0x561c1fccc2b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb9ce0 0x561c1fcb7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 --2- 172.21.15.5:0/1072813688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcb93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 -- 172.21.15.5:0/1072813688 >> 172.21.15.5:0/1072813688 conn(0x561c1fcaaa60 msgr2=0x561c1fcaae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 -- 172.21.15.5:0/1072813688 shutdown_connections 2024-04-22T13:27:25.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 -- 172.21.15.5:0/1072813688 wait complete. 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 -- start start 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 Processor -- start 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcd49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.154+0000 7f8992d67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 0x561c1fcd4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f8992d67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb9ce0 0x561c1fccde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f8992d67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c1fccc800 con 0x561c1fcb9ce0 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f8992d67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c1fccc130 con 0x561c1fcb1160 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f8992d67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c1fccced0 con 0x561c1fcb07b0 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f8992d67740 10 client.0 _ll_register_callbacks cb 0x7f8984005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898e630640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb9ce0 0x561c1fccde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 0x561c1fcd4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 0x561c1fcd4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:39954/0 (socket says 172.21.15.5:39954) 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 -- 172.21.15.5:0/91057644 learned_addr learned my addr 172.21.15.5:0/91057644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898de2f640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcd49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 -- 172.21.15.5:0/91057644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 msgr2=0x561c1fcd49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcd49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 -- 172.21.15.5:0/91057644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb9ce0 msgr2=0x561c1fccde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb9ce0 0x561c1fccde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 -- 172.21.15.5:0/91057644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1fcce5d0 con 0x561c1fcb1160 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898e630640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb9ce0 0x561c1fccde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f898d62e640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 0x561c1fcd4f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89780982e0 tx=0x7f89780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d6c4d5b4ce584f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f89717fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:27:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f89717fa640 1 -- 172.21.15.5:0/91057644 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897809f070 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f89717fa640 1 -- 172.21.15.5:0/91057644 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8978099f20 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.155+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8978085170 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f89717fa640 1 -- 172.21.15.5:0/91057644 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89780b5240 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 10 client.25394 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c1fcd2e90 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f89717fa640 1 -- 172.21.15.5:0/91057644 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8978085170 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f89717fa640 1 -- 172.21.15.5:0/91057644 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8978097db0 con 0x561c1fcb1160 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 10 client.25394 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 20 client.25394 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fd352f0 0x561c1fd34000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 4 client.25394 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "status"}) -- 0x561c1fb98960 con 0x561c1fd352f0 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.156+0000 7f898de2f640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fd352f0 0x561c1fd34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f898de2f640 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fd352f0 0x561c1fd34000 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72dd3638e6b2958b server_cookie=ca03157d78e6ce7b in_seq=0 out_seq=0 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f89717fa640 10 client.25394 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f89717fa640 1 -- 172.21.15.5:0/91057644 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561c1fb98960 con 0x561c1fd352f0 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f89717fa640 10 client.25394 handle_command_reply: tid=0 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 client.25394 shutdown 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 msgr2=0x561c1fcd4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 0x561c1fcd4f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89780982e0 tx=0x7f89780046c0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 shutdown_connections 2024-04-22T13:27:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x561c1fd352f0 0x561c1fd34000 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c1fcb9ce0 0x561c1fccde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c1fcb1160 0x561c1fcd4f10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 --2- 172.21.15.5:0/91057644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1fcb07b0 0x561c1fcd49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f89717fa640 0 client.25394 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 >> 172.21.15.5:0/91057644 conn(0x561c1fcaaa60 msgr2=0x561c1fcb5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 shutdown_connections 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 1 -- 172.21.15.5:0/91057644 wait complete. 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.157+0000 7f8992d67740 20 client.25394 trim_cache size 0 max 16384 2024-04-22T13:27:25.167 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24512, 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 131.246587038, 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 92, 2024-04-22T13:27:25.168 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 91, 2024-04-22T13:27:25.169 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 150.55940028000001 2024-04-22T13:27:25.169 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:25.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f89740a3ec0 msgr2=0x7f89740a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f89740a3ec0 0x7f89740a6380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f898400bbe0 tx=0x7f898400b390 comp rx=0 tx=0).stop 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 msgr2=0x7f898819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 0x7f898819eae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f897800b4a0 tx=0x7f897800b960 comp rx=0 tx=0).stop 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 shutdown_connections 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988102a60 0x7f898819eae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f89740a3ec0 0x7f89740a6380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8988103400 0x7f8988198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 --2- 172.21.15.5:0/114061583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8988101180 0x7f898819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 >> 172.21.15.5:0/114061583 conn(0x7f8988077f70 msgr2=0x7f898806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 shutdown_connections 2024-04-22T13:27:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.158+0000 7f8990093640 1 -- 172.21.15.5:0/114061583 wait complete. 2024-04-22T13:27:25.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:24 smithi005 ceph-mon[28006]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:27:25.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:24 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3922175716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:25.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:24 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/469899295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:25.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T13:27:25.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:24 smithi049 ceph-mon[31379]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:27:25.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:24 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3922175716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:25.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:24 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/469899295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1209034412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c103ae0 msgr2=0x7fe89c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1209034412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c103ae0 0x7fe89c10bfc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009900 tx=0x7fe88c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1209034412 shutdown_connections 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1209034412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c103ae0 0x7fe89c10bfc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1209034412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 0x7fe89c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1209034412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c1031c0 0x7fe89c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1209034412 >> 172.21.15.5:0/1209034412 conn(0x7fe89c078000 msgr2=0x7fe89c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1209034412 shutdown_connections 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1209034412 wait complete. 2024-04-22T13:27:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 Processor -- start 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- start start 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 0x7fe89c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c103ae0 0x7fe89c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 0x7fe89c101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c112170 con 0x7fe89c1031c0 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c111ff0 con 0x7fe89c10c500 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.280+0000 7fe8a36eb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1122f0 con 0x7fe89c103ae0 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 0x7fe89c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 0x7fe89c100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47158/0 (socket says 172.21.15.5:47158) 2024-04-22T13:27:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 -- 172.21.15.5:0/1291224708 learned_addr learned my addr 172.21.15.5:0/1291224708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 0x7fe89c101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c103ae0 0x7fe89c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 -- 172.21.15.5:0/1291224708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c103ae0 msgr2=0x7fe89c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c103ae0 0x7fe89c101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 -- 172.21.15.5:0/1291224708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 msgr2=0x7fe89c101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 0x7fe89c101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 -- 172.21.15.5:0/1291224708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c1a6bd0 con 0x7fe89c1031c0 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 0x7fe89c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c103ae0 0x7fe89c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a1487640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 0x7fe89c100e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe89800cce0 tx=0x7fe89800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8927fc640 1 -- 172.21.15.5:0/1291224708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898004b20 con 0x7fe89c1031c0 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8927fc640 1 -- 172.21.15.5:0/1291224708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe898004cc0 con 0x7fe89c1031c0 2024-04-22T13:27:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c1a6ec0 con 0x7fe89c1031c0 2024-04-22T13:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8927fc640 1 -- 172.21.15.5:0/1291224708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89800f6b0 con 0x7fe89c1031c0 2024-04-22T13:27:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.281+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c1a73d0 con 0x7fe89c1031c0 2024-04-22T13:27:25.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:24 smithi107 ceph-mon[31859]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:27:25.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:24 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3922175716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:27:25.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:24 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/469899295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.283+0000 7fe8927fc640 1 -- 172.21.15.5:0/1291224708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fe89800f850 con 0x7fe89c1031c0 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 Processor -- start 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 -- start start 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b87b0 0x56440d7b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b9160 0x56440d7c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 0x56440d7c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56440d71cb10 con 0x56440d7c1ce0 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56440d6ba240 con 0x56440d7b9160 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56440d718ad0 con 0x56440d7b87b0 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a0c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 0x56440d7c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a0c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 0x56440d7c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47170/0 (socket says 172.21.15.5:47170) 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a0c86640 1 -- 172.21.15.5:0/3282720277 learned_addr learned my addr 172.21.15.5:0/3282720277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8927fc640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe8880a3ef0 0x7fe8880a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b87b0 0x56440d7b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe8880a3ef0 0x7fe8880a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.284+0000 7fe8927fc640 1 -- 172.21.15.5:0/1291224708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe8980d0b80 con 0x7fe89c1031c0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a0c86640 1 -- 172.21.15.5:0/3282720277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b87b0 msgr2=0x56440d7b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b87b0 0x56440d7b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a0c86640 1 -- 172.21.15.5:0/3282720277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b9160 msgr2=0x56440d7c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a1487640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b9160 0x56440d7c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b9160 0x56440d7c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a0c86640 1 -- 172.21.15.5:0/3282720277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440d42f2d0 con 0x56440d7c1ce0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b87b0 0x56440d7b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a1487640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b9160 0x56440d7c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 0x56440d7c4130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe88c006010 tx=0x7fe88c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755211b4e8ca7de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe8880a3ef0 0x7fe8880a63b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe894009e30 tx=0x7fe894014040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8877fe640 1 -- 172.21.15.5:0/3282720277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c041020 con 0x56440d7c1ce0 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8877fe640 1 -- 172.21.15.5:0/3282720277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88c005040 con 0x56440d7c1ce0 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.285+0000 7fe8877fe640 1 -- 172.21.15.5:0/3282720277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c02cb50 con 0x56440d7c1ce0 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3282720277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 msgr2=0x56440d7c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 0x56440d7c4130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe88c006010 tx=0x7fe88c005e60 comp rx=0 tx=0).stop 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3282720277 shutdown_connections 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7c1ce0 0x56440d7c4130 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b9160 0x56440d7c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3282720277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b87b0 0x56440d7b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3282720277 >> 172.21.15.5:0/3282720277 conn(0x56440d7b2a60 msgr2=0x56440d7b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3282720277 shutdown_connections 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3282720277 wait complete. 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 -- start start 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.286+0000 7fe8a63bc740 1 Processor -- start 2024-04-22T13:27:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 0x56440d7bec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b9160 0x56440d7bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7c1ce0 0x56440d7bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56440d7c4bf0 con 0x56440d7b87b0 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56440d6a5190 con 0x56440d7c1ce0 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56440d7c4d70 con 0x56440d7b9160 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 10 client.0 _ll_register_callbacks cb 0x56440d6a08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 0x56440d7bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 0x56440d7bec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47178/0 (socket says 172.21.15.5:47178) 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 -- 172.21.15.5:0/3245899753 learned_addr learned my addr 172.21.15.5:0/3245899753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7c1ce0 0x56440d7bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b9160 0x56440d7bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 -- 172.21.15.5:0/3245899753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b9160 msgr2=0x56440d7bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b9160 0x56440d7bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 -- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7c1ce0 msgr2=0x56440d7bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7c1ce0 0x56440d7bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 -- 172.21.15.5:0/3245899753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440d4f96b0 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b9160 0x56440d7bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7c1ce0 0x56440d7bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a1487640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 0x56440d7bec10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe89809c5a0 tx=0x7fe89809c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa129b484edacf1b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe884ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe884ff9640 1 -- 172.21.15.5:0/3245899753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89801d040 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe884ff9640 1 -- 172.21.15.5:0/3245899753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe89800fb40 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.287+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c005040 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe884ff9640 1 -- 172.21.15.5:0/3245899753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8980a0c50 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 10 client.15798 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56440d7d0da0 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe884ff9640 1 -- 172.21.15.5:0/3245899753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88c005040 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe884ff9640 1 -- 172.21.15.5:0/3245899753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe89809bcf0 con 0x56440d7b87b0 2024-04-22T13:27:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 10 client.15798 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 10 client.15798 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 20 client.15798 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d7d3480 0x56440d7d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 4 client.15798 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56440d72a530 con 0x56440d7d3480 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.288+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d7d3480 0x56440d7d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.289+0000 7fe884ff9640 10 client.15798 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.289+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d7d3480 0x56440d7d48f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94ab5932a857055 server_cookie=b49c68c4bed5de66 in_seq=0 out_seq=0 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.289+0000 7fe884ff9640 1 -- 172.21.15.5:0/3245899753 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56440d72a530 con 0x56440d7d3480 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.289+0000 7fe884ff9640 10 client.15798 handle_command_reply: tid=0 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.289+0000 7fe8a63bc740 1 client.15798 shutdown 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 msgr2=0x56440d7bec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 0x56440d7bec10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe89809c5a0 tx=0x7fe89809c4d0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 shutdown_connections 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d7d3480 0x56440d7d48f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7c1ce0 0x56440d7bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe884ff9640 0 client.15798 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7b9160 0x56440d7bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/3245899753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b87b0 0x56440d7bec10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 >> 172.21.15.5:0/3245899753 conn(0x56440d7b2a60 msgr2=0x56440d7b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 shutdown_connections 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 1 -- 172.21.15.5:0/3245899753 wait complete. 2024-04-22T13:27:25.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.290+0000 7fe8a63bc740 20 client.15798 trim_cache size 0 max 16384 2024-04-22T13:27:25.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 Processor -- start 2024-04-22T13:27:25.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 -- start start 2024-04-22T13:27:25.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7bbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56440d7d2c30 con 0x56440d7b9160 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56440d7d3300 con 0x56440d7b87b0 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56440d7d39d0 con 0x56440d7c1ce0 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47184/0 (socket says 172.21.15.5:47184) 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 -- 172.21.15.5:0/552156610 learned_addr learned my addr 172.21.15.5:0/552156610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7bbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a1487640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 -- 172.21.15.5:0/552156610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 msgr2=0x56440d7bbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7bbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 -- 172.21.15.5:0/552156610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 msgr2=0x56440d7c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 -- 172.21.15.5:0/552156610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440d7ba800 con 0x56440d7b9160 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a1487640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bf4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005ab0 tx=0x7fe88c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9bdc8b1cea3555 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe886ffd640 1 -- 172.21.15.5:0/552156610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c051070 con 0x56440d7b9160 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.309+0000 7fe886ffd640 1 -- 172.21.15.5:0/552156610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88c04a070 con 0x56440d7b9160 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe886ffd640 1 -- 172.21.15.5:0/552156610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c041040 con 0x56440d7b9160 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 -- 172.21.15.5:0/552156610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 msgr2=0x56440d7bf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bf4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005ab0 tx=0x7fe88c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 -- 172.21.15.5:0/552156610 shutdown_connections 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bf4b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/552156610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 -- 172.21.15.5:0/552156610 >> 172.21.15.5:0/552156610 conn(0x56440d7b2a60 msgr2=0x56440d7ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 -- 172.21.15.5:0/552156610 shutdown_connections 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 -- 172.21.15.5:0/552156610 wait complete. 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.310+0000 7fe8a63bc740 1 -- start start 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 Processor -- start 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56440d7d4770 con 0x56440d7b9160 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56440d7d40a0 con 0x56440d7b87b0 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56440d7d4e40 con 0x56440d7c1ce0 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a63bc740 10 client.0 _ll_register_callbacks cb 0x7fe89800af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47190/0 (socket says 172.21.15.5:47190) 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 -- 172.21.15.5:0/956837079 learned_addr learned my addr 172.21.15.5:0/956837079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a1c88640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a1487640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 -- 172.21.15.5:0/956837079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 msgr2=0x56440d7dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7dba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 -- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 msgr2=0x56440d7bb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7bb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.311+0000 7fe8a0c86640 1 -- 172.21.15.5:0/956837079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440d7dbf90 con 0x56440d7b9160 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a0c86640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bbbb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009900 tx=0x7fe88c02cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575e7fc01cb18db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe884ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe884ff9640 1 -- 172.21.15.5:0/956837079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c051030 con 0x56440d7b9160 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe884ff9640 1 -- 172.21.15.5:0/956837079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe88c04b2c0 con 0x56440d7b9160 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c04a070 con 0x56440d7b9160 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe884ff9640 1 -- 172.21.15.5:0/956837079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe88c041040 con 0x56440d7b9160 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56440d7dd200 con 0x56440d7b9160 2024-04-22T13:27:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe884ff9640 1 -- 172.21.15.5:0/956837079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88c04a070 con 0x56440d7b9160 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe884ff9640 1 -- 172.21.15.5:0/956837079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe88c034a10 con 0x56440d7b9160 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 10 client.15804 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 20 client.15804 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d83d2c0 0x56440d83bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 4 client.15804 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.312+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "status"}) -- 0x56440d7243a0 con 0x56440d83d2c0 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a1487640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d83d2c0 0x56440d83bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe884ff9640 10 client.15804 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a1487640 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d83d2c0 0x56440d83bfd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbbab132ccce9d6 server_cookie=d9072385e34145d7 in_seq=0 out_seq=0 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe884ff9640 1 -- 172.21.15.5:0/956837079 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56440d7243a0 con 0x56440d83d2c0 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe884ff9640 10 client.15804 handle_command_reply: tid=0 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a63bc740 1 client.15804 shutdown 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 msgr2=0x56440d7bbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bbbb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009900 tx=0x7fe88c02cf80 comp rx=0 tx=0).stop 2024-04-22T13:27:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 shutdown_connections 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56440d83d2c0 0x56440d83bfd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.313+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56440d7b9160 0x56440d7bbbb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe884ff9640 0 client.15804 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56440d7c1ce0 0x56440d7dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a63bc740 1 --2- 172.21.15.5:0/956837079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56440d7b87b0 0x56440d7bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 >> 172.21.15.5:0/956837079 conn(0x56440d7b2a60 msgr2=0x56440d7bd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 shutdown_connections 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a63bc740 1 -- 172.21.15.5:0/956837079 wait complete. 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a63bc740 20 client.15804 trim_cache size 0 max 16384 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "73e95e94-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24557, 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 134.423726426, 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 92, 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 91, 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 139.745695568 2024-04-22T13:27:25.317 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe8880a3ef0 msgr2=0x7fe8880a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe8880a3ef0 0x7fe8880a63b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe894009e30 tx=0x7fe894014040 comp rx=0 tx=0).stop 2024-04-22T13:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 msgr2=0x7fe89c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 0x7fe89c100e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe89800cce0 tx=0x7fe89800a620 comp rx=0 tx=0).stop 2024-04-22T13:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 shutdown_connections 2024-04-22T13:27:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe8880a3ef0 0x7fe8880a63b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89c10c500 0x7fe89c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89c103ae0 0x7fe89c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 --2- 172.21.15.5:0/1291224708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe89c1031c0 0x7fe89c100e10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 >> 172.21.15.5:0/1291224708 conn(0x7fe89c078000 msgr2=0x7fe89c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 shutdown_connections 2024-04-22T13:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:25.314+0000 7fe8a36eb640 1 -- 172.21.15.5:0/1291224708 wait complete. 2024-04-22T13:27:25.321 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-22T13:27:27.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:26 smithi049 ceph-mon[31379]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:27:27.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:26 smithi107 ceph-mon[31859]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:27:27.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:26 smithi005 ceph-mon[28006]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:27:29.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:28 smithi049 ceph-mon[31379]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-04-22T13:27:29.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:28 smithi107 ceph-mon[31859]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-04-22T13:27:29.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:28 smithi005 ceph-mon[28006]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-04-22T13:27:31.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:30 smithi049 ceph-mon[31379]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-04-22T13:27:31.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:30 smithi107 ceph-mon[31859]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-04-22T13:27:31.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:30 smithi005 ceph-mon[28006]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 -- 172.21.15.5:0/1601902026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 msgr2=0x7f39081035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 --2- 172.21.15.5:0/1601902026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 0x7f39081035a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009860 tx=0x7f38f802bc00 comp rx=0 tx=0).stop 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 -- 172.21.15.5:0/1601902026 shutdown_connections 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 --2- 172.21.15.5:0/1601902026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 --2- 172.21.15.5:0/1601902026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 0x7f390810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 --2- 172.21.15.5:0/1601902026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 0x7f39081035a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 -- 172.21.15.5:0/1601902026 >> 172.21.15.5:0/1601902026 conn(0x7f3908077f70 msgr2=0x7f3908078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 -- 172.21.15.5:0/1601902026 shutdown_connections 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.428+0000 7f390e504640 1 -- 172.21.15.5:0/1601902026 wait complete. 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.429+0000 7f390e504640 1 Processor -- start 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.429+0000 7f390e504640 1 -- start start 2024-04-22T13:27:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.429+0000 7f390e504640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 0x7f390819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.429+0000 7f390e504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.429+0000 7f390e504640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 0x7f39081985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390e504640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39081121c0 con 0x7f3908103ae0 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390e504640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3908112040 con 0x7f390810c590 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390e504640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3908112340 con 0x7f39081031c0 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48126/0 (socket says 172.21.15.5:48126) 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39077fe640 1 -- 172.21.15.5:0/1155852051 learned_addr learned my addr 172.21.15.5:0/1155852051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 0x7f39081985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f3907fff640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 0x7f390819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 -- 172.21.15.5:0/1155852051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 msgr2=0x7f390819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 0x7f390819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 -- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 msgr2=0x7f390819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 -- 172.21.15.5:0/1155852051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3908198d40 con 0x7f390810c590 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39077fe640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390caa1640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 0x7f39081985c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3900005fd0 tx=0x7f390000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39057fa640 1 -- 172.21.15.5:0/1155852051 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900003b70 con 0x7f390810c590 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3908199030 con 0x7f390810c590 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39057fa640 1 -- 172.21.15.5:0/1155852051 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3900003d10 con 0x7f390810c590 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.430+0000 7f39057fa640 1 -- 172.21.15.5:0/1155852051 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39000107a0 con 0x7f390810c590 2024-04-22T13:27:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.431+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a52d0 con 0x7f390810c590 2024-04-22T13:27:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39057fa640 1 -- 172.21.15.5:0/1155852051 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f3900003fc0 con 0x7f390810c590 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 Processor -- start 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 -- start start 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 0x557b6de06620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de06fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b6dd6b8d0 con 0x557b6de06b60 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557b6dd08e80 con 0x557b6de078d0 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39111d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557b6dd67890 con 0x557b6de100e0 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f3907fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de06fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 0x557b6de06620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f3907fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de06fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48130/0 (socket says 172.21.15.5:48130) 2024-04-22T13:27:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f3907fff640 1 -- 172.21.15.5:0/3519206819 learned_addr learned my addr 172.21.15.5:0/3519206819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 0x557b6de06620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33050/0 (socket says 172.21.15.5:33050) 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.433+0000 7f39077fe640 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39057fa640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e30 0x7f38f00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39077fe640 1 -- 172.21.15.5:0/3519206819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 msgr2=0x557b6de124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39077fe640 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39077fe640 1 -- 172.21.15.5:0/3519206819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 msgr2=0x557b6de06fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39077fe640 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de06fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39077fe640 1 -- 172.21.15.5:0/3519206819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6dd2a320 con 0x557b6de078d0 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f3907fff640 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de06fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f390caa1640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e30 0x7f38f00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39057fa640 1 -- 172.21.15.5:0/1155852051 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f39000d0f30 con 0x7f390810c590 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f39077fe640 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 0x557b6de06620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3908199700 tx=0x7f38fc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5aa06cac4a34356 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f38ee7fc640 1 -- 172.21.15.5:0/3519206819 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00fc20 con 0x557b6de078d0 2024-04-22T13:27:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f38ee7fc640 1 -- 172.21.15.5:0/3519206819 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38fc00fdc0 con 0x557b6de078d0 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.434+0000 7f38ee7fc640 1 -- 172.21.15.5:0/3519206819 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0047a0 con 0x557b6de078d0 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 -- 172.21.15.5:0/3519206819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 msgr2=0x557b6de06620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f390caa1640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e30 0x7f38f00a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39000ce590 tx=0x7f3900007b90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 0x557b6de06620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3908199700 tx=0x7f38fc009bb0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 -- 172.21.15.5:0/3519206819 shutdown_connections 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de06fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 --2- 172.21.15.5:0/3519206819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de078d0 0x557b6de06620 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 -- 172.21.15.5:0/3519206819 >> 172.21.15.5:0/3519206819 conn(0x557b6ddcdc90 msgr2=0x557b6de01850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 -- 172.21.15.5:0/3519206819 shutdown_connections 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 -- 172.21.15.5:0/3519206819 wait complete. 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.435+0000 7f39111d1740 1 -- start start 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 Processor -- start 2024-04-22T13:27:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de0fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 0x557b6de0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de0d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b6de12e10 con 0x557b6de078d0 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557b6dcf3eb0 con 0x557b6de06b60 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557b6de12f90 con 0x557b6de100e0 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39111d1740 10 client.0 _ll_register_callbacks cb 0x557b6dd20b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 0x557b6de0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de0fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 0x557b6de0ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48134/0 (socket says 172.21.15.5:48134) 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 -- 172.21.15.5:0/2163317372 learned_addr learned my addr 172.21.15.5:0/2163317372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f39077fe640 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de0d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 -- 172.21.15.5:0/2163317372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 msgr2=0x557b6de0d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de0d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 -- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 msgr2=0x557b6de0fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de0fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.436+0000 7f3907fff640 1 -- 172.21.15.5:0/2163317372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6da7e210 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f3907fff640 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 0x557b6de0ce00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38f8005940 tx=0x7f38f8034960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759adc88bd9f9b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f38e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f38e77fe640 1 -- 172.21.15.5:0/2163317372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8039040 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f38e77fe640 1 -- 172.21.15.5:0/2163317372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f38f8004040 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38fc00fdc0 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f38e77fe640 1 -- 172.21.15.5:0/2163317372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38f804e050 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 10 client.15816 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b6db4a620 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f38e77fe640 1 -- 172.21.15.5:0/2163317372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38fc00fdc0 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f38e77fe640 1 -- 172.21.15.5:0/2163317372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f38f8003680 con 0x557b6de078d0 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 20 client.15816 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de20130 0x557b6de22520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 4 client.15816 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.437+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6dd7f880 con 0x557b6de20130 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.438+0000 7f39077fe640 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de20130 0x557b6de22520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.438+0000 7f39077fe640 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de20130 0x557b6de22520 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d05a0caa4eb7ce server_cookie=ef484cbbaa2de847 in_seq=0 out_seq=0 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.438+0000 7f38e77fe640 10 client.15816 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f38e77fe640 1 -- 172.21.15.5:0/2163317372 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b6dd7f880 con 0x557b6de20130 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f38e77fe640 10 client.15816 handle_command_reply: tid=0 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 client.15816 shutdown 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 msgr2=0x557b6de0ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 0x557b6de0ce00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38f8005940 tx=0x7f38f8034960 comp rx=0 tx=0).stop 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 shutdown_connections 2024-04-22T13:27:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de20130 0x557b6de22520 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de078d0 0x557b6de0ce00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f38e77fe640 0 client.15816 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de100e0 0x557b6de0d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 --2- 172.21.15.5:0/2163317372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de0fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 >> 172.21.15.5:0/2163317372 conn(0x557b6ddcdc90 msgr2=0x557b6de035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 shutdown_connections 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 1 -- 172.21.15.5:0/2163317372 wait complete. 2024-04-22T13:27:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.439+0000 7f39111d1740 20 client.15816 trim_cache size 0 max 16384 2024-04-22T13:27:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.454+0000 7f39111d1740 1 Processor -- start 2024-04-22T13:27:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.454+0000 7f39111d1740 1 -- start start 2024-04-22T13:27:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39111d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de1bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39111d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de0c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39111d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de100e0 0x557b6de27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39111d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39081adb40 con 0x557b6de06b60 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39111d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39081af680 con 0x557b6de100e0 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39111d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39081ae210 con 0x557b6de078d0 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de1bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de1bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48136/0 (socket says 172.21.15.5:48136) 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 -- 172.21.15.5:0/2843524616 learned_addr learned my addr 172.21.15.5:0/2843524616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f3907fff640 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de0c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de100e0 0x557b6de27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 -- 172.21.15.5:0/2843524616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 msgr2=0x557b6de0c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de0c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 -- 172.21.15.5:0/2843524616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de100e0 msgr2=0x557b6de27e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de100e0 0x557b6de27e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 -- 172.21.15.5:0/2843524616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6de22ba0 con 0x557b6de06b60 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f39077fe640 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de1bd40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0058c0 tx=0x7f38fc0114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b178aabe0ecba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f38edffb640 1 -- 172.21.15.5:0/2843524616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00fd80 con 0x557b6de06b60 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f38edffb640 1 -- 172.21.15.5:0/2843524616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38fc011050 con 0x557b6de06b60 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.455+0000 7f38edffb640 1 -- 172.21.15.5:0/2843524616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc003aa0 con 0x557b6de06b60 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 -- 172.21.15.5:0/2843524616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 msgr2=0x557b6de1bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de1bd40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0058c0 tx=0x7f38fc0114c0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 -- 172.21.15.5:0/2843524616 shutdown_connections 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de100e0 0x557b6de27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de0c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 --2- 172.21.15.5:0/2843524616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de06b60 0x557b6de1bd40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 -- 172.21.15.5:0/2843524616 >> 172.21.15.5:0/2843524616 conn(0x557b6ddcdc90 msgr2=0x557b6de01380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 -- 172.21.15.5:0/2843524616 shutdown_connections 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 -- 172.21.15.5:0/2843524616 wait complete. 2024-04-22T13:27:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.456+0000 7f39111d1740 1 -- start start 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 Processor -- start 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de09670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 0x557b6de09bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39081aefb0 con 0x557b6de100e0 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39081ae8e0 con 0x557b6de06b60 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39081abb20 con 0x557b6de078d0 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39111d1740 10 client.0 _ll_register_callbacks cb 0x7f39000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de09670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 0x557b6de09bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de09670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:49938/0 (socket says 172.21.15.5:49938) 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 0x557b6de09bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48146/0 (socket says 172.21.15.5:48146) 2024-04-22T13:27:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 -- 172.21.15.5:0/757956256 learned_addr learned my addr 172.21.15.5:0/757956256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39077fe640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 -- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 msgr2=0x557b6de184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 -- 172.21.15.5:0/757956256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 msgr2=0x557b6de09bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 0x557b6de09bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f3907fff640 1 -- 172.21.15.5:0/757956256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6de05e30 con 0x557b6de078d0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.457+0000 7f39077fe640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 0x557b6de09bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f3907fff640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de09670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38f802c140 tx=0x7f38f8005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d0447a554196b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f38e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f38e77fe640 1 -- 172.21.15.5:0/757956256 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8056020 con 0x557b6de078d0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38fc011050 con 0x557b6de078d0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f38e77fe640 1 -- 172.21.15.5:0/757956256 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f38f8003800 con 0x557b6de078d0 2024-04-22T13:27:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f38e77fe640 1 -- 172.21.15.5:0/757956256 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38f8061050 con 0x557b6de078d0 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 10 client.25483 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b6de86370 con 0x557b6de078d0 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f38e77fe640 1 -- 172.21.15.5:0/757956256 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38fc011050 con 0x557b6de078d0 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f38e77fe640 1 -- 172.21.15.5:0/757956256 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f38f8003ef0 con 0x557b6de078d0 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 10 client.25483 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 10 client.25483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 20 client.25483 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de86b70 0x557b6de88f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 4 client.25483 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.458+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b6dd0b3d0 con 0x557b6de86b70 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.459+0000 7f39077fe640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de86b70 0x557b6de88f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.459+0000 7f38e77fe640 10 client.25483 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.459+0000 7f39077fe640 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de86b70 0x557b6de88f90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa57d91551a2fdc server_cookie=66014a802580b9e0 in_seq=0 out_seq=0 2024-04-22T13:27:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f38e77fe640 1 -- 172.21.15.5:0/757956256 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b6dd0b3d0 con 0x557b6de86b70 2024-04-22T13:27:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f38e77fe640 10 client.25483 handle_command_reply: tid=0 2024-04-22T13:27:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 client.25483 shutdown 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 msgr2=0x557b6de09670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de09670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38f802c140 tx=0x7f38f8005d10 comp rx=0 tx=0).stop 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 shutdown_connections 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557b6de86b70 0x557b6de88f90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b6de100e0 0x557b6de09bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b6de078d0 0x557b6de09670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 --2- 172.21.15.5:0/757956256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557b6de06b60 0x557b6de184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f38e77fe640 0 client.25483 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 >> 172.21.15.5:0/757956256 conn(0x557b6ddcdc90 msgr2=0x557b6de0b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 shutdown_connections 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 1 -- 172.21.15.5:0/757956256 wait complete. 2024-04-22T13:27:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.470+0000 7f39111d1740 20 client.25483 trim_cache size 0 max 16384 2024-04-22T13:27:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:27:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:27:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:27:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e30 msgr2=0x7f38f00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e30 0x7f38f00a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39000ce590 tx=0x7f3900007b90 comp rx=0 tx=0).stop 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 msgr2=0x7f39081985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 0x7f39081985c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3900005fd0 tx=0x7f390000c6a0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 shutdown_connections 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3908103ae0 0x7f390819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f38f00a3e30 0x7f38f00a62f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390810c590 0x7f39081985c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 --2- 172.21.15.5:0/1155852051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081031c0 0x7f390819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 >> 172.21.15.5:0/1155852051 conn(0x7f3908077f70 msgr2=0x7f39080fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 shutdown_connections 2024-04-22T13:27:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:31.471+0000 7f390e504640 1 -- 172.21.15.5:0/1155852051 wait complete. 2024-04-22T13:27:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:32 smithi049 ceph-mon[31379]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-22T13:27:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:32 smithi107 ceph-mon[31859]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-22T13:27:33.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:32 smithi005 ceph-mon[28006]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-22T13:27:35.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:34 smithi049 ceph-mon[31379]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-22T13:27:35.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:34 smithi107 ceph-mon[31859]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-22T13:27:35.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:34 smithi005 ceph-mon[28006]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-22T13:27:37.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:36 smithi049 ceph-mon[31379]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-22T13:27:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:36 smithi107 ceph-mon[31859]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-22T13:27:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:36 smithi005 ceph-mon[28006]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-22T13:27:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:38 smithi049 ceph-mon[31379]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-04-22T13:27:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:38 smithi107 ceph-mon[31859]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-04-22T13:27:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:38 smithi005 ceph-mon[28006]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-04-22T13:27:40.321 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-22T13:27:40.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 -- 172.21.15.5:0/3995693528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 msgr2=0x7f3fec101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3995693528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec101450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009900 tx=0x7f3fe002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 -- 172.21.15.5:0/3995693528 shutdown_connections 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3995693528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fec106140 0x7f3fec10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3995693528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fec101a20 0x7f3fec105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3995693528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec101450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 -- 172.21.15.5:0/3995693528 >> 172.21.15.5:0/3995693528 conn(0x7f3fec0771d0 msgr2=0x7f3fec0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 -- 172.21.15.5:0/3995693528 shutdown_connections 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.443+0000 7f3ff1715640 1 -- 172.21.15.5:0/3995693528 wait complete. 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 Processor -- start 2024-04-22T13:27:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 -- start start 2024-04-22T13:27:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fec106140 0x7f3fec198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fec19cfb0 0x7f3fec19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1121f0 con 0x7f3fec101070 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112070 con 0x7f3fec106140 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3ff1715640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112370 con 0x7f3fec19cfb0 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3feaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3feaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58716/0 (socket says 172.21.15.5:58716) 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3feaffd640 1 -- 172.21.15.5:0/3328138283 learned_addr learned my addr 172.21.15.5:0/3328138283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.444+0000 7f3fea7fc640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fec106140 0x7f3fec198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feb7fe640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fec19cfb0 0x7f3fec19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feaffd640 1 -- 172.21.15.5:0/3328138283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fec19cfb0 msgr2=0x7f3fec19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feaffd640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fec19cfb0 0x7f3fec19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feaffd640 1 -- 172.21.15.5:0/3328138283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fec106140 msgr2=0x7f3fec198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feaffd640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fec106140 0x7f3fec198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feaffd640 1 -- 172.21.15.5:0/3328138283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec19daa0 con 0x7f3fec101070 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3feaffd640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec1986f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0043020 con 0x7f3fec101070 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec19dd30 con 0x7f3fec101070 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe002cbb0 con 0x7f3fec101070 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0003ac0 con 0x7f3fec101070 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.445+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec1a5550 con 0x7f3fec101070 2024-04-22T13:27:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.446+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fec068e00 con 0x7f3fec101070 2024-04-22T13:27:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.448+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f3fe0046020 con 0x7f3fec101070 2024-04-22T13:27:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.448+0000 7f3fd3fff640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fd40a3e40 0x7f3fd40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.448+0000 7f3fea7fc640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fd40a3e40 0x7f3fd40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.448+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f3fe00f2f30 con 0x7f3fec101070 2024-04-22T13:27:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.449+0000 7f3fea7fc640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fd40a3e40 0x7f3fd40a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0ff700 tx=0x7f3fdc005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.452+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fe004d020 con 0x7f3fec101070 2024-04-22T13:27:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.684+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fec199a40 con 0x7f3fec101070 2024-04-22T13:27:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.685+0000 7f3fd3fff640 1 -- 172.21.15.5:0/3328138283 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f3fec199a40 con 0x7f3fec101070 2024-04-22T13:27:40.681 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:40.681 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.687+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fd40a3e40 msgr2=0x7f3fd40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fd40a3e40 0x7f3fd40a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0ff700 tx=0x7f3fdc005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 msgr2=0x7f3fec1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec1986f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe002c6c0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 shutdown_connections 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fd40a3e40 0x7f3fd40a6300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fec19cfb0 0x7f3fec19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fec106140 0x7f3fec198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 --2- 172.21.15.5:0/3328138283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fec101070 0x7f3fec1986f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 >> 172.21.15.5:0/3328138283 conn(0x7f3fec0771d0 msgr2=0x7f3fec107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 shutdown_connections 2024-04-22T13:27:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.688+0000 7f3ff1715640 1 -- 172.21.15.5:0/3328138283 wait complete. 2024-04-22T13:27:40.695 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db / --timeout 60 --expiration 70 --await' 2024-04-22T13:27:40.695 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db / --timeout 60 --expiration 70 --await 2024-04-22T13:27:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 -- 172.21.15.5:0/3910978623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 msgr2=0x7f824810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 --2- 172.21.15.5:0/3910978623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 0x7f824810e9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8240009be0 tx=0x7f824002bf80 comp rx=0 tx=0).stop 2024-04-22T13:27:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 -- 172.21.15.5:0/3910978623 shutdown_connections 2024-04-22T13:27:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 --2- 172.21.15.5:0/3910978623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 0x7f824810e9c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 --2- 172.21.15.5:0/3910978623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8248103aa0 0x7f824810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 --2- 172.21.15.5:0/3910978623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f8248103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 -- 172.21.15.5:0/3910978623 >> 172.21.15.5:0/3910978623 conn(0x7f8248078030 msgr2=0x7f8248078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 -- 172.21.15.5:0/3910978623 shutdown_connections 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 -- 172.21.15.5:0/3910978623 wait complete. 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 Processor -- start 2024-04-22T13:27:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.811+0000 7f824e70d640 1 -- start start 2024-04-22T13:27:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824e70d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f82481986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824e70d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8248103aa0 0x7f8248198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824e70d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 0x7f824819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824e70d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8248112190 con 0x7f8248103180 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824e70d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8248112010 con 0x7f8248103aa0 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824e70d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8248112310 con 0x7f824810c550 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f82481986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f82481986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58742/0 (socket says 172.21.15.5:58742) 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 -- 172.21.15.5:0/2091110421 learned_addr learned my addr 172.21.15.5:0/2091110421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f824ccaa640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 0x7f824819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f82477fe640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8248103aa0 0x7f8248198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 -- 172.21.15.5:0/2091110421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 msgr2=0x7f824819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 0x7f824819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 -- 172.21.15.5:0/2091110421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8248103aa0 msgr2=0x7f8248198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8248103aa0 0x7f8248198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 -- 172.21.15.5:0/2091110421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824819d6c0 con 0x7f8248103180 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.812+0000 7f8247fff640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f82481986d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f823800e9c0 tx=0x7f823800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.813+0000 7f82457fa640 1 -- 172.21.15.5:0/2091110421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823800ccc0 con 0x7f8248103180 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.813+0000 7f82457fa640 1 -- 172.21.15.5:0/2091110421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82380054d0 con 0x7f8248103180 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.813+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824819d9b0 con 0x7f8248103180 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.813+0000 7f82457fa640 1 -- 172.21.15.5:0/2091110421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238010430 con 0x7f8248103180 2024-04-22T13:27:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.813+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a5240 con 0x7f8248103180 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82457fa640 1 -- 172.21.15.5:0/2091110421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f8238004f80 con 0x7f8248103180 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 Processor -- start 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 -- start start 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb22da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb2b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb2e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4bca830b0 con 0x55a4bcb229c0 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4bca34680 con 0x55a4bcb232e0 2024-04-22T13:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82513cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a4bca40c60 con 0x55a4bcb2bec0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb22da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb22da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58750/0 (socket says 172.21.15.5:58750) 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 -- 172.21.15.5:0/3345675305 learned_addr learned my addr 172.21.15.5:0/3345675305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82477fe640 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb2e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f8247fff640 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb2b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 -- 172.21.15.5:0/3345675305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 msgr2=0x55a4bcb2e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb2e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 -- 172.21.15.5:0/3345675305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 msgr2=0x55a4bcb2b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb2b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f824ccaa640 1 -- 172.21.15.5:0/3345675305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4bc865c90 con 0x55a4bcb229c0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82457fa640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82340a3e40 0x7f82340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.816+0000 7f82477fe640 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb2e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f82477fe640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82340a3e40 0x7f82340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f82457fa640 1 -- 172.21.15.5:0/2091110421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82380d0e60 con 0x7f8248103180 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f824ccaa640 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb22da0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8240009be0 tx=0x7f82400098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2909602ea8ee4655 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f823e7fc640 1 -- 172.21.15.5:0/3345675305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240043030 con 0x55a4bcb229c0 2024-04-22T13:27:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f823e7fc640 1 -- 172.21.15.5:0/3345675305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82400333e0 con 0x55a4bcb229c0 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f823e7fc640 1 -- 172.21.15.5:0/3345675305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240003d50 con 0x55a4bcb229c0 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f82513cf740 1 -- 172.21.15.5:0/3345675305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 msgr2=0x55a4bcb22da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f82477fe640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82340a3e40 0x7f82340a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55a4bcb1e280 tx=0x7f8230005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.817+0000 7f82513cf740 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb22da0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8240009be0 tx=0x7f82400098b0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- 172.21.15.5:0/3345675305 shutdown_connections 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb2e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb2b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 --2- 172.21.15.5:0/3345675305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb22da0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- 172.21.15.5:0/3345675305 >> 172.21.15.5:0/3345675305 conn(0x55a4bcb1d0f0 msgr2=0x55a4bcb1d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- 172.21.15.5:0/3345675305 shutdown_connections 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- 172.21.15.5:0/3345675305 wait complete. 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- start start 2024-04-22T13:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 Processor -- start 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 0x55a4bcb28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 0x55a4bcb29400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb29940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4bca86400 con 0x55a4bcb232e0 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4bca42840 con 0x55a4bcb229c0 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a4bca8b720 con 0x55a4bcb2bec0 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.818+0000 7f82513cf740 10 client.0 _ll_register_callbacks cb 0x55a4bca93ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 0x55a4bcb29400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f8247fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 0x55a4bcb28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 0x55a4bcb29400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58756/0 (socket says 172.21.15.5:58756) 2024-04-22T13:27:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f8247fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 0x55a4bcb28ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:57046/0 (socket says 172.21.15.5:57046) 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 -- 172.21.15.5:0/988725588 learned_addr learned my addr 172.21.15.5:0/988725588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f824ccaa640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb29940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 -- 172.21.15.5:0/988725588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 msgr2=0x55a4bcb29940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb29940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 -- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 msgr2=0x55a4bcb28ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 0x55a4bcb28ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 -- 172.21.15.5:0/988725588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4bc8c6420 con 0x55a4bcb232e0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f824ccaa640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb29940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f8247fff640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 0x55a4bcb28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82477fe640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 0x55a4bcb29400 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f823001e9d0 tx=0x7f8230004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e179bd7af92a2e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82277fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82277fe640 1 -- 172.21.15.5:0/988725588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230009850 con 0x55a4bcb232e0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82277fe640 1 -- 172.21.15.5:0/988725588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8230018030 con 0x55a4bcb232e0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82400333e0 con 0x55a4bcb232e0 2024-04-22T13:27:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.819+0000 7f82277fe640 1 -- 172.21.15.5:0/988725588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8230004020 con 0x55a4bcb232e0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 10 client.15840 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4bc759580 con 0x55a4bcb232e0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82277fe640 1 -- 172.21.15.5:0/988725588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82400333e0 con 0x55a4bcb232e0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82277fe640 1 -- 172.21.15.5:0/988725588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8230029420 con 0x55a4bcb232e0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 10 client.15840 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 10 client.15840 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 20 client.15840 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcb3bba0 0x55a4bcb3e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 4 client.15840 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4bca94b10 con 0x55a4bcb3bba0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.820+0000 7f8247fff640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcb3bba0 0x55a4bcb3e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.821+0000 7f82277fe640 10 client.15840 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.821+0000 7f8247fff640 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcb3bba0 0x55a4bcb3e040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5ef43c4a6dedbe server_cookie=5584ae159d70b108 in_seq=0 out_seq=0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.821+0000 7f82277fe640 1 -- 172.21.15.5:0/988725588 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4bca94b10 con 0x55a4bcb3bba0 2024-04-22T13:27:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.821+0000 7f82277fe640 10 client.15840 handle_command_reply: tid=0 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.821+0000 7f82513cf740 1 client.15840 shutdown 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 msgr2=0x55a4bcb29400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 0x55a4bcb29400 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f823001e9d0 tx=0x7f8230004730 comp rx=0 tx=0).stop 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 shutdown_connections 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcb3bba0 0x55a4bcb3e040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb232e0 0x55a4bcb29400 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb29940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82277fe640 0 client.15840 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 --2- 172.21.15.5:0/988725588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb229c0 0x55a4bcb28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 >> 172.21.15.5:0/988725588 conn(0x55a4bcb1d0f0 msgr2=0x55a4bcb1ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 shutdown_connections 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 1 -- 172.21.15.5:0/988725588 wait complete. 2024-04-22T13:27:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.822+0000 7f82513cf740 20 client.15840 trim_cache size 0 max 16384 2024-04-22T13:27:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 Processor -- start 2024-04-22T13:27:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 -- start start 2024-04-22T13:27:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb37840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb27790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82481af600 con 0x55a4bcb229c0 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82481ae190 con 0x55a4bcb232e0 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82513cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82481ae860 con 0x55a4bcb2bec0 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb1bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58766/0 (socket says 172.21.15.5:58766) 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 -- 172.21.15.5:0/464710561 learned_addr learned my addr 172.21.15.5:0/464710561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f824ccaa640 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb27790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f82477fe640 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb37840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 -- 172.21.15.5:0/464710561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 msgr2=0x55a4bcb27790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb27790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 -- 172.21.15.5:0/464710561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 msgr2=0x55a4bcb37840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb37840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.839+0000 7f8247fff640 1 -- 172.21.15.5:0/464710561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4bcb175e0 con 0x55a4bcb229c0 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82477fe640 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb37840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f8247fff640 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb1bfb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f823809c8e0 tx=0x7f82380d4970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a009b4633b7cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f823dffb640 1 -- 172.21.15.5:0/464710561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823809c720 con 0x55a4bcb229c0 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f823dffb640 1 -- 172.21.15.5:0/464710561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823809d2e0 con 0x55a4bcb229c0 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f823dffb640 1 -- 172.21.15.5:0/464710561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823800ce60 con 0x55a4bcb229c0 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 -- 172.21.15.5:0/464710561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 msgr2=0x55a4bcb1bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb1bfb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f823809c8e0 tx=0x7f82380d4970 comp rx=0 tx=0).stop 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 -- 172.21.15.5:0/464710561 shutdown_connections 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb2bec0 0x55a4bcb27790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb232e0 0x55a4bcb37840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 --2- 172.21.15.5:0/464710561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb1bfb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 -- 172.21.15.5:0/464710561 >> 172.21.15.5:0/464710561 conn(0x55a4bcb1d0f0 msgr2=0x55a4bcb29710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 -- 172.21.15.5:0/464710561 shutdown_connections 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.840+0000 7f82513cf740 1 -- 172.21.15.5:0/464710561 wait complete. 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 -- start start 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 Processor -- start 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb232e0 0x55a4bcb30480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb2bec0 0x55a4bcba9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82481abaa0 con 0x55a4bcb229c0 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82481aef30 con 0x55a4bcb2bec0 2024-04-22T13:27:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82481ac110 con 0x55a4bcb232e0 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82513cf740 10 client.0 _ll_register_callbacks cb 0x7f8230009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f8247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f8247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb2ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58780/0 (socket says 172.21.15.5:58780) 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f8247fff640 1 -- 172.21.15.5:0/3421725571 learned_addr learned my addr 172.21.15.5:0/3421725571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f824ccaa640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb2bec0 0x55a4bcba9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.841+0000 7f82477fe640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb232e0 0x55a4bcb30480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f8247fff640 1 -- 172.21.15.5:0/3421725571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb232e0 msgr2=0x55a4bcb30480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f8247fff640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb232e0 0x55a4bcb30480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f8247fff640 1 -- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb2bec0 msgr2=0x55a4bcba9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f8247fff640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb2bec0 0x55a4bcba9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f8247fff640 1 -- 172.21.15.5:0/3421725571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4bcba9f80 con 0x55a4bcb229c0 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f824ccaa640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb2bec0 0x55a4bcba9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82477fe640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb232e0 0x55a4bcb30480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f8247fff640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb2ff40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8238099560 tx=0x7f8238009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ae555f8e19422 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82277fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82277fe640 1 -- 172.21.15.5:0/3421725571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82380b5070 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82277fe640 1 -- 172.21.15.5:0/3421725571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f823809aee0 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823809d2e0 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82277fe640 1 -- 172.21.15.5:0/3421725571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f823809c540 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 10 client.15846 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4bcbab1b0 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82277fe640 1 -- 172.21.15.5:0/3421725571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823809d2e0 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82277fe640 1 -- 172.21.15.5:0/3421725571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f82380ce740 con 0x55a4bcb229c0 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 10 client.15846 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 20 client.15846 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcbab890 0x55a4bcbadc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 4 client.15846 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await": true}] 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.842+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await": true}) -- 0x55a4bca8e980 con 0x55a4bcbab890 2024-04-22T13:27:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.843+0000 7f82477fe640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcbab890 0x55a4bcbadc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:40.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.843+0000 7f82277fe640 10 client.15846 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:40.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.843+0000 7f82477fe640 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcbab890 0x55a4bcbadc80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c627c237a00684e server_cookie=a5da15a48a20265d in_seq=0 out_seq=0 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82277fe640 1 -- 172.21.15.5:0/3421725571 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55a4bca8e980 con 0x55a4bcbab890 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82277fe640 10 client.15846 handle_command_reply: tid=0 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82513cf740 1 client.15846 shutdown 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 msgr2=0x55a4bcb2ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82513cf740 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb2ff40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8238099560 tx=0x7f8238009f90 comp rx=0 tx=0).stop 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 shutdown_connections 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.856+0000 7f82513cf740 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4bcbab890 0x55a4bcbadc80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4bcb2bec0 0x55a4bcba9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4bcb232e0 0x55a4bcb30480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82277fe640 0 client.15846 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 1 --2- 172.21.15.5:0/3421725571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4bcb229c0 0x55a4bcb2ff40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 >> 172.21.15.5:0/3421725571 conn(0x55a4bcb1d0f0 msgr2=0x55a4bcb29710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 shutdown_connections 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 1 -- 172.21.15.5:0/3421725571 wait complete. 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f82513cf740 20 client.15846 trim_cache size 0 max 16384 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 2, 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "4a1cd248": { 2024-04-22T13:27:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2, 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 70.0, 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:40.856 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:40.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82340a3e40 msgr2=0x7f82340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82340a3e40 0x7f82340a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55a4bcb1e280 tx=0x7f8230005f90 comp rx=0 tx=0).stop 2024-04-22T13:27:40.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 msgr2=0x7f82481986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f82481986d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f823800e9c0 tx=0x7f823800ee80 comp rx=0 tx=0).stop 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 shutdown_connections 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82340a3e40 0x7f82340a6300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824810c550 0x7f824819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8248103aa0 0x7f8248198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 --2- 172.21.15.5:0/2091110421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8248103180 0x7f82481986d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 >> 172.21.15.5:0/2091110421 conn(0x7f8248078030 msgr2=0x7f82480fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 shutdown_connections 2024-04-22T13:27:40.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:40.857+0000 7f824e70d640 1 -- 172.21.15.5:0/2091110421 wait complete. 2024-04-22T13:27:40.864 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 4a1cd248, quiesce took 0.5 seconds. Will release after: 10 2024-04-22T13:27:41.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:40 smithi005 ceph-mon[28006]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-22T13:27:41.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:40 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3328138283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:41.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:40 smithi049 ceph-mon[31379]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-22T13:27:41.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:40 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3328138283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:41.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:40 smithi107 ceph-mon[31859]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-22T13:27:41.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:40 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3328138283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:43.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:42 smithi049 ceph-mon[31379]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-22T13:27:43.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:42 smithi107 ceph-mon[31859]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-22T13:27:43.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:42 smithi005 ceph-mon[28006]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-22T13:27:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.569+0000 7f42ed79f640 1 -- 172.21.15.5:0/649283859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e81070b0 msgr2=0x7f42e810ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.569+0000 7f42ed79f640 1 --2- 172.21.15.5:0/649283859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e81070b0 0x7f42e810ec80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009ca0 tx=0x7f42e002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.569+0000 7f42ed79f640 1 -- 172.21.15.5:0/649283859 shutdown_connections 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 --2- 172.21.15.5:0/649283859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e81070b0 0x7f42e810ec80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 --2- 172.21.15.5:0/649283859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e8102990 0x7f42e81069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 --2- 172.21.15.5:0/649283859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e8101fe0 0x7f42e81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 -- 172.21.15.5:0/649283859 >> 172.21.15.5:0/649283859 conn(0x7f42e80fd900 msgr2=0x7f42e80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 -- 172.21.15.5:0/649283859 shutdown_connections 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 -- 172.21.15.5:0/649283859 wait complete. 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 Processor -- start 2024-04-22T13:27:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.570+0000 7f42ed79f640 1 -- start start 2024-04-22T13:27:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ed79f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e8101fe0 0x7f42e81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ed79f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e81070b0 0x7f42e8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ed79f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 0x7f42e819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ed79f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8112020 con 0x7f42e819ce80 2024-04-22T13:27:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ed79f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8111ea0 con 0x7f42e8101fe0 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ed79f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81121a0 con 0x7f42e81070b0 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 0x7f42e819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 0x7f42e819d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58796/0 (socket says 172.21.15.5:58796) 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 -- 172.21.15.5:0/4170622825 learned_addr learned my addr 172.21.15.5:0/4170622825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42e7fff640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e8101fe0 0x7f42e81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42e77fe640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e81070b0 0x7f42e8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 -- 172.21.15.5:0/4170622825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e81070b0 msgr2=0x7f42e8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e81070b0 0x7f42e8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 -- 172.21.15.5:0/4170622825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e8101fe0 msgr2=0x7f42e81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e8101fe0 0x7f42e81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.571+0000 7f42ecf9e640 1 -- 172.21.15.5:0/4170622825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e819d970 con 0x7f42e819ce80 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.572+0000 7f42ecf9e640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 0x7f42e819d330 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005e30 tx=0x7f42e0032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.572+0000 7f42e57fa640 1 -- 172.21.15.5:0/4170622825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0041020 con 0x7f42e819ce80 2024-04-22T13:27:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.572+0000 7f42e57fa640 1 -- 172.21.15.5:0/4170622825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e0032970 con 0x7f42e819ce80 2024-04-22T13:27:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.572+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e819dc00 con 0x7f42e819ce80 2024-04-22T13:27:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.572+0000 7f42e57fa640 1 -- 172.21.15.5:0/4170622825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0039040 con 0x7f42e819ce80 2024-04-22T13:27:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.572+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a53c0 con 0x7f42e819ce80 2024-04-22T13:27:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42e57fa640 1 -- 172.21.15.5:0/4170622825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f42e0044020 con 0x7f42e819ce80 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 Processor -- start 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 -- start start 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309e9ac0 0x556e309e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e309e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309f66e0 0x556e309f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e3094d8d0 con 0x556e309e8d50 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556e308eae80 con 0x556e309f66e0 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.575+0000 7f42f15f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e30949890 con 0x556e309e9ac0 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e309e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e57fa640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42d40a3e20 0x7f42d40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e309e91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58804/0 (socket says 172.21.15.5:58804) 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e7fff640 1 -- 172.21.15.5:0/342105266 learned_addr learned my addr 172.21.15.5:0/342105266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309f66e0 0x556e309f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309e9ac0 0x556e309e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42ecf9e640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42d40a3e20 0x7f42d40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 -- 172.21.15.5:0/342105266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309e9ac0 msgr2=0x556e309e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309e9ac0 0x556e309e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 -- 172.21.15.5:0/342105266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 msgr2=0x556e309e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e309e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e57fa640 1 -- 172.21.15.5:0/4170622825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f42e00f26f0 con 0x7f42e819ce80 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 -- 172.21.15.5:0/342105266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e3090c320 con 0x556e309f66e0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42e77fe640 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309f66e0 0x556e309f8ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42e80fde20 tx=0x7f42d000b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92e65acc573b626 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42ecf9e640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42d40a3e20 0x7f42d40a62e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f42e00efe00 tx=0x7f42e0098040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42de7fc640 1 -- 172.21.15.5:0/342105266 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004030 con 0x556e309f66e0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.576+0000 7f42de7fc640 1 -- 172.21.15.5:0/342105266 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d0005020 con 0x556e309f66e0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42de7fc640 1 -- 172.21.15.5:0/342105266 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004c50 con 0x556e309f66e0 2024-04-22T13:27:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 -- 172.21.15.5:0/342105266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309f66e0 msgr2=0x556e309f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309f66e0 0x556e309f8ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42e80fde20 tx=0x7f42d000b5b0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 -- 172.21.15.5:0/342105266 shutdown_connections 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e309e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309f66e0 0x556e309f8ad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 --2- 172.21.15.5:0/342105266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309e9ac0 0x556e309e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 -- 172.21.15.5:0/342105266 >> 172.21.15.5:0/342105266 conn(0x556e309afc90 msgr2=0x556e309e3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 -- 172.21.15.5:0/342105266 shutdown_connections 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.577+0000 7f42f15f1740 1 -- 172.21.15.5:0/342105266 wait complete. 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 -- start start 2024-04-22T13:27:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 Processor -- start 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e309f9410 con 0x556e309e9ac0 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556e308d5eb0 con 0x556e309e8d50 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e309f9590 con 0x556e309f66e0 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42f15f1740 10 client.0 _ll_register_callbacks cb 0x556e30902b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58818/0 (socket says 172.21.15.5:58818) 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42e7fff640 1 -- 172.21.15.5:0/492497252 learned_addr learned my addr 172.21.15.5:0/492497252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.578+0000 7f42e77fe640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e77fe640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e7fff640 1 -- 172.21.15.5:0/492497252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 msgr2=0x556e309f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e7fff640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e7fff640 1 -- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 msgr2=0x556e309f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e7fff640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e7fff640 1 -- 172.21.15.5:0/492497252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e30660210 con 0x556e309e9ac0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e77fe640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42e7fff640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f3420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f42d8012840 tx=0x7f42d8012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24372b1432fe583a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42c77fe640 1 -- 172.21.15.5:0/492497252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8004050 con 0x556e309e9ac0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42c77fe640 1 -- 172.21.15.5:0/492497252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f42d80041f0 con 0x556e309e9ac0 2024-04-22T13:27:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d0005020 con 0x556e309e9ac0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42c77fe640 1 -- 172.21.15.5:0/492497252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42d80043f0 con 0x556e309e9ac0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42f15f1740 10 client.15858 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.579+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e3072c620 con 0x556e309e9ac0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42c77fe640 1 -- 172.21.15.5:0/492497252 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d0005020 con 0x556e309e9ac0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42c77fe640 1 -- 172.21.15.5:0/492497252 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f42d8003ab0 con 0x556e309e9ac0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42f15f1740 10 client.15858 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42f15f1740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42f15f1740 20 client.15858 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42f15f1740 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a06730 0x556e30a08b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42f15f1740 4 client.15858 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e30961880 con 0x556e30a06730 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42e77fe640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a06730 0x556e30a08b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42c77fe640 10 client.15858 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.580+0000 7f42e77fe640 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a06730 0x556e30a08b60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=558a3d357749704e server_cookie=52e9fd4db4437e4 in_seq=0 out_seq=0 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42c77fe640 1 -- 172.21.15.5:0/492497252 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e30961880 con 0x556e30a06730 2024-04-22T13:27:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42c77fe640 10 client.15858 handle_command_reply: tid=0 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 client.15858 shutdown 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 msgr2=0x556e309f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f3420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f42d8012840 tx=0x7f42d8012d00 comp rx=0 tx=0).stop 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 shutdown_connections 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a06730 0x556e30a08b60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f3420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 --2- 172.21.15.5:0/492497252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42c77fe640 0 client.15858 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.581+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 >> 172.21.15.5:0/492497252 conn(0x556e309afc90 msgr2=0x556e309e5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.582+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 shutdown_connections 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.582+0000 7f42f15f1740 1 -- 172.21.15.5:0/492497252 wait complete. 2024-04-22T13:27:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.582+0000 7f42f15f1740 20 client.15858 trim_cache size 0 max 16384 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 Processor -- start 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 -- start start 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e30a02340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e9ac0 0x556e309f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ab820 con 0x556e309e8d50 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ad360 con 0x556e309e9ac0 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42f15f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81abef0 con 0x556e309f66e0 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e30a02340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e30a02340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58830/0 (socket says 172.21.15.5:58830) 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 -- 172.21.15.5:0/523978580 learned_addr learned my addr 172.21.15.5:0/523978580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 -- 172.21.15.5:0/523978580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 msgr2=0x556e309f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 -- 172.21.15.5:0/523978580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e9ac0 msgr2=0x556e309f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e9ac0 0x556e309f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 -- 172.21.15.5:0/523978580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e309df470 con 0x556e309e8d50 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.597+0000 7f42e77fe640 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e30a02340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x556e309e97f0 tx=0x7f42d001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a028e86b3ac4a167 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42ddffb640 1 -- 172.21.15.5:0/523978580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0030070 con 0x556e309e8d50 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42ddffb640 1 -- 172.21.15.5:0/523978580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d002b810 con 0x556e309e8d50 2024-04-22T13:27:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42ddffb640 1 -- 172.21.15.5:0/523978580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004bb0 con 0x556e309e8d50 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 -- 172.21.15.5:0/523978580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 msgr2=0x556e30a02340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e30a02340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x556e309e97f0 tx=0x7f42d001b040 comp rx=0 tx=0).stop 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 -- 172.21.15.5:0/523978580 shutdown_connections 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e9ac0 0x556e309f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 --2- 172.21.15.5:0/523978580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e8d50 0x556e30a02340 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 -- 172.21.15.5:0/523978580 >> 172.21.15.5:0/523978580 conn(0x556e309afc90 msgr2=0x556e309e3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 -- 172.21.15.5:0/523978580 shutdown_connections 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.598+0000 7f42f15f1740 1 -- 172.21.15.5:0/523978580 wait complete. 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 -- start start 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 Processor -- start 2024-04-22T13:27:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81acc90 con 0x556e309e9ac0 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ac5c0 con 0x556e309e8d50 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81a9820 con 0x556e309f66e0 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42f15f1740 10 client.0 _ll_register_callbacks cb 0x7f42d8009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58836/0 (socket says 172.21.15.5:58836) 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42e7fff640 1 -- 172.21.15.5:0/3144381706 learned_addr learned my addr 172.21.15.5:0/3144381706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.599+0000 7f42e77fe640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e77fe640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e7fff640 1 -- 172.21.15.5:0/3144381706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 msgr2=0x556e309f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e7fff640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e7fff640 1 -- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 msgr2=0x556e309febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e7fff640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e7fff640 1 -- 172.21.15.5:0/3144381706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e309f7e50 con 0x556e309e9ac0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e77fe640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42e7fff640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f70e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f42d8001eb0 tx=0x7f42d8012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff91fb3d93c73236 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42c77fe640 1 -- 172.21.15.5:0/3144381706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8025750 con 0x556e309e9ac0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42c77fe640 1 -- 172.21.15.5:0/3144381706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f42d8003fd0 con 0x556e309e9ac0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d002b810 con 0x556e309e9ac0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42c77fe640 1 -- 172.21.15.5:0/3144381706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42d80041d0 con 0x556e309e9ac0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42f15f1740 10 client.15864 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e30a6cce0 con 0x556e309e9ac0 2024-04-22T13:27:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.600+0000 7f42c77fe640 1 -- 172.21.15.5:0/3144381706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d002b810 con 0x556e309e9ac0 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42c77fe640 1 -- 172.21.15.5:0/3144381706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f42d80035f0 con 0x556e309e9ac0 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42f15f1740 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42f15f1740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42f15f1740 20 client.15864 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42f15f1740 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a6d400 0x556e30a6f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42f15f1740 4 client.15864 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e308ed3d0 con 0x556e30a6d400 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42e77fe640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a6d400 0x556e30a6f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42c77fe640 10 client.15864 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.601+0000 7f42e77fe640 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a6d400 0x556e30a6f820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8fa336400c90e1 server_cookie=9d15ff0fa9e3574d in_seq=0 out_seq=0 2024-04-22T13:27:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42c77fe640 1 -- 172.21.15.5:0/3144381706 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e308ed3d0 con 0x556e30a6d400 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42c77fe640 10 client.15864 handle_command_reply: tid=0 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 client.15864 shutdown 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 msgr2=0x556e309f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f70e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f42d8001eb0 tx=0x7f42d8012320 comp rx=0 tx=0).stop 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 shutdown_connections 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556e30a6d400 0x556e30a6f820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e309e9ac0 0x556e309f70e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e309f66e0 0x556e309f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 --2- 172.21.15.5:0/3144381706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556e309e8d50 0x556e309febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42c77fe640 0 client.15864 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 >> 172.21.15.5:0/3144381706 conn(0x556e309afc90 msgr2=0x556e309f18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 shutdown_connections 2024-04-22T13:27:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 1 -- 172.21.15.5:0/3144381706 wait complete. 2024-04-22T13:27:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.609+0000 7f42f15f1740 20 client.15864 trim_cache size 0 max 16384 2024-04-22T13:27:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:27:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:27:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:27:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42d40a3e20 msgr2=0x7f42d40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42d40a3e20 0x7f42d40a62e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f42e00efe00 tx=0x7f42e0098040 comp rx=0 tx=0).stop 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 msgr2=0x7f42e819d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 0x7f42e819d330 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005e30 tx=0x7f42e0032420 comp rx=0 tx=0).stop 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 shutdown_connections 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e819ce80 0x7f42e819d330 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f42d40a3e20 0x7f42d40a62e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e81070b0 0x7f42e8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 --2- 172.21.15.5:0/4170622825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42e8101fe0 0x7f42e81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 >> 172.21.15.5:0/4170622825 conn(0x7f42e80fd900 msgr2=0x7f42e80ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 shutdown_connections 2024-04-22T13:27:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:44.610+0000 7f42ed79f640 1 -- 172.21.15.5:0/4170622825 wait complete. 2024-04-22T13:27:45.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:44 smithi049 ceph-mon[31379]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-22T13:27:45.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:44 smithi107 ceph-mon[31859]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-22T13:27:45.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:44 smithi005 ceph-mon[28006]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-22T13:27:46.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:27:46.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:27:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.097+0000 7f6dfb06a640 1 -- 172.21.15.5:0/1304929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4102b30 msgr2=0x7f6df4106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.097+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/1304929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4102b30 0x7f6df4106b70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009900 tx=0x7f6de402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- 172.21.15.5:0/1304929588 shutdown_connections 2024-04-22T13:27:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/1304929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4102b30 0x7f6df4106b70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/1304929588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df4107250 0x7f6df410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/1304929588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 0x7f6df4102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- 172.21.15.5:0/1304929588 >> 172.21.15.5:0/1304929588 conn(0x7f6df40fda00 msgr2=0x7f6df40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- 172.21.15.5:0/1304929588 shutdown_connections 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- 172.21.15.5:0/1304929588 wait complete. 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 Processor -- start 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- start start 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 0x7f6df4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 0x7f6df4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df419cff0 0x7f6df419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4112190 con 0x7f6df4107250 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4112010 con 0x7f6df4102180 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.098+0000 7f6dfb06a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4112310 con 0x7f6df419cff0 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 0x7f6df4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 0x7f6df4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58854/0 (socket says 172.21.15.5:58854) 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 -- 172.21.15.5:0/9761079 learned_addr learned my addr 172.21.15.5:0/9761079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6df8e06640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 0x7f6df4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6df9607640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df419cff0 0x7f6df419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 -- 172.21.15.5:0/9761079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df419cff0 msgr2=0x7f6df419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df419cff0 0x7f6df419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 -- 172.21.15.5:0/9761079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 msgr2=0x7f6df4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 0x7f6df4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 -- 172.21.15.5:0/9761079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df419dae0 con 0x7f6df4107250 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6df9607640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df419cff0 0x7f6df419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6df8e06640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 0x7f6df4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6debfff640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 0x7f6df4198c70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6de4036040 tx=0x7f6de40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6de9ffb640 1 -- 172.21.15.5:0/9761079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4043020 con 0x7f6df4107250 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df419dd70 con 0x7f6df4107250 2024-04-22T13:27:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6de9ffb640 1 -- 172.21.15.5:0/9761079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6de402cb70 con 0x7f6df4107250 2024-04-22T13:27:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.099+0000 7f6de9ffb640 1 -- 172.21.15.5:0/9761079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4003bf0 con 0x7f6df4107250 2024-04-22T13:27:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.100+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41a55c0 con 0x7f6df4107250 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.102+0000 7f6de9ffb640 1 -- 172.21.15.5:0/9761079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f6de4046020 con 0x7f6df4107250 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.102+0000 7f6dfdc5e740 1 Processor -- start 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.102+0000 7f6dfdc5e740 1 -- start start 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.102+0000 7f6de9ffb640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ddc0a3e40 0x7f6ddc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.102+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de1705690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de1712950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1715280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558de166a1e0 con 0x558de1705bd0 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558de1607390 con 0x558de17052b0 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558de16661a0 con 0x558de1712e90 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6df9607640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ddc0a3e40 0x7f6ddc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6de9ffb640 1 -- 172.21.15.5:0/9761079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6de40f2b90 con 0x7f6df4107250 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de1712950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de1712950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58862/0 (socket says 172.21.15.5:58862) 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 -- 172.21.15.5:0/4023582279 learned_addr learned my addr 172.21.15.5:0/4023582279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6df8e06640 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de1705690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6df9607640 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1715280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 -- 172.21.15.5:0/4023582279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 msgr2=0x558de1715280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1715280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 -- 172.21.15.5:0/4023582279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 msgr2=0x558de1705690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de1705690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 -- 172.21.15.5:0/4023582279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de14467d0 con 0x558de1705bd0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6df8e06640 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de1705690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6df9607640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ddc0a3e40 0x7f6ddc0a6300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005fd0 tx=0x7f6dec012b30 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6debfff640 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de1712950 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6de40bae20 tx=0x7f6de40057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471f8781b1780e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6ddaffd640 1 -- 172.21.15.5:0/4023582279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de40bf0c0 con 0x558de1705bd0 2024-04-22T13:27:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.103+0000 7f6ddaffd640 1 -- 172.21.15.5:0/4023582279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6de40be9c0 con 0x558de1705bd0 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6ddaffd640 1 -- 172.21.15.5:0/4023582279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de40bb030 con 0x558de1705bd0 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/4023582279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 msgr2=0x558de1712950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de1712950 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6de40bae20 tx=0x7f6de40057e0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/4023582279 shutdown_connections 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de1712950 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1715280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/4023582279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de1705690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/4023582279 >> 172.21.15.5:0/4023582279 conn(0x558de16ffcb0 msgr2=0x558de1700090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/4023582279 shutdown_connections 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.104+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/4023582279 wait complete. 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 -- start start 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 Processor -- start 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 0x558de170ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de1705bd0 0x558de17104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1710a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558de1715a10 con 0x558de17052b0 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558de16232f0 con 0x558de1705bd0 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558de1715b90 con 0x558de1712e90 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6dfdc5e740 10 client.0 _ll_register_callbacks cb 0x558de1630580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6df8e06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 0x558de170ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6df8e06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 0x558de170ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58868/0 (socket says 172.21.15.5:58868) 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6df8e06640 1 -- 172.21.15.5:0/75602395 learned_addr learned my addr 172.21.15.5:0/75602395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6df9607640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1710a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.105+0000 7f6debfff640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de1705bd0 0x558de17104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6df8e06640 1 -- 172.21.15.5:0/75602395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 msgr2=0x558de1710a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6df8e06640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1710a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6df8e06640 1 -- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de1705bd0 msgr2=0x558de17104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6df8e06640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de1705bd0 0x558de17104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6df8e06640 1 -- 172.21.15.5:0/75602395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de137c3f0 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6df8e06640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 0x558de170ff80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6df0011870 tx=0x7f6df0011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bcb139d73e4de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dd3fff640 1 -- 172.21.15.5:0/75602395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0010ba0 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dd3fff640 1 -- 172.21.15.5:0/75602395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6df0010d40 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de40be9c0 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dd3fff640 1 -- 172.21.15.5:0/75602395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6df0005020 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dfdc5e740 10 client.15876 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558de133c0f0 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.106+0000 7f6dd3fff640 1 -- 172.21.15.5:0/75602395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6de40be9c0 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dd3fff640 1 -- 172.21.15.5:0/75602395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6df001a410 con 0x558de17052b0 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dfdc5e740 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dfdc5e740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dfdc5e740 20 client.15876 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de1722ca0 0x558de17250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dfdc5e740 4 client.15876 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558de1677c00 con 0x558de1722ca0 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6debfff640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de1722ca0 0x558de17250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6dd3fff640 10 client.15876 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.107+0000 7f6debfff640 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de1722ca0 0x558de17250f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9fcb3a2b322b06 server_cookie=8ca8ba650029ec05 in_seq=0 out_seq=0 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dd3fff640 1 -- 172.21.15.5:0/75602395 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558de1677c00 con 0x558de1722ca0 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dd3fff640 10 client.15876 handle_command_reply: tid=0 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 client.15876 shutdown 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 msgr2=0x558de170ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 0x558de170ff80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6df0011870 tx=0x7f6df0011d30 comp rx=0 tx=0).stop 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 shutdown_connections 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de1722ca0 0x558de17250f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de1710a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de1705bd0 0x558de17104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dd3fff640 0 client.15876 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/75602395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de17052b0 0x558de170ff80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.108+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 >> 172.21.15.5:0/75602395 conn(0x558de16ffcb0 msgr2=0x558de1701a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.109+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 shutdown_connections 2024-04-22T13:27:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.109+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/75602395 wait complete. 2024-04-22T13:27:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.109+0000 7f6dfdc5e740 20 client.15876 trim_cache size 0 max 16384 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 Processor -- start 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 -- start start 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de17126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de171e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de170eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ab9a0 con 0x558de1705bd0 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ad4e0 con 0x558de17052b0 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.126+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ac070 con 0x558de1712e90 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de171e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de171e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58870/0 (socket says 172.21.15.5:58870) 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 -- 172.21.15.5:0/595037338 learned_addr learned my addr 172.21.15.5:0/595037338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6df9607640 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de170eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 -- 172.21.15.5:0/595037338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 msgr2=0x558de170eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de170eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 -- 172.21.15.5:0/595037338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 msgr2=0x558de17126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de17126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 -- 172.21.15.5:0/595037338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de1704ee0 con 0x558de1705bd0 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6debfff640 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de171e8d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6de4000c00 tx=0x7f6de40df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a129c76c77229bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6dda7fc640 1 -- 172.21.15.5:0/595037338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de40bf0c0 con 0x558de1705bd0 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6dda7fc640 1 -- 172.21.15.5:0/595037338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6de40c7630 con 0x558de1705bd0 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.127+0000 7f6dda7fc640 1 -- 172.21.15.5:0/595037338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de40bb030 con 0x558de1705bd0 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/595037338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 msgr2=0x558de171e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de171e8d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6de4000c00 tx=0x7f6de40df040 comp rx=0 tx=0).stop 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/595037338 shutdown_connections 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1705bd0 0x558de171e8d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1712e90 0x558de170eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/595037338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de17126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/595037338 >> 172.21.15.5:0/595037338 conn(0x558de16ffcb0 msgr2=0x558de16fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/595037338 shutdown_connections 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/595037338 wait complete. 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 -- start start 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.128+0000 7f6dfdc5e740 1 Processor -- start 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de171ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1705bd0 0x558de171b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 0x558de1717db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ace10 con 0x558de1712e90 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ac740 con 0x558de17052b0 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41a99c0 con 0x558de1705bd0 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dfdc5e740 10 client.0 _ll_register_callbacks cb 0x7f6df000bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 0x558de1717db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 0x558de1717db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58878/0 (socket says 172.21.15.5:58878) 2024-04-22T13:27:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 -- 172.21.15.5:0/1448897997 learned_addr learned my addr 172.21.15.5:0/1448897997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6debfff640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1705bd0 0x558de171b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df8e06640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de171ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 -- 172.21.15.5:0/1448897997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1705bd0 msgr2=0x558de171b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1705bd0 0x558de171b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 -- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 msgr2=0x558de171ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de171ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 -- 172.21.15.5:0/1448897997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de1718340 con 0x558de1712e90 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6debfff640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1705bd0 0x558de171b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6df9607640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 0x558de1717db0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6dec020640 tx=0x7f6dec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83402673c43a8a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.129+0000 7f6dd3fff640 1 -- 172.21.15.5:0/1448897997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec02e060 con 0x558de1712e90 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de40c7630 con 0x558de1712e90 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dd3fff640 1 -- 172.21.15.5:0/1448897997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6dec00f690 con 0x558de1712e90 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dd3fff640 1 -- 172.21.15.5:0/1448897997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6dec00f890 con 0x558de1712e90 2024-04-22T13:27:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 10 client.15882 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558de1789b20 con 0x558de1712e90 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dd3fff640 1 -- 172.21.15.5:0/1448897997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6de40c7630 con 0x558de1712e90 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dd3fff640 1 -- 172.21.15.5:0/1448897997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6dec028070 con 0x558de1712e90 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 20 client.15882 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de178a250 0x558de178c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 4 client.15882 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558de1671a70 con 0x558de178a250 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.130+0000 7f6df8e06640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de178a250 0x558de178c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6df8e06640 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de178a250 0x558de178c670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282272083a4fbac7 server_cookie=73401b8c1ea258ee in_seq=0 out_seq=0 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dd3fff640 10 client.15882 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dd3fff640 1 -- 172.21.15.5:0/1448897997 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558de1671a70 con 0x558de178a250 2024-04-22T13:27:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dd3fff640 10 client.15882 handle_command_reply: tid=0 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 client.15882 shutdown 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 msgr2=0x558de1717db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 0x558de1717db0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6dec020640 tx=0x7f6dec01a000 comp rx=0 tx=0).stop 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 shutdown_connections 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558de178a250 0x558de178c670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558de1712e90 0x558de1717db0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dd3fff640 0 client.15882 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558de1705bd0 0x558de171b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 --2- 172.21.15.5:0/1448897997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558de17052b0 0x558de171ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 >> 172.21.15.5:0/1448897997 conn(0x558de16ffcb0 msgr2=0x558de170ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 shutdown_connections 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 1 -- 172.21.15.5:0/1448897997 wait complete. 2024-04-22T13:27:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.131+0000 7f6dfdc5e740 20 client.15882 trim_cache size 0 max 16384 2024-04-22T13:27:47.135 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:27:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:27:47.135 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ddc0a3e40 msgr2=0x7f6ddc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ddc0a3e40 0x7f6ddc0a6300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005fd0 tx=0x7f6dec012b30 comp rx=0 tx=0).stop 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 msgr2=0x7f6df4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 0x7f6df4198c70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6de4036040 tx=0x7f6de40045e0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 shutdown_connections 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df4107250 0x7f6df4198c70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ddc0a3e40 0x7f6ddc0a6300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6df419cff0 0x7f6df419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 --2- 172.21.15.5:0/9761079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6df4102180 0x7f6df4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 >> 172.21.15.5:0/9761079 conn(0x7f6df40fda00 msgr2=0x7f6df40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 shutdown_connections 2024-04-22T13:27:47.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.132+0000 7f6dfb06a640 1 -- 172.21.15.5:0/9761079 wait complete. 2024-04-22T13:27:47.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:27:47.140 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 -- 172.21.15.5:0/2649943107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8105dd0 msgr2=0x7f20f81061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 --2- 172.21.15.5:0/2649943107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8105dd0 0x7f20f81061b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009900 tx=0x7f20e802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 -- 172.21.15.5:0/2649943107 shutdown_connections 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 --2- 172.21.15.5:0/2649943107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f8101a40 0x7f20f810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 --2- 172.21.15.5:0/2649943107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f81010a0 0x7f20f8101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 --2- 172.21.15.5:0/2649943107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8105dd0 0x7f20f81061b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 -- 172.21.15.5:0/2649943107 >> 172.21.15.5:0/2649943107 conn(0x7f20f8077c40 msgr2=0x7f20f8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 -- 172.21.15.5:0/2649943107 shutdown_connections 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.246+0000 7f20fd831640 1 -- 172.21.15.5:0/2649943107 wait complete. 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 Processor -- start 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 -- start start 2024-04-22T13:27:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f81010a0 0x7f20f819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 0x7f20f819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8105dd0 0x7f20f8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8111150 con 0x7f20f8101a40 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8110fd0 con 0x7f20f81010a0 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20fd831640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20f81112d0 con 0x7f20f8105dd0 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 0x7f20f819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 0x7f20f819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58894/0 (socket says 172.21.15.5:58894) 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 -- 172.21.15.5:0/911181151 learned_addr learned my addr 172.21.15.5:0/911181151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20f6ffd640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f81010a0 0x7f20f819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20f77fe640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8105dd0 0x7f20f8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 -- 172.21.15.5:0/911181151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8105dd0 msgr2=0x7f20f8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8105dd0 0x7f20f8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 -- 172.21.15.5:0/911181151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f81010a0 msgr2=0x7f20f819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.247+0000 7f20ee7fc640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f81010a0 0x7f20f819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20ee7fc640 1 -- 172.21.15.5:0/911181151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8198d10 con 0x7f20f8101a40 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20f6ffd640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f81010a0 0x7f20f819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20ee7fc640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 0x7f20f819e920 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d950 tx=0x7f20e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0017070 con 0x7f20f8101a40 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0009c30 con 0x7f20f8101a40 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f8199000 con 0x7f20f8101a40 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000f620 con 0x7f20f8101a40 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.248+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81a52a0 con 0x7f20f8101a40 2024-04-22T13:27:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.249+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f8102540 con 0x7f20f8101a40 2024-04-22T13:27:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.250+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f20e0005030 con 0x7f20f8101a40 2024-04-22T13:27:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.251+0000 7f20fc82f640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f20e40a3e40 0x7f20e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.251+0000 7f20f6ffd640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f20e40a3e40 0x7f20e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.251+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f20e00d07d0 con 0x7f20f8101a40 2024-04-22T13:27:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.251+0000 7f20f6ffd640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f20e40a3e40 0x7f20e40a6300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005ab0 tx=0x7f20e8005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.255+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20e009cbf0 con 0x7f20f8101a40 2024-04-22T13:27:47.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:46 smithi049 ceph-mon[31379]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-04-22T13:27:47.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:46 smithi107 ceph-mon[31859]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-04-22T13:27:47.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:46 smithi005 ceph-mon[28006]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-04-22T13:27:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.481+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20f810a370 con 0x7f20f8101a40 2024-04-22T13:27:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.482+0000 7f20fc82f640 1 -- 172.21.15.5:0/911181151 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f20e0009de0 con 0x7f20f8101a40 2024-04-22T13:27:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:47.479 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f20e40a3e40 msgr2=0x7f20e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f20e40a3e40 0x7f20e40a6300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005ab0 tx=0x7f20e8005a20 comp rx=0 tx=0).stop 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 msgr2=0x7f20f819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 0x7f20f819e920 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d950 tx=0x7f20e000de10 comp rx=0 tx=0).stop 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 shutdown_connections 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20f8101a40 0x7f20f819e920 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f20e40a3e40 0x7f20e40a6300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f8105dd0 0x7f20f8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 --2- 172.21.15.5:0/911181151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20f81010a0 0x7f20f819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 >> 172.21.15.5:0/911181151 conn(0x7f20f8077c40 msgr2=0x7f20f806a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 shutdown_connections 2024-04-22T13:27:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.484+0000 7f20fd831640 1 -- 172.21.15.5:0/911181151 wait complete. 2024-04-22T13:27:47.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:27:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 -- 172.21.15.5:0/4288767479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70103400 msgr2=0x7fcd7010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 --2- 172.21.15.5:0/4288767479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70103400 0x7fcd7010de00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009cd0 tx=0x7fcd6802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 -- 172.21.15.5:0/4288767479 shutdown_connections 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 --2- 172.21.15.5:0/4288767479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70103400 0x7fcd7010de00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 --2- 172.21.15.5:0/4288767479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70102a60 0x7fcd70102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 --2- 172.21.15.5:0/4288767479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd70101180 0x7fcd70101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 -- 172.21.15.5:0/4288767479 >> 172.21.15.5:0/4288767479 conn(0x7fcd70077f70 msgr2=0x7fcd70078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 -- 172.21.15.5:0/4288767479 shutdown_connections 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.599+0000 7fcd76811640 1 -- 172.21.15.5:0/4288767479 wait complete. 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 Processor -- start 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 -- start start 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 0x7fcd700fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd70102a60 0x7fcd700fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70103400 0x7fcd700fe870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701111a0 con 0x7fcd70101180 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111020 con 0x7fcd70102a60 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd76811640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111320 con 0x7fcd70103400 2024-04-22T13:27:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 0x7fcd700fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 0x7fcd700fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58914/0 (socket says 172.21.15.5:58914) 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1074038501 learned_addr learned my addr 172.21.15.5:0/1074038501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd70102a60 0x7fcd700fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd74dae640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70103400 0x7fcd700fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1074038501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70103400 msgr2=0x7fcd700fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70103400 0x7fcd700fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1074038501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd70102a60 msgr2=0x7fcd700fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.600+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd70102a60 0x7fcd700fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1074038501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd701a6c80 con 0x7fcd70101180 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd74dae640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70103400 0x7fcd700fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 0x7fcd700fddf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400d950 tx=0x7fcd6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd6d7fa640 1 -- 172.21.15.5:0/1074038501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64017070 con 0x7fcd70101180 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd701a6f70 con 0x7fcd70101180 2024-04-22T13:27:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd6d7fa640 1 -- 172.21.15.5:0/1074038501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd64009c30 con 0x7fcd70101180 2024-04-22T13:27:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd6d7fa640 1 -- 172.21.15.5:0/1074038501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400f620 con 0x7fcd70101180 2024-04-22T13:27:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.601+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701a7480 con 0x7fcd70101180 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.603+0000 7fcd6d7fa640 1 -- 172.21.15.5:0/1074038501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fcd64010460 con 0x7fcd70101180 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.603+0000 7fcd79404740 1 Processor -- start 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.603+0000 7fcd79404740 1 -- start start 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd79404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d04d0 0x55d8549d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd79404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd79404740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd79404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d8549351f0 con 0x55d8549d0e80 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd79404740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d8548d24b0 con 0x55d8549d9a00 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd79404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8549311b0 con 0x55d8549d04d0 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58916/0 (socket says 172.21.15.5:58916) 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 -- 172.21.15.5:0/762849000 learned_addr learned my addr 172.21.15.5:0/762849000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 -- 172.21.15.5:0/762849000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d04d0 msgr2=0x55d8549d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d04d0 0x55d8549d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 -- 172.21.15.5:0/762849000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 msgr2=0x55d8549dbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549dbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6d7fa640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcd580a3e40 0x7fcd580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 -- 172.21.15.5:0/762849000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8548c41b0 con 0x55d8549d0e80 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd74dae640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcd580a3e40 0x7fcd580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6d7fa640 1 -- 172.21.15.5:0/1074038501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fcd640d06a0 con 0x7fcd70101180 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d9430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd640cddb0 tx=0x7fcd64098780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6697483411af2c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.604+0000 7fcd567fc640 1 -- 172.21.15.5:0/762849000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64020070 con 0x55d8549d0e80 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd567fc640 1 -- 172.21.15.5:0/762849000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6409cb20 con 0x55d8549d0e80 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd567fc640 1 -- 172.21.15.5:0/762849000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6409bf60 con 0x55d8549d0e80 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 -- 172.21.15.5:0/762849000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 msgr2=0x55d8549d9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd74dae640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcd580a3e40 0x7fcd580a6300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006010 tx=0x7fcd68005d80 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d9430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd640cddb0 tx=0x7fcd64098780 comp rx=0 tx=0).stop 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 -- 172.21.15.5:0/762849000 shutdown_connections 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d9430 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549dbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 --2- 172.21.15.5:0/762849000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d04d0 0x55d8549d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 -- 172.21.15.5:0/762849000 >> 172.21.15.5:0/762849000 conn(0x55d8549caba0 msgr2=0x55d8549caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 -- 172.21.15.5:0/762849000 shutdown_connections 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.605+0000 7fcd79404740 1 -- 172.21.15.5:0/762849000 wait complete. 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 -- start start 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 Processor -- start 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549d9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d8549dc8b0 con 0x55d8549d0e80 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d8548ee500 con 0x55d8549d04d0 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8549dca30 con 0x55d8549d9a00 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd79404740 10 client.0 _ll_register_callbacks cb 0x55d8548bcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58920/0 (socket says 172.21.15.5:58920) 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/4123057502 learned_addr learned my addr 172.21.15.5:0/4123057502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd74dae640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549d9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 msgr2=0x55d8549d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549d6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/4123057502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 msgr2=0x55d8549d9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549d9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/4123057502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8546473f0 con 0x55d8549d0e80 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549d9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.606+0000 7fcd74dae640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d66b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcd60011690 tx=0x7fcd60011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6019eea18c9d4a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/4123057502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6001f060 con 0x55d8549d0e80 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/4123057502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fcd6000fb40 con 0x55d8549d0e80 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6409cb20 con 0x55d8549d0e80 2024-04-22T13:27:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/4123057502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd6000fd40 con 0x55d8549d0e80 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8547117d0 con 0x55d8549d0e80 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/4123057502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6409cb20 con 0x55d8549d0e80 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/4123057502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcd600124e0 con 0x55d8549d0e80 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 10 client.15900 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 20 client.15900 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d8549e9be0 0x55d8549ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 4 client.15900 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.607+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8548d4930 con 0x55d8549e9be0 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.608+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d8549e9be0 0x55d8549ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.608+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d8549e9be0 0x55d8549ec020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f73b0126a8a91a02 server_cookie=5d403cd9d3b68b44 in_seq=0 out_seq=0 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.608+0000 7fcd4f7fe640 10 client.15900 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/4123057502 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8548d4930 con 0x55d8549e9be0 2024-04-22T13:27:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd4f7fe640 10 client.15900 handle_command_reply: tid=0 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 client.15900 shutdown 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 msgr2=0x55d8549d66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d66b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcd60011690 tx=0x7fcd60011b50 comp rx=0 tx=0).stop 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 shutdown_connections 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d8549e9be0 0x55d8549ec020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549d66b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd4f7fe640 0 client.15900 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 --2- 172.21.15.5:0/4123057502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549d9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 >> 172.21.15.5:0/4123057502 conn(0x55d8549caba0 msgr2=0x55d8549cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 shutdown_connections 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 1 -- 172.21.15.5:0/4123057502 wait complete. 2024-04-22T13:27:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.609+0000 7fcd79404740 20 client.15900 trim_cache size 0 max 16384 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.622+0000 7fcd79404740 1 Processor -- start 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 -- start start 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 0x55d8549e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d0e80 0x55d8549d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701afdf0 con 0x55d8549d04d0 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701b1930 con 0x55d8549d9a00 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd79404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701b04c0 con 0x55d8549d0e80 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 0x55d8549e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 0x55d8549e57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58924/0 (socket says 172.21.15.5:58924) 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1918302098 learned_addr learned my addr 172.21.15.5:0/1918302098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d0e80 0x55d8549d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1918302098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d0e80 msgr2=0x55d8549d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d0e80 0x55d8549d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1918302098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 msgr2=0x55d8549d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 -- 172.21.15.5:0/1918302098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8549d4c00 con 0x55d8549d04d0 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d0e80 0x55d8549d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 0x55d8549e57a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcd640ce420 tx=0x7fcd64098050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f55ce0c7842b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd55ffb640 1 -- 172.21.15.5:0/1918302098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64020040 con 0x55d8549d04d0 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd55ffb640 1 -- 172.21.15.5:0/1918302098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd64009de0 con 0x55d8549d04d0 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.623+0000 7fcd55ffb640 1 -- 172.21.15.5:0/1918302098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6409bd80 con 0x55d8549d04d0 2024-04-22T13:27:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 -- 172.21.15.5:0/1918302098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 msgr2=0x55d8549e57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 0x55d8549e57a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcd640ce420 tx=0x7fcd64098050 comp rx=0 tx=0).stop 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 -- 172.21.15.5:0/1918302098 shutdown_connections 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d9a00 0x55d8549d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d0e80 0x55d8549d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 --2- 172.21.15.5:0/1918302098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d04d0 0x55d8549e57a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 -- 172.21.15.5:0/1918302098 >> 172.21.15.5:0/1918302098 conn(0x55d8549caba0 msgr2=0x55d8549d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 -- 172.21.15.5:0/1918302098 shutdown_connections 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 -- 172.21.15.5:0/1918302098 wait complete. 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.624+0000 7fcd79404740 1 -- start start 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 Processor -- start 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701b1260 con 0x55d8549d0e80 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701b0b90 con 0x55d8549d04d0 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701addf0 con 0x55d8549d9a00 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd79404740 10 client.0 _ll_register_callbacks cb 0x7fcd6000ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549dd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58926/0 (socket says 172.21.15.5:58926) 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/1154188614 learned_addr learned my addr 172.21.15.5:0/1154188614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd74dae640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 msgr2=0x55d8549ddcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549ddcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/1154188614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 msgr2=0x55d8549e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 -- 172.21.15.5:0/1154188614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d854a4f160 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd6f7fe640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549dd5b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000c00 tx=0x7fcd6000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ce9cd627fc46f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.625+0000 7fcd4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/1154188614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000f730 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/1154188614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fcd60005040 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd64009de0 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/1154188614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd6001f030 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 10 client.15906 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d854a502c0 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/1154188614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd64009de0 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/1154188614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcd60004a40 con 0x55d8549d0e80 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 10 client.15906 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 10 client.15906 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:27:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 20 client.15906 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d854a50a90 0x55d854a52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 4 client.15906 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.626+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d854942c10 con 0x55d854a50a90 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d854a50a90 0x55d854a52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd6ffff640 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d854a50a90 0x55d854a52eb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d7543db85a15d1 server_cookie=3c1163e343250b2d in_seq=0 out_seq=0 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd4f7fe640 10 client.15906 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd4f7fe640 1 -- 172.21.15.5:0/1154188614 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d854942c10 con 0x55d854a50a90 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd4f7fe640 10 client.15906 handle_command_reply: tid=0 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 client.15906 shutdown 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 msgr2=0x55d8549dd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549dd5b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000c00 tx=0x7fcd6000fbd0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 shutdown_connections 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55d854a50a90 0x55d854a52eb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8549d0e80 0x55d8549dd5b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd4f7fe640 0 client.15906 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8549d9a00 0x55d8549ddcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 --2- 172.21.15.5:0/1154188614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d8549d04d0 0x55d8549e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 >> 172.21.15.5:0/1154188614 conn(0x55d8549caba0 msgr2=0x55d8549cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 shutdown_connections 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 1 -- 172.21.15.5:0/1154188614 wait complete. 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.627+0000 7fcd79404740 20 client.15906 trim_cache size 0 max 16384 2024-04-22T13:27:47.631 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcd580a3e40 msgr2=0x7fcd580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcd580a3e40 0x7fcd580a6300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006010 tx=0x7fcd68005d80 comp rx=0 tx=0).stop 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 msgr2=0x7fcd700fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 0x7fcd700fddf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400d950 tx=0x7fcd6400de10 comp rx=0 tx=0).stop 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 shutdown_connections 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcd580a3e40 0x7fcd580a6300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70103400 0x7fcd700fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd70102a60 0x7fcd700fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 --2- 172.21.15.5:0/1074038501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd70101180 0x7fcd700fddf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 >> 172.21.15.5:0/1074038501 conn(0x7fcd70077f70 msgr2=0x7fcd7006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 shutdown_connections 2024-04-22T13:27:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.628+0000 7fcd76811640 1 -- 172.21.15.5:0/1074038501 wait complete. 2024-04-22T13:27:47.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:27:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 -- 172.21.15.5:0/2898093398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 msgr2=0x7fe954102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 --2- 172.21.15.5:0/2898093398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe954102ec0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe948009930 tx=0x7fe94802bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 -- 172.21.15.5:0/2898093398 shutdown_connections 2024-04-22T13:27:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 --2- 172.21.15.5:0/2898093398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe954102ec0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 --2- 172.21.15.5:0/2898093398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe954103400 0x7fe95410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 --2- 172.21.15.5:0/2898093398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe954101180 0x7fe954101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 -- 172.21.15.5:0/2898093398 >> 172.21.15.5:0/2898093398 conn(0x7fe954077f70 msgr2=0x7fe954078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 -- 172.21.15.5:0/2898093398 shutdown_connections 2024-04-22T13:27:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.743+0000 7fe958f46640 1 -- 172.21.15.5:0/2898093398 wait complete. 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 Processor -- start 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 -- start start 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe954101180 0x7fe95419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe95419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe954103400 0x7fe954198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541111a0 con 0x7fe954102a60 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe954111020 con 0x7fe954103400 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe958f46640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe954111320 con 0x7fe954101180 2024-04-22T13:27:47.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe95419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe95419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58954/0 (socket says 172.21.15.5:58954) 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 -- 172.21.15.5:0/983228423 learned_addr learned my addr 172.21.15.5:0/983228423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe95259c640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe954101180 0x7fe95419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 -- 172.21.15.5:0/983228423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe954101180 msgr2=0x7fe95419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe952d9d640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe954103400 0x7fe954198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe954101180 0x7fe95419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 -- 172.21.15.5:0/983228423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe954103400 msgr2=0x7fe954198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe954103400 0x7fe954198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.744+0000 7fe951d9b640 1 -- 172.21.15.5:0/983228423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe954198dd0 con 0x7fe954102a60 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.745+0000 7fe951d9b640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe95419e990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe948009230 tx=0x7fe9480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.745+0000 7fe9437fe640 1 -- 172.21.15.5:0/983228423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948043020 con 0x7fe954102a60 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.745+0000 7fe9437fe640 1 -- 172.21.15.5:0/983228423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94802cb70 con 0x7fe954102a60 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.745+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe954199060 con 0x7fe954102a60 2024-04-22T13:27:47.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.745+0000 7fe9437fe640 1 -- 172.21.15.5:0/983228423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948003b50 con 0x7fe954102a60 2024-04-22T13:27:47.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.745+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541a5310 con 0x7fe954102a60 2024-04-22T13:27:47.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe9437fe640 1 -- 172.21.15.5:0/983228423 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fe948046020 con 0x7fe954102a60 2024-04-22T13:27:47.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 Processor -- start 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 -- start start 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce152430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5646ce0ae1f0 con 0x5646ce1494d0 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5646ce04b4b0 con 0x5646ce152a00 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5646ce0aa1b0 con 0x5646ce149e80 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce1498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58968/0 (socket says 172.21.15.5:58968) 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe952d9d640 1 -- 172.21.15.5:0/3234852297 learned_addr learned my addr 172.21.15.5:0/3234852297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.748+0000 7fe951d9b640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe95259c640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe9437fe640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe93c0a3e40 0x7fe93c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 -- 172.21.15.5:0/3234852297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 msgr2=0x5646ce152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 -- 172.21.15.5:0/3234852297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 msgr2=0x5646ce154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 -- 172.21.15.5:0/3234852297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ce03d1b0 con 0x5646ce1494d0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe951d9b640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe95259c640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe9437fe640 1 -- 172.21.15.5:0/983228423 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe9480f2c40 con 0x7fe954102a60 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce1498b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009e70 tx=0x7fe94c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528ca7a5d673c342 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe952d9d640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe93c0a3e40 0x7fe93c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe940ff9640 1 -- 172.21.15.5:0/3234852297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c016070 con 0x5646ce1494d0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe940ff9640 1 -- 172.21.15.5:0/3234852297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c011de0 con 0x5646ce1494d0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.749+0000 7fe940ff9640 1 -- 172.21.15.5:0/3234852297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0105d0 con 0x5646ce1494d0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3234852297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 msgr2=0x5646ce1498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe952d9d640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe93c0a3e40 0x7fe93c0a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00f010 tx=0x7fe94c010cd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce1498b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009e70 tx=0x7fe94c00e6c0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3234852297 shutdown_connections 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3234852297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce1498b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3234852297 >> 172.21.15.5:0/3234852297 conn(0x5646ce143ba0 msgr2=0x5646ce143f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3234852297 shutdown_connections 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3234852297 wait complete. 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 -- start start 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.750+0000 7fe95bc0b740 1 Processor -- start 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce1494d0 0x5646ce1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 0x5646ce14f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce152a00 0x5646ce14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5646ce155970 con 0x5646ce149e80 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5646ce067500 con 0x5646ce1494d0 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5646ce155af0 con 0x5646ce152a00 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 10 client.0 _ll_register_callbacks cb 0x5646ce035b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 0x5646ce14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 0x5646ce14f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58970/0 (socket says 172.21.15.5:58970) 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 -- 172.21.15.5:0/3327641462 learned_addr learned my addr 172.21.15.5:0/3327641462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95259c640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce1494d0 0x5646ce1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 -- 172.21.15.5:0/3327641462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce152a00 msgr2=0x5646ce14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce152a00 0x5646ce14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce152a00 0x5646ce14fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 -- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce1494d0 msgr2=0x5646ce1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce1494d0 0x5646ce1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 -- 172.21.15.5:0/3327641462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cddc03f0 con 0x5646ce149e80 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95259c640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce1494d0 0x5646ce1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe951d9b640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 0x5646ce14f650 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe948000c00 tx=0x7fe9480035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2c9e95580fcd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe939ffb640 1 -- 172.21.15.5:0/3327641462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94803c350 con 0x5646ce149e80 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe939ffb640 1 -- 172.21.15.5:0/3327641462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe9480bcdd0 con 0x5646ce149e80 2024-04-22T13:27:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.751+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe94c011de0 con 0x5646ce149e80 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe939ffb640 1 -- 172.21.15.5:0/3327641462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9480bf1d0 con 0x5646ce149e80 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 10 client.15918 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646cde8a7d0 con 0x5646ce149e80 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe939ffb640 1 -- 172.21.15.5:0/3327641462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c011de0 con 0x5646ce149e80 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe939ffb640 1 -- 172.21.15.5:0/3327641462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9480c72b0 con 0x5646ce149e80 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 10 client.15918 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 20 client.15918 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce162b00 0x5646ce164f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 4 client.15918 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646ce04d930 con 0x5646ce162b00 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.752+0000 7fe95259c640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce162b00 0x5646ce164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.753+0000 7fe939ffb640 10 client.15918 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.753+0000 7fe95259c640 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce162b00 0x5646ce164f30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66b83654f50ee52e server_cookie=2cd3a634c33b99f4 in_seq=0 out_seq=0 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.753+0000 7fe939ffb640 1 -- 172.21.15.5:0/3327641462 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646ce04d930 con 0x5646ce162b00 2024-04-22T13:27:47.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.753+0000 7fe939ffb640 10 client.15918 handle_command_reply: tid=0 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 client.15918 shutdown 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 msgr2=0x5646ce14f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 0x5646ce14f650 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe948000c00 tx=0x7fe9480035f0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 shutdown_connections 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce162b00 0x5646ce164f30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce149e80 0x5646ce14f650 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce152a00 0x5646ce14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe939ffb640 0 client.15918 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3327641462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce1494d0 0x5646ce1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 >> 172.21.15.5:0/3327641462 conn(0x5646ce143ba0 msgr2=0x5646ce1458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 shutdown_connections 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3327641462 wait complete. 2024-04-22T13:27:47.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.754+0000 7fe95bc0b740 20 client.15918 trim_cache size 0 max 16384 2024-04-22T13:27:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 Processor -- start 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 -- start start 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce1494d0 0x5646ce142370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce149e80 0x5646ce151d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 0x5646ce1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541ada30 con 0x5646ce152a00 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541af570 con 0x5646ce149e80 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541ae100 con 0x5646ce1494d0 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 0x5646ce1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 0x5646ce1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58980/0 (socket says 172.21.15.5:58980) 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.767+0000 7fe952d9d640 1 -- 172.21.15.5:0/3816605550 learned_addr learned my addr 172.21.15.5:0/3816605550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe95259c640 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce1494d0 0x5646ce142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe951d9b640 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce149e80 0x5646ce151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe952d9d640 1 -- 172.21.15.5:0/3816605550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce1494d0 msgr2=0x5646ce142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce1494d0 0x5646ce142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe952d9d640 1 -- 172.21.15.5:0/3816605550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce149e80 msgr2=0x5646ce151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce149e80 0x5646ce151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe952d9d640 1 -- 172.21.15.5:0/3816605550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ce143220 con 0x5646ce152a00 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe952d9d640 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 0x5646ce1522c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe94c019730 tx=0x7fe94c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ab3b983900b809 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe93bfff640 1 -- 172.21.15.5:0/3816605550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c016040 con 0x5646ce152a00 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe93bfff640 1 -- 172.21.15.5:0/3816605550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c010e00 con 0x5646ce152a00 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.768+0000 7fe93bfff640 1 -- 172.21.15.5:0/3816605550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c011b90 con 0x5646ce152a00 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3816605550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 msgr2=0x5646ce1522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 0x5646ce1522c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe94c019730 tx=0x7fe94c022000 comp rx=0 tx=0).stop 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3816605550 shutdown_connections 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce152a00 0x5646ce1522c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce149e80 0x5646ce151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/3816605550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce1494d0 0x5646ce142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3816605550 >> 172.21.15.5:0/3816605550 conn(0x5646ce143ba0 msgr2=0x5646ce14ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3816605550 shutdown_connections 2024-04-22T13:27:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- 172.21.15.5:0/3816605550 wait complete. 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- start start 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 Processor -- start 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce157d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce158280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541aeea0 con 0x5646ce1494d0 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541ae7d0 con 0x5646ce152a00 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541aba10 con 0x5646ce149e80 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95bc0b740 10 client.0 _ll_register_callbacks cb 0x5646ce035b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce15afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58994/0 (socket says 172.21.15.5:58994) 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe95259c640 1 -- 172.21.15.5:0/546017960 learned_addr learned my addr 172.21.15.5:0/546017960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.769+0000 7fe951d9b640 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95259c640 1 -- 172.21.15.5:0/546017960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 msgr2=0x5646ce157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95259c640 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95259c640 1 -- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 msgr2=0x5646ce158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95259c640 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95259c640 1 -- 172.21.15.5:0/546017960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ce1588c0 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95259c640 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce15afd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x5646ce165750 tx=0x7fe94401af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea33beb62dc496f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe939ffb640 1 -- 172.21.15.5:0/546017960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944005050 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe939ffb640 1 -- 172.21.15.5:0/546017960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe9440051f0 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe94c010e00 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe939ffb640 1 -- 172.21.15.5:0/546017960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9440053f0 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 10 client.15924 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646ce1c9ee0 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe939ffb640 1 -- 172.21.15.5:0/546017960 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c010e00 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe939ffb640 1 -- 172.21.15.5:0/546017960 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe94400f040 con 0x5646ce1494d0 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 10 client.15924 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:27:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 20 client.15924 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce1ca5c0 0x5646ce1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 4 client.15924 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.770+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646ce0bbc10 con 0x5646ce1ca5c0 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe951d9b640 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce1ca5c0 0x5646ce1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe951d9b640 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce1ca5c0 0x5646ce1cca00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60757511fcdd76be server_cookie=3aed4837f48c47ee in_seq=0 out_seq=0 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe939ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe939ffb640 1 -- 172.21.15.5:0/546017960 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646ce0bbc10 con 0x5646ce1ca5c0 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe939ffb640 10 client.15924 handle_command_reply: tid=0 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 client.15924 shutdown 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 msgr2=0x5646ce15afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce15afd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x5646ce165750 tx=0x7fe94401af90 comp rx=0 tx=0).stop 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 shutdown_connections 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5646ce1ca5c0 0x5646ce1cca00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5646ce152a00 0x5646ce158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe939ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646ce149e80 0x5646ce157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 --2- 172.21.15.5:0/546017960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646ce1494d0 0x5646ce15afd0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.771+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 >> 172.21.15.5:0/546017960 conn(0x5646ce143ba0 msgr2=0x5646ce144e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 shutdown_connections 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe95bc0b740 1 -- 172.21.15.5:0/546017960 wait complete. 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe95bc0b740 20 client.15924 trim_cache size 0 max 16384 2024-04-22T13:27:47.775 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe93c0a3e40 msgr2=0x7fe93c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe93c0a3e40 0x7fe93c0a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00f010 tx=0x7fe94c010cd0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 msgr2=0x7fe95419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe95419e990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe948009230 tx=0x7fe9480045e0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 shutdown_connections 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954102a60 0x7fe95419e990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe93c0a3e40 0x7fe93c0a6300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe954103400 0x7fe954198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 --2- 172.21.15.5:0/983228423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe954101180 0x7fe95419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 >> 172.21.15.5:0/983228423 conn(0x7fe954077f70 msgr2=0x7fe95406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 shutdown_connections 2024-04-22T13:27:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.772+0000 7fe958f46640 1 -- 172.21.15.5:0/983228423 wait complete. 2024-04-22T13:27:47.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:27:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 -- 172.21.15.5:0/3200678728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a81044f0 msgr2=0x7fc4a810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 --2- 172.21.15.5:0/3200678728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a81044f0 0x7fc4a810eef0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009cd0 tx=0x7fc4a002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 -- 172.21.15.5:0/3200678728 shutdown_connections 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 --2- 172.21.15.5:0/3200678728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a81044f0 0x7fc4a810eef0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 --2- 172.21.15.5:0/3200678728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 0x7fc4a8103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 --2- 172.21.15.5:0/3200678728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4a8102270 0x7fc4a8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 -- 172.21.15.5:0/3200678728 >> 172.21.15.5:0/3200678728 conn(0x7fc4a80fda10 msgr2=0x7fc4a80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 -- 172.21.15.5:0/3200678728 shutdown_connections 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.889+0000 7fc4af784640 1 -- 172.21.15.5:0/3200678728 wait complete. 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 Processor -- start 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 -- start start 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 0x7fc4a8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 0x7fc4a8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4a81044f0 0x7fc4a8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8112270 con 0x7fc4a8102270 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81120f0 con 0x7fc4a81044f0 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4af784640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81123f0 con 0x7fc4a8103b50 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 0x7fc4a8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 0x7fc4a8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59012/0 (socket says 172.21.15.5:59012) 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 -- 172.21.15.5:0/384720173 learned_addr learned my addr 172.21.15.5:0/384720173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4add21640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4a81044f0 0x7fc4a8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 0x7fc4a8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 -- 172.21.15.5:0/384720173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 msgr2=0x7fc4a8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 0x7fc4a8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 -- 172.21.15.5:0/384720173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4a81044f0 msgr2=0x7fc4a8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4a81044f0 0x7fc4a8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 -- 172.21.15.5:0/384720173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a81a6bb0 con 0x7fc4a8102270 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 0x7fc4a8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.890+0000 7fc4ad520640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 0x7fc4a8074250 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.891+0000 7fc49e7fc640 1 -- 172.21.15.5:0/384720173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4017070 con 0x7fc4a8102270 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.891+0000 7fc49e7fc640 1 -- 172.21.15.5:0/384720173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a4009c30 con 0x7fc4a8102270 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.891+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81a6ea0 con 0x7fc4a8102270 2024-04-22T13:27:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.891+0000 7fc49e7fc640 1 -- 172.21.15.5:0/384720173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400f620 con 0x7fc4a8102270 2024-04-22T13:27:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.891+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a73b0 con 0x7fc4a8102270 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 Processor -- start 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc49e7fc640 1 -- 172.21.15.5:0/384720173 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fc4a4010460 con 0x7fc4a8102270 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 -- start start 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 0x563f23d51e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f23cab1f0 con 0x563f23d4fa00 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563f23c484b0 con 0x563f23d46e80 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4b237b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f23ca71b0 con 0x563f23d464d0 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4acd1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 0x563f23d51e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4acd1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 0x563f23d51e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59016/0 (socket says 172.21.15.5:59016) 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4acd1f640 1 -- 172.21.15.5:0/3298800848 learned_addr learned my addr 172.21.15.5:0/3298800848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4add21640 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.894+0000 7fc4ad520640 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4acd1f640 1 -- 172.21.15.5:0/3298800848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 msgr2=0x563f23d468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4acd1f640 1 -- 172.21.15.5:0/3298800848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 msgr2=0x563f23d4f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4acd1f640 1 -- 172.21.15.5:0/3298800848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f23c3a1b0 con 0x563f23d4fa00 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4ad520640 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc49e7fc640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc4940a3ec0 0x7fc4940a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 0x563f23d51e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc498008ba0 tx=0x7fc49800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab95b0c132ddfb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4add21640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc4940a3ec0 0x7fc4940a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc49e7fc640 1 -- 172.21.15.5:0/384720173 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc4a40d0620 con 0x7fc4a8102270 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4937fe640 1 -- 172.21.15.5:0/3298800848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498014070 con 0x563f23d4fa00 2024-04-22T13:27:47.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4937fe640 1 -- 172.21.15.5:0/3298800848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4980047a0 con 0x563f23d4fa00 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.895+0000 7fc4937fe640 1 -- 172.21.15.5:0/3298800848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498010670 con 0x563f23d4fa00 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 -- 172.21.15.5:0/3298800848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 msgr2=0x563f23d51e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4add21640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc4940a3ec0 0x7fc4940a6380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005cc0 tx=0x7fc4a0005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 0x563f23d51e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc498008ba0 tx=0x7fc49800e720 comp rx=0 tx=0).stop 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 -- 172.21.15.5:0/3298800848 shutdown_connections 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d4fa00 0x563f23d51e50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 --2- 172.21.15.5:0/3298800848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 -- 172.21.15.5:0/3298800848 >> 172.21.15.5:0/3298800848 conn(0x563f23d40ba0 msgr2=0x563f23d40f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 -- 172.21.15.5:0/3298800848 shutdown_connections 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 -- 172.21.15.5:0/3298800848 wait complete. 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 -- start start 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.896+0000 7fc4b237b740 1 Processor -- start 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d4f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f23d52970 con 0x563f23d464d0 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563f23c64500 con 0x563f23d46e80 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f23d52af0 con 0x563f23d4fa00 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 10 client.0 _ll_register_callbacks cb 0x563f23c32b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d4f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d4f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59020/0 (socket says 172.21.15.5:59020) 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 -- 172.21.15.5:0/4216754736 learned_addr learned my addr 172.21.15.5:0/4216754736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4add21640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 -- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 msgr2=0x563f23d4cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 -- 172.21.15.5:0/4216754736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 msgr2=0x563f23d4c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 -- 172.21.15.5:0/4216754736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f239bd3f0 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4ad520640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d4f2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4098800 tx=0x7fc4a409bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4170f2bdba1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc490ff9640 1 -- 172.21.15.5:0/4216754736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4020050 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc490ff9640 1 -- 172.21.15.5:0/4216754736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc4a40ce100 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.897+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4980047a0 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc490ff9640 1 -- 172.21.15.5:0/4216754736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc4a409c3a0 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 10 client.15936 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f23a877d0 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc490ff9640 1 -- 172.21.15.5:0/4216754736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4980047a0 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc490ff9640 1 -- 172.21.15.5:0/4216754736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc4a40d0ad0 con 0x563f23d464d0 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 20 client.15936 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23d5fbb0 0x563f23d62020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 4 client.15936 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f23c4a930 con 0x563f23d5fbb0 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.898+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23d5fbb0 0x563f23d62020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23d5fbb0 0x563f23d62020 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d4a245d480b6fc2 server_cookie=8ff1e8c4e60c7724 in_seq=0 out_seq=0 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc490ff9640 10 client.15936 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc490ff9640 1 -- 172.21.15.5:0/4216754736 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f23c4a930 con 0x563f23d5fbb0 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc490ff9640 10 client.15936 handle_command_reply: tid=0 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc4b237b740 1 client.15936 shutdown 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 msgr2=0x563f23d4f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.899+0000 7fc4b237b740 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d4f2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4098800 tx=0x7fc4a409bdc0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 shutdown_connections 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23d5fbb0 0x563f23d62020 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d4c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 --2- 172.21.15.5:0/4216754736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d4f2f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 >> 172.21.15.5:0/4216754736 conn(0x563f23d40ba0 msgr2=0x563f23d428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc490ff9640 0 client.15936 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 shutdown_connections 2024-04-22T13:27:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 1 -- 172.21.15.5:0/4216754736 wait complete. 2024-04-22T13:27:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.900+0000 7fc4b237b740 20 client.15936 trim_cache size 0 max 16384 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 Processor -- start 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 -- start start 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81afb20 con 0x563f23d464d0 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81b1660 con 0x563f23d46e80 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4b237b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81b01f0 con 0x563f23d4fa00 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.913+0000 7fc4ad520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59022/0 (socket says 172.21.15.5:59022) 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 -- 172.21.15.5:0/2029585162 learned_addr learned my addr 172.21.15.5:0/2029585162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4add21640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 -- 172.21.15.5:0/2029585162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 msgr2=0x563f23d4edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 -- 172.21.15.5:0/2029585162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 msgr2=0x563f23d47ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d47ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 -- 172.21.15.5:0/2029585162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f23d4abd0 con 0x563f23d464d0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4add21640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc4ad520640 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d5b830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4a409c0a0 tx=0x7fc4a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483a9897dd06f2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc492ffd640 1 -- 172.21.15.5:0/2029585162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4020070 con 0x563f23d464d0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc492ffd640 1 -- 172.21.15.5:0/2029585162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a4015020 con 0x563f23d464d0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.914+0000 7fc492ffd640 1 -- 172.21.15.5:0/2029585162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40cdf00 con 0x563f23d464d0 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- 172.21.15.5:0/2029585162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 msgr2=0x563f23d5b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d5b830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4a409c0a0 tx=0x7fc4a4005880 comp rx=0 tx=0).stop 2024-04-22T13:27:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- 172.21.15.5:0/2029585162 shutdown_connections 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d4fa00 0x563f23d4edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d46e80 0x563f23d47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- 172.21.15.5:0/2029585162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d464d0 0x563f23d5b830 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- 172.21.15.5:0/2029585162 >> 172.21.15.5:0/2029585162 conn(0x563f23d40ba0 msgr2=0x563f23d473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- 172.21.15.5:0/2029585162 shutdown_connections 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- 172.21.15.5:0/2029585162 wait complete. 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- start start 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 Processor -- start 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d57f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 0x563f23d48cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d4fa00 0x563f23d49200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81b0f90 con 0x563f23d46e80 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81b08c0 con 0x563f23d4fa00 2024-04-22T13:27:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81adb00 con 0x563f23d464d0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4b237b740 10 client.0 _ll_register_callbacks cb 0x7fc4a409bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.915+0000 7fc4acd1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 0x563f23d48cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 0x563f23d48cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59030/0 (socket says 172.21.15.5:59030) 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 -- 172.21.15.5:0/1978238047 learned_addr learned my addr 172.21.15.5:0/1978238047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4ad520640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d57f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4add21640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d4fa00 0x563f23d49200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 -- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 msgr2=0x563f23d57f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d57f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 -- 172.21.15.5:0/1978238047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d4fa00 msgr2=0x563f23d49200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d4fa00 0x563f23d49200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 -- 172.21.15.5:0/1978238047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f23d49740 con 0x563f23d46e80 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4acd1f640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 0x563f23d48cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc49800ec20 tx=0x7fc49800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940babbcc443bf09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc490ff9640 1 -- 172.21.15.5:0/1978238047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498014040 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc490ff9640 1 -- 172.21.15.5:0/1978238047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc4980054d0 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a4015020 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc490ff9640 1 -- 172.21.15.5:0/1978238047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc498010650 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4b237b740 10 client.15942 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.916+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f23dc5d80 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc490ff9640 1 -- 172.21.15.5:0/1978238047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a4015020 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc490ff9640 1 -- 172.21.15.5:0/1978238047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc498003990 con 0x563f23d46e80 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4b237b740 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4b237b740 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4b237b740 20 client.15942 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4b237b740 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23dc62b0 0x563f23dc86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4b237b740 4 client.15942 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f23cb8c10 con 0x563f23dc62b0 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4ad520640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23dc62b0 0x563f23dc86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc490ff9640 10 client.15942 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.917+0000 7fc4ad520640 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23dc62b0 0x563f23dc86d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cd8eacb16dac30a server_cookie=108c84c730908926 in_seq=0 out_seq=0 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc490ff9640 1 -- 172.21.15.5:0/1978238047 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f23cb8c10 con 0x563f23dc62b0 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc490ff9640 10 client.15942 handle_command_reply: tid=0 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 client.15942 shutdown 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 msgr2=0x563f23d48cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 0x563f23d48cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc49800ec20 tx=0x7fc49800ef50 comp rx=0 tx=0).stop 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 shutdown_connections 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563f23dc62b0 0x563f23dc86d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f23d46e80 0x563f23d48cc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc490ff9640 0 client.15942 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f23d4fa00 0x563f23d49200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 --2- 172.21.15.5:0/1978238047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f23d464d0 0x563f23d57f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 >> 172.21.15.5:0/1978238047 conn(0x563f23d40ba0 msgr2=0x563f23d42780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 shutdown_connections 2024-04-22T13:27:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 1 -- 172.21.15.5:0/1978238047 wait complete. 2024-04-22T13:27:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4b237b740 20 client.15942 trim_cache size 0 max 16384 2024-04-22T13:27:47.922 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc4940a3ec0 msgr2=0x7fc4940a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4af784640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc4940a3ec0 0x7fc4940a6380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005cc0 tx=0x7fc4a0005c50 comp rx=0 tx=0).stop 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 msgr2=0x7fc4a8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.918+0000 7fc4af784640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 0x7fc4a8074250 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400de10 comp rx=0 tx=0).stop 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 shutdown_connections 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc4940a3ec0 0x7fc4940a6380 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4a81044f0 0x7fc4a8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4a8103b50 0x7fc4a8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 --2- 172.21.15.5:0/384720173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4a8102270 0x7fc4a8074250 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:47.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 >> 172.21.15.5:0/384720173 conn(0x7fc4a80fda10 msgr2=0x7fc4a80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:47.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 shutdown_connections 2024-04-22T13:27:47.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:47.919+0000 7fc4af784640 1 -- 172.21.15.5:0/384720173 wait complete. 2024-04-22T13:27:47.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:27:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 -- 172.21.15.5:0/3616779839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 msgr2=0x7f03c410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 --2- 172.21.15.5:0/3616779839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c410ef20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ca0 tx=0x7f03bc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 -- 172.21.15.5:0/3616779839 shutdown_connections 2024-04-22T13:27:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 --2- 172.21.15.5:0/3616779839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c410ef20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 --2- 172.21.15.5:0/3616779839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4100880 0x7f03c4100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 --2- 172.21.15.5:0/3616779839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4107080 0x7f03c4107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 -- 172.21.15.5:0/3616779839 >> 172.21.15.5:0/3616779839 conn(0x7f03c40760d0 msgr2=0x7f03c40764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 -- 172.21.15.5:0/3616779839 shutdown_connections 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.034+0000 7f03c988c640 1 -- 172.21.15.5:0/3616779839 wait complete. 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 Processor -- start 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 -- start start 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4100880 0x7f03c4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4107080 0x7f03c419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41122b0 con 0x7f03c4101240 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4112130 con 0x7f03c4100880 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c988c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4112430 con 0x7f03c4107080 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.035+0000 7f03c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59054/0 (socket says 172.21.15.5:59054) 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 -- 172.21.15.5:0/2768706506 learned_addr learned my addr 172.21.15.5:0/2768706506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4100880 0x7f03c4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4107080 0x7f03c419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 -- 172.21.15.5:0/2768706506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4107080 msgr2=0x7f03c419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4107080 0x7f03c419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 -- 172.21.15.5:0/2768706506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4100880 msgr2=0x7f03c4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4100880 0x7f03c4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 -- 172.21.15.5:0/2768706506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c419d720 con 0x7f03c4101240 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4107080 0x7f03c419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c4198c70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f03b800d770 tx=0x7f03b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c888a640 1 -- 172.21.15.5:0/2768706506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8017070 con 0x7f03c4101240 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c888a640 1 -- 172.21.15.5:0/2768706506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b80054d0 con 0x7f03c4101240 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c419da10 con 0x7f03c4101240 2024-04-22T13:27:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c888a640 1 -- 172.21.15.5:0/2768706506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8004c30 con 0x7f03c4101240 2024-04-22T13:27:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.036+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c41a52a0 con 0x7f03c4101240 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.038+0000 7f03c888a640 1 -- 172.21.15.5:0/2768706506 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f03b8004e10 con 0x7f03c4101240 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 Processor -- start 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 -- start start 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a794d0 0x55fc05a798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 0x55fc05a82430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a84e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fc059de1f0 con 0x55fc05a79e80 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fc0597b4b0 con 0x55fc05a794d0 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03cc122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fc059da1b0 con 0x55fc05a82a00 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c888a640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03ac0a3e40 0x7f03ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 0x55fc05a82430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 0x55fc05a82430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59064/0 (socket says 172.21.15.5:59064) 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786931332 learned_addr learned my addr 172.21.15.5:0/3786931332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03ac0a3e40 0x7f03ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c888a640 1 -- 172.21.15.5:0/2768706506 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f03b80d0550 con 0x7f03c4101240 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c27fc640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a84e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c37fe640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a794d0 0x55fc05a798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.039+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786931332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 msgr2=0x55fc05a84e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a84e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786931332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a794d0 msgr2=0x55fc05a798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a794d0 0x55fc05a798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c2ffd640 1 -- 172.21.15.5:0/3786931332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc0596d1b0 con 0x55fc05a79e80 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c27fc640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a84e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c37fe640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a794d0 0x55fc05a798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 0x55fc05a82430 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009cf0 tx=0x7f03b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26d69a4d0b36108 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03a9ffb640 1 -- 172.21.15.5:0/3786931332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004230 con 0x55fc05a79e80 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03a9ffb640 1 -- 172.21.15.5:0/3786931332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b40043d0 con 0x55fc05a79e80 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03ac0a3e40 0x7f03ac0a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005e30 tx=0x7f03bc004a80 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.040+0000 7f03a9ffb640 1 -- 172.21.15.5:0/3786931332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4019880 con 0x55fc05a79e80 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- 172.21.15.5:0/3786931332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 msgr2=0x55fc05a82430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 0x55fc05a82430 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009cf0 tx=0x7f03b4011810 comp rx=0 tx=0).stop 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- 172.21.15.5:0/3786931332 shutdown_connections 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a79e80 0x55fc05a82430 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a84e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- 172.21.15.5:0/3786931332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a794d0 0x55fc05a798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- 172.21.15.5:0/3786931332 >> 172.21.15.5:0/3786931332 conn(0x55fc05a73ba0 msgr2=0x55fc05a73f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- 172.21.15.5:0/3786931332 shutdown_connections 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- 172.21.15.5:0/3786931332 wait complete. 2024-04-22T13:27:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- start start 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 Processor -- start 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a79e80 0x55fc05a7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a82a00 0x55fc05a7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fc05a85970 con 0x55fc05a794d0 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.041+0000 7f03cc122740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fc05997500 con 0x55fc05a82a00 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03cc122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fc05a85af0 con 0x55fc05a79e80 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03cc122740 10 client.0 _ll_register_callbacks cb 0x55fc05965b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a821f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59080/0 (socket says 172.21.15.5:59080) 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2048037428 learned_addr learned my addr 172.21.15.5:0/2048037428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a79e80 0x55fc05a7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a82a00 0x55fc05a7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2048037428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a79e80 msgr2=0x55fc05a7f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a79e80 0x55fc05a7f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a82a00 msgr2=0x55fc05a7fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a82a00 0x55fc05a7fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2048037428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc056f03f0 con 0x55fc05a794d0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a82a00 0x55fc05a7fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a821f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f03b4003f50 tx=0x7f03b40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918484b4e0395c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03a2ffd640 1 -- 172.21.15.5:0/2048037428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4027070 con 0x55fc05a794d0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03a2ffd640 1 -- 172.21.15.5:0/2048037428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f03b4005240 con 0x55fc05a794d0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b40043d0 con 0x55fc05a794d0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03a2ffd640 1 -- 172.21.15.5:0/2048037428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03b4033050 con 0x55fc05a794d0 2024-04-22T13:27:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03cc122740 10 client.15954 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.042+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc057ba7d0 con 0x55fc05a794d0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03a2ffd640 1 -- 172.21.15.5:0/2048037428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b40043d0 con 0x55fc05a794d0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03a2ffd640 1 -- 172.21.15.5:0/2048037428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03b4004be0 con 0x55fc05a794d0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03cc122740 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03cc122740 10 client.15954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03cc122740 20 client.15954 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03cc122740 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05a929b0 0x55fc05a94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03cc122740 4 client.15954 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc0597d930 con 0x55fc05a929b0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05a929b0 0x55fc05a94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03a2ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.043+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05a929b0 0x55fc05a94da0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5d1489e66ab8521 server_cookie=b7fe9aef78e421a7 in_seq=0 out_seq=0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03a2ffd640 1 -- 172.21.15.5:0/2048037428 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc0597d930 con 0x55fc05a929b0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03a2ffd640 10 client.15954 handle_command_reply: tid=0 2024-04-22T13:27:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 client.15954 shutdown 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 msgr2=0x55fc05a821f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a821f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f03b4003f50 tx=0x7f03b40038a0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 shutdown_connections 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05a929b0 0x55fc05a94da0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a82a00 0x55fc05a7fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a79e80 0x55fc05a7f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 --2- 172.21.15.5:0/2048037428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a821f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03a2ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 >> 172.21.15.5:0/2048037428 conn(0x55fc05a73ba0 msgr2=0x55fc05a758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.044+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 shutdown_connections 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.045+0000 7f03cc122740 1 -- 172.21.15.5:0/2048037428 wait complete. 2024-04-22T13:27:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.045+0000 7f03cc122740 20 client.15954 trim_cache size 0 max 16384 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 Processor -- start 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 -- start start 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a72c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a81cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ada00 con 0x55fc05a794d0 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41af540 con 0x55fc05a79e80 2024-04-22T13:27:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03cc122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ae0d0 con 0x55fc05a82a00 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a72c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59092/0 (socket says 172.21.15.5:59092) 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2448596343 learned_addr learned my addr 172.21.15.5:0/2448596343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c37fe640 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a81cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2448596343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 msgr2=0x55fc05a821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2448596343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 msgr2=0x55fc05a81cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a81cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c2ffd640 1 -- 172.21.15.5:0/2448596343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc05a79c20 con 0x55fc05a794d0 2024-04-22T13:27:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.058+0000 7f03c27fc640 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a81cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a72c00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f03b4000c00 tx=0x7f03b4003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bc9518aad06084 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03a97fa640 1 -- 172.21.15.5:0/2448596343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4027040 con 0x55fc05a794d0 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03a97fa640 1 -- 172.21.15.5:0/2448596343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b4011d10 con 0x55fc05a794d0 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03a97fa640 1 -- 172.21.15.5:0/2448596343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4019b60 con 0x55fc05a794d0 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 -- 172.21.15.5:0/2448596343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 msgr2=0x55fc05a72c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a72c00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f03b4000c00 tx=0x7f03b4003e30 comp rx=0 tx=0).stop 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 -- 172.21.15.5:0/2448596343 shutdown_connections 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05a821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a81cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 --2- 172.21.15.5:0/2448596343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a72c00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 -- 172.21.15.5:0/2448596343 >> 172.21.15.5:0/2448596343 conn(0x55fc05a73ba0 msgr2=0x55fc05a75890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 -- 172.21.15.5:0/2448596343 shutdown_connections 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.059+0000 7f03cc122740 1 -- 172.21.15.5:0/2448596343 wait complete. 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 -- start start 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 Processor -- start 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a87d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05af6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41aee70 con 0x55fc05a794d0 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ae7a0 con 0x55fc05a79e80 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ab9e0 con 0x55fc05a82a00 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03cc122740 10 client.0 _ll_register_callbacks cb 0x7f03b4005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a87d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a87d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59104/0 (socket says 172.21.15.5:59104) 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c2ffd640 1 -- 172.21.15.5:0/786916502 learned_addr learned my addr 172.21.15.5:0/786916502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c27fc640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c2ffd640 1 -- 172.21.15.5:0/786916502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 msgr2=0x55fc05af6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c37fe640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05af6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.060+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05af6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03c2ffd640 1 -- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 msgr2=0x55fc05a882b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a882b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03c2ffd640 1 -- 172.21.15.5:0/786916502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc05af7350 con 0x55fc05a794d0 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03c2ffd640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a87d70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f03b4006010 tx=0x7f03b4019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ee0a95d43b3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:47 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/911181151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03a2ffd640 1 -- 172.21.15.5:0/786916502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4019b60 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03a2ffd640 1 -- 172.21.15.5:0/786916502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f03b4019d00 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b4011d10 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03a2ffd640 1 -- 172.21.15.5:0/786916502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03b404d050 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03cc122740 10 client.15960 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc05af8450 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.061+0000 7f03a2ffd640 1 -- 172.21.15.5:0/786916502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b4011d10 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03a2ffd640 1 -- 172.21.15.5:0/786916502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03b4004730 con 0x55fc05a794d0 2024-04-22T13:27:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03cc122740 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03cc122740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03cc122740 20 client.15960 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03cc122740 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05af8a60 0x55fc05afae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03cc122740 4 client.15960 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc059ebc10 con 0x55fc05af8a60 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03c27fc640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05af8a60 0x55fc05afae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03c27fc640 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05af8a60 0x55fc05afae80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=399c87a340a89e28 server_cookie=e8b0f9b252f20445 in_seq=0 out_seq=0 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.062+0000 7f03a2ffd640 10 client.15960 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03a2ffd640 1 -- 172.21.15.5:0/786916502 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc059ebc10 con 0x55fc05af8a60 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03a2ffd640 10 client.15960 handle_command_reply: tid=0 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 client.15960 shutdown 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 msgr2=0x55fc05a87d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a87d70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f03b4006010 tx=0x7f03b4019420 comp rx=0 tx=0).stop 2024-04-22T13:27:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 shutdown_connections 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55fc05af8a60 0x55fc05afae80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc05a82a00 0x55fc05af6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fc05a79e80 0x55fc05a882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 --2- 172.21.15.5:0/786916502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc05a794d0 0x55fc05a87d70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 >> 172.21.15.5:0/786916502 conn(0x55fc05a73ba0 msgr2=0x55fc05a7aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03a2ffd640 0 client.15960 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 shutdown_connections 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 1 -- 172.21.15.5:0/786916502 wait complete. 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03cc122740 20 client.15960 trim_cache size 0 max 16384 2024-04-22T13:27:48.067 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03ac0a3e40 msgr2=0x7f03ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03c988c640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03ac0a3e40 0x7f03ac0a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005e30 tx=0x7f03bc004a80 comp rx=0 tx=0).stop 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 msgr2=0x7f03c4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.063+0000 7f03c988c640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c4198c70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f03b800d770 tx=0x7f03b800dc30 comp rx=0 tx=0).stop 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 shutdown_connections 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c4101240 0x7f03c4198c70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f03ac0a3e40 0x7f03ac0a6300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c4107080 0x7f03c419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 --2- 172.21.15.5:0/2768706506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c4100880 0x7f03c4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 >> 172.21.15.5:0/2768706506 conn(0x7f03c40760d0 msgr2=0x7f03c4104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 shutdown_connections 2024-04-22T13:27:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.064+0000 7f03c988c640 1 -- 172.21.15.5:0/2768706506 wait complete. 2024-04-22T13:27:48.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 -- 172.21.15.5:0/661458858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8102a60 msgr2=0x7f1bd8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/661458858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8102a60 0x7f1bd8102ec0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009930 tx=0x7f1bc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 -- 172.21.15.5:0/661458858 shutdown_connections 2024-04-22T13:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/661458858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8102a60 0x7f1bd8102ec0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/661458858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bd8103400 0x7f1bd810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/661458858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bd8101180 0x7f1bd8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 -- 172.21.15.5:0/661458858 >> 172.21.15.5:0/661458858 conn(0x7f1bd8077f70 msgr2=0x7f1bd8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 -- 172.21.15.5:0/661458858 shutdown_connections 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.181+0000 7f1bdf19d640 1 -- 172.21.15.5:0/661458858 wait complete. 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 Processor -- start 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 -- start start 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bd8101180 0x7f1bd81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bd8102a60 0x7f1bd8199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 0x7f1bd81a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81111a0 con 0x7f1bd8103400 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8111020 con 0x7f1bd8102a60 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdf19d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8111320 con 0x7f1bd8101180 2024-04-22T13:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 0x7f1bd81a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 0x7f1bd81a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59114/0 (socket says 172.21.15.5:59114) 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 -- 172.21.15.5:0/2065440131 learned_addr learned my addr 172.21.15.5:0/2065440131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bd8101180 0x7f1bd81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bd8102a60 0x7f1bd8199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 -- 172.21.15.5:0/2065440131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bd8101180 msgr2=0x7f1bd81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bd8101180 0x7f1bd81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 -- 172.21.15.5:0/2065440131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bd8102a60 msgr2=0x7f1bd8199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bd8102a60 0x7f1bd8199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 -- 172.21.15.5:0/2065440131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd81a4c60 con 0x7f1bd8103400 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.182+0000 7f1bdd73a640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 0x7f1bd81a4720 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000cad0 tx=0x7f1bd000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.183+0000 7f1bd5ffb640 1 -- 172.21.15.5:0/2065440131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000ae20 con 0x7f1bd8103400 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.183+0000 7f1bd5ffb640 1 -- 172.21.15.5:0/2065440131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0005020 con 0x7f1bd8103400 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.183+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81a4e30 con 0x7f1bd8103400 2024-04-22T13:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.183+0000 7f1bd5ffb640 1 -- 172.21.15.5:0/2065440131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000f6b0 con 0x7f1bd8103400 2024-04-22T13:27:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.183+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd81a5300 con 0x7f1bd8103400 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1bd5ffb640 1 -- 172.21.15.5:0/2065440131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f1bd00051d0 con 0x7f1bd8103400 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 Processor -- start 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 -- start start 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9565e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95667430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95669e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c955c31f0 con 0x560c9565e4d0 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c955604b0 con 0x560c9565ee80 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c955bf1b0 con 0x560c95667a00 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.185+0000 7f1bd5ffb640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bc80a3e40 0x7f1bc80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9565e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bdd73a640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bc80a3e40 0x7f1bc80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9565e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59118/0 (socket says 172.21.15.5:59118) 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 -- 172.21.15.5:0/104703501 learned_addr learned my addr 172.21.15.5:0/104703501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95667430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd5ffb640 1 -- 172.21.15.5:0/2065440131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1bd00d0b70 con 0x7f1bd8103400 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95669e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 -- 172.21.15.5:0/104703501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 msgr2=0x560c95669e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95669e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 -- 172.21.15.5:0/104703501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 msgr2=0x560c95667430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95667430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 -- 172.21.15.5:0/104703501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c955521b0 con 0x560c9565e4d0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9565e8b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005940 tx=0x7f1bc40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d354836b561cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bdd73a640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bc80a3e40 0x7f1bc80a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00ce170 tx=0x7f1bd0087040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bc2ffd640 1 -- 172.21.15.5:0/104703501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc402c7e0 con 0x560c9565e4d0 2024-04-22T13:27:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bc2ffd640 1 -- 172.21.15.5:0/104703501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc4048070 con 0x560c9565e4d0 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.186+0000 7f1bc2ffd640 1 -- 172.21.15.5:0/104703501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4004990 con 0x560c9565e4d0 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 -- 172.21.15.5:0/104703501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 msgr2=0x560c9565e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9565e8b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005940 tx=0x7f1bc40058b0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 -- 172.21.15.5:0/104703501 shutdown_connections 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95669e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95667430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/104703501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9565e8b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 -- 172.21.15.5:0/104703501 >> 172.21.15.5:0/104703501 conn(0x560c95658ba0 msgr2=0x560c95658f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 -- 172.21.15.5:0/104703501 shutdown_connections 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 -- 172.21.15.5:0/104703501 wait complete. 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 -- start start 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.187+0000 7f1be1d8e740 1 Processor -- start 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c956672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95664660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95664bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c9566a910 con 0x560c9565e4d0 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c9557c500 con 0x560c9565ee80 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c9566aa90 con 0x560c95667a00 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1be1d8e740 10 client.0 _ll_register_callbacks cb 0x560c9554ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c956672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c956672d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59126/0 (socket says 172.21.15.5:59126) 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 -- 172.21.15.5:0/63519296 learned_addr learned my addr 172.21.15.5:0/63519296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95664bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95664660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 -- 172.21.15.5:0/63519296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 msgr2=0x560c95664bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95664bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 -- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 msgr2=0x560c95664660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95664660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 -- 172.21.15.5:0/63519296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c952d53f0 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c956672d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4045040 tx=0x7f1bc4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55927952b06de95e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.188+0000 7f1bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1bb7fff640 1 -- 172.21.15.5:0/63519296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4005450 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1bb7fff640 1 -- 172.21.15.5:0/63519296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1bc4004730 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc4048070 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1bb7fff640 1 -- 172.21.15.5:0/63519296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bc4061050 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c9539f7d0 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1bb7fff640 1 -- 172.21.15.5:0/63519296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc4048070 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1bb7fff640 1 -- 172.21.15.5:0/63519296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc40039c0 con 0x560c9565e4d0 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 20 client.15972 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c95677a00 0x560c95679df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 4 client.15972 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.189+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c95562930 con 0x560c95677a00 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.190+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c95677a00 0x560c95679df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.190+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c95677a00 0x560c95679df0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8494211b1fc4b657 server_cookie=3dce0fbd4f9f47f9 in_seq=0 out_seq=0 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.190+0000 7f1bb7fff640 10 client.15972 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1bb7fff640 1 -- 172.21.15.5:0/63519296 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c95562930 con 0x560c95677a00 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1bb7fff640 10 client.15972 handle_command_reply: tid=0 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 client.15972 shutdown 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 msgr2=0x560c956672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c956672d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4045040 tx=0x7f1bc4005d50 comp rx=0 tx=0).stop 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 shutdown_connections 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c95677a00 0x560c95679df0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c95667a00 0x560c95664bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c9565ee80 0x560c95664660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/63519296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c956672d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1bb7fff640 0 client.15972 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 >> 172.21.15.5:0/63519296 conn(0x560c95658ba0 msgr2=0x560c9565a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 shutdown_connections 2024-04-22T13:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 1 -- 172.21.15.5:0/63519296 wait complete. 2024-04-22T13:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.191+0000 7f1be1d8e740 20 client.15972 trim_cache size 0 max 16384 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.204+0000 7f1be1d8e740 1 Processor -- start 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.204+0000 7f1be1d8e740 1 -- start start 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.204+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c95657a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c95666d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c956672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81ada80 con 0x560c9565e4d0 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81af5c0 con 0x560c95667a00 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81ae150 con 0x560c9565ee80 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c95657a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c95657a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59132/0 (socket says 172.21.15.5:59132) 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 -- 172.21.15.5:0/2833032984 learned_addr learned my addr 172.21.15.5:0/2833032984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c95666d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 -- 172.21.15.5:0/2833032984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 msgr2=0x560c95666d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c95666d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 -- 172.21.15.5:0/2833032984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 msgr2=0x560c956672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c956672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 -- 172.21.15.5:0/2833032984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c956583d0 con 0x560c9565e4d0 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c95657a80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009830 tx=0x7f1bc402c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7bcc0ef53704fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bc27fc640 1 -- 172.21.15.5:0/2833032984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4009520 con 0x560c9565e4d0 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bc27fc640 1 -- 172.21.15.5:0/2833032984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc4048040 con 0x560c9565e4d0 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.205+0000 7f1bc27fc640 1 -- 172.21.15.5:0/2833032984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003ac0 con 0x560c9565e4d0 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 -- 172.21.15.5:0/2833032984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 msgr2=0x560c95657a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c95657a80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009830 tx=0x7f1bc402c730 comp rx=0 tx=0).stop 2024-04-22T13:27:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 -- 172.21.15.5:0/2833032984 shutdown_connections 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c956672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c95666d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/2833032984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c95657a80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 -- 172.21.15.5:0/2833032984 >> 172.21.15.5:0/2833032984 conn(0x560c95658ba0 msgr2=0x560c95657370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 -- 172.21.15.5:0/2833032984 shutdown_connections 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 -- 172.21.15.5:0/2833032984 wait complete. 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 -- start start 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.206+0000 7f1be1d8e740 1 Processor -- start 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9567a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c9567a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c9567c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81aeef0 con 0x560c9565e4d0 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81ae820 con 0x560c95667a00 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81aba80 con 0x560c9565ee80 2024-04-22T13:27:48.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 10 client.0 _ll_register_callbacks cb 0x7f1bc402c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9567a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9567a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59134/0 (socket says 172.21.15.5:59134) 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 -- 172.21.15.5:0/3001452957 learned_addr learned my addr 172.21.15.5:0/3001452957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c9567c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c9567a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 -- 172.21.15.5:0/3001452957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 msgr2=0x560c9567a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c9567a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 -- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 msgr2=0x560c9567c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c9567c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 -- 172.21.15.5:0/3001452957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9567cea0 con 0x560c9565e4d0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bd7fff640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9567a2b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005ec0 tx=0x7f1bc402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b0c9ff647a8b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bb7fff640 1 -- 172.21.15.5:0/3001452957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003840 con 0x560c9565e4d0 2024-04-22T13:27:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1bb7fff640 1 -- 172.21.15.5:0/3001452957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1bc40039e0 con 0x560c9565e4d0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.207+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc4048040 con 0x560c9565e4d0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1bb7fff640 1 -- 172.21.15.5:0/3001452957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bc4003be0 con 0x560c9565e4d0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 10 client.15978 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c956dd740 con 0x560c9565e4d0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1bb7fff640 1 -- 172.21.15.5:0/3001452957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc4048040 con 0x560c9565e4d0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1bb7fff640 1 -- 172.21.15.5:0/3001452957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc4004ce0 con 0x560c9565e4d0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 10 client.15978 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 20 client.15978 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c956dde20 0x560c956e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 4 client.15978 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c955d0c10 con 0x560c956dde20 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.208+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c956dde20 0x560c956e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1bdcf39640 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c956dde20 0x560c956e0240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28cb687fe1466225 server_cookie=3b39cb7cb1d6892f in_seq=0 out_seq=0 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1bb7fff640 10 client.15978 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1bb7fff640 1 -- 172.21.15.5:0/3001452957 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c955d0c10 con 0x560c956dde20 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1bb7fff640 10 client.15978 handle_command_reply: tid=0 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 client.15978 shutdown 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 msgr2=0x560c9567a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9567a2b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005ec0 tx=0x7f1bc402c330 comp rx=0 tx=0).stop 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 shutdown_connections 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560c956dde20 0x560c956e0240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c95667a00 0x560c9567c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1bb7fff640 0 client.15978 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c9565ee80 0x560c9567a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 --2- 172.21.15.5:0/3001452957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c9565e4d0 0x560c9567a2b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 >> 172.21.15.5:0/3001452957 conn(0x560c95658ba0 msgr2=0x560c95657c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 shutdown_connections 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 1 -- 172.21.15.5:0/3001452957 wait complete. 2024-04-22T13:27:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.209+0000 7f1be1d8e740 20 client.15978 trim_cache size 0 max 16384 2024-04-22T13:27:48.213 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:27:48.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bc80a3e40 msgr2=0x7f1bc80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bc80a3e40 0x7f1bc80a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00ce170 tx=0x7f1bd0087040 comp rx=0 tx=0).stop 2024-04-22T13:27:48.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 msgr2=0x7f1bd81a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 0x7f1bd81a4720 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000cad0 tx=0x7f1bd000cf90 comp rx=0 tx=0).stop 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 shutdown_connections 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bd8103400 0x7f1bd81a4720 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bc80a3e40 0x7f1bc80a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1bd8102a60 0x7f1bd8199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 --2- 172.21.15.5:0/2065440131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bd8101180 0x7f1bd81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 >> 172.21.15.5:0/2065440131 conn(0x7f1bd8077f70 msgr2=0x7f1bd8104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 shutdown_connections 2024-04-22T13:27:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:48.210+0000 7f1bdf19d640 1 -- 172.21.15.5:0/2065440131 wait complete. 2024-04-22T13:27:48.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:47 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/911181151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:48.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:47 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/911181151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:49.218 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:27:49.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:49.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:27:49.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:48 smithi005 ceph-mon[28006]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-04-22T13:27:49.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:48 smithi049 ceph-mon[31379]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-04-22T13:27:49.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:48 smithi107 ceph-mon[31859]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/3833568210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4102180 msgr2=0x7fa7f4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/3833568210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4102180 0x7fa7f4102560 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009900 tx=0x7fa7e402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/3833568210 shutdown_connections 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/3833568210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7f4107250 0x7fa7f410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/3833568210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7f4102b30 0x7fa7f4106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/3833568210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4102180 0x7fa7f4102560 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/3833568210 >> 172.21.15.5:0/3833568210 conn(0x7fa7f40fda00 msgr2=0x7fa7f40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/3833568210 shutdown_connections 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.342+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/3833568210 wait complete. 2024-04-22T13:27:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 Processor -- start 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 -- start start 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7f4102180 0x7fa7f4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7f4107250 0x7fa7f4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 0x7fa7f41a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41121c0 con 0x7fa7f4074cd0 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4112040 con 0x7fa7f4102180 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.343+0000 7fa7fa6fd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4112340 con 0x7fa7f4107250 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 0x7fa7f41a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 0x7fa7f41a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40570/0 (socket says 172.21.15.5:40570) 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 -- 172.21.15.5:0/2307498842 learned_addr learned my addr 172.21.15.5:0/2307498842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7f4102180 0x7fa7f4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7f4107250 0x7fa7f4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 -- 172.21.15.5:0/2307498842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7f4107250 msgr2=0x7fa7f4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7f4107250 0x7fa7f4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 -- 172.21.15.5:0/2307498842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7f4102180 msgr2=0x7fa7f4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7f4102180 0x7fa7f4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 -- 172.21.15.5:0/2307498842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f41a7070 con 0x7fa7f4074cd0 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f8c9a640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 0x7fa7f41a6970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00eaa0 tx=0x7fa7ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f17fa640 1 -- 172.21.15.5:0/2307498842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00cdb0 con 0x7fa7f4074cd0 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f17fa640 1 -- 172.21.15.5:0/2307498842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec005020 con 0x7fa7f4074cd0 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f41a7360 con 0x7fa7f4074cd0 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7f17fa640 1 -- 172.21.15.5:0/2307498842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0116d0 con 0x7fa7f4074cd0 2024-04-22T13:27:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.344+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f41a78b0 con 0x7fa7f4074cd0 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.347+0000 7fa7f17fa640 1 -- 172.21.15.5:0/2307498842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fa7ec0051d0 con 0x7fa7f4074cd0 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.347+0000 7fa7fd3ca740 1 Processor -- start 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 -- start start 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac404cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac405290 0x55d0ac40d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 0x55d0ac410260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac368f00 con 0x55d0ac40de10 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac306590 con 0x55d0ac4048e0 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac364ec0 con 0x55d0ac405290 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 0x55d0ac410260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 0x55d0ac410260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40576/0 (socket says 172.21.15.5:40576) 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 -- 172.21.15.5:0/19040502 learned_addr learned my addr 172.21.15.5:0/19040502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac404cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac405290 0x55d0ac40d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f17fa640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7dc0a3e70 0x7fa7dc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f8c9a640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7dc0a3e70 0x7fa7dc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 -- 172.21.15.5:0/19040502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac405290 msgr2=0x55d0ac40d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac405290 0x55d0ac40d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 -- 172.21.15.5:0/19040502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 msgr2=0x55d0ac404cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac404cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f37fe640 1 -- 172.21.15.5:0/19040502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ac1437e0 con 0x55d0ac40de10 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.348+0000 7fa7f17fa640 1 -- 172.21.15.5:0/2307498842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa7ec0d0bc0 con 0x7fa7f4074cd0 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 0x55d0ac410260 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8014990 tx=0x7fa7e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8c2649e21eb531 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7da7fc640 1 -- 172.21.15.5:0/19040502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80142d0 con 0x55d0ac40de10 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7f8c9a640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7dc0a3e70 0x7fa7dc0a6330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ce220 tx=0x7fa7ec0bd040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7da7fc640 1 -- 172.21.15.5:0/19040502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e800fc10 con 0x55d0ac40de10 2024-04-22T13:27:49.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7da7fc640 1 -- 172.21.15.5:0/19040502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80129e0 con 0x55d0ac40de10 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/19040502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 msgr2=0x55d0ac410260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 0x55d0ac410260 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8014990 tx=0x7fa7e800c490 comp rx=0 tx=0).stop 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/19040502 shutdown_connections 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.349+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac40de10 0x55d0ac410260 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac405290 0x55d0ac40d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/19040502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac404cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/19040502 >> 172.21.15.5:0/19040502 conn(0x55d0ac3fefb0 msgr2=0x55d0ac3ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/19040502 shutdown_connections 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/19040502 wait complete. 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- start start 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 Processor -- start 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac4048e0 0x55d0ac40ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac40b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac40de10 0x55d0ac40b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac410d80 con 0x55d0ac405290 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac322580 con 0x55d0ac40de10 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.350+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d0ac410f00 con 0x55d0ac4048e0 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7fd3ca740 10 client.0 _ll_register_callbacks cb 0x55d0ac328650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac40b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac40b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40578/0 (socket says 172.21.15.5:40578) 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 -- 172.21.15.5:0/1821103240 learned_addr learned my addr 172.21.15.5:0/1821103240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac40de10 0x55d0ac40b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac4048e0 0x55d0ac40ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 -- 172.21.15.5:0/1821103240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac4048e0 msgr2=0x55d0ac40ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac4048e0 0x55d0ac40ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 -- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac40de10 msgr2=0x55d0ac40b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac40de10 0x55d0ac40b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 -- 172.21.15.5:0/1821103240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ac1a3f70 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac40de10 0x55d0ac40b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac40b360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009900 tx=0x7fa7e402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d164c4a797c5f631 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7d37fe640 1 -- 172.21.15.5:0/1821103240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4005030 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7d37fe640 1 -- 172.21.15.5:0/1821103240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa7e40051d0 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e800fc10 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.351+0000 7fa7d37fe640 1 -- 172.21.15.5:0/1821103240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7e40053d0 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 10 client.15990 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0ac03b170 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7d37fe640 1 -- 172.21.15.5:0/1821103240 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e800fc10 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7d37fe640 1 -- 172.21.15.5:0/1821103240 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa7e4003710 con 0x55d0ac405290 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 20 client.15990 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac41e310 0x55d0ac420770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 4 client.15990 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0ac376920 con 0x55d0ac41e310 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.352+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac41e310 0x55d0ac420770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.353+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac41e310 0x55d0ac420770 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4254816e6aa9fe6e server_cookie=ac887b02b6689463 in_seq=0 out_seq=0 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.353+0000 7fa7d37fe640 10 client.15990 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.353+0000 7fa7d37fe640 1 -- 172.21.15.5:0/1821103240 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0ac376920 con 0x55d0ac41e310 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.353+0000 7fa7d37fe640 10 client.15990 handle_command_reply: tid=0 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 client.15990 shutdown 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 msgr2=0x55d0ac40b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac40b360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009900 tx=0x7fa7e402ca10 comp rx=0 tx=0).stop 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 shutdown_connections 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac41e310 0x55d0ac420770 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac40b360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac40de10 0x55d0ac40b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7d37fe640 0 client.15990 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/1821103240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac4048e0 0x55d0ac40ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 >> 172.21.15.5:0/1821103240 conn(0x55d0ac3fefb0 msgr2=0x55d0ac400d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 shutdown_connections 2024-04-22T13:27:49.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/1821103240 wait complete. 2024-04-22T13:27:49.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.354+0000 7fa7fd3ca740 20 client.15990 trim_cache size 0 max 16384 2024-04-22T13:27:49.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 Processor -- start 2024-04-22T13:27:49.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 -- start start 2024-04-22T13:27:49.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac40d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac419c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac426040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41afe30 con 0x55d0ac405290 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41b1970 con 0x55d0ac4048e0 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41b0500 con 0x55d0ac3fefb0 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac426040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac426040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40588/0 (socket says 172.21.15.5:40588) 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 -- 172.21.15.5:0/4057623558 learned_addr learned my addr 172.21.15.5:0/4057623558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac40d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac419c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 -- 172.21.15.5:0/4057623558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 msgr2=0x55d0ac40d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac40d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 -- 172.21.15.5:0/4057623558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 msgr2=0x55d0ac419c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac419c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.372+0000 7fa7f37fe640 1 -- 172.21.15.5:0/4057623558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ac406560 con 0x55d0ac405290 2024-04-22T13:27:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac426040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8014990 tx=0x7fa7e8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f648925053e3732 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7d9ffb640 1 -- 172.21.15.5:0/4057623558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8014620 con 0x55d0ac405290 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7d9ffb640 1 -- 172.21.15.5:0/4057623558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e8005500 con 0x55d0ac405290 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7d9ffb640 1 -- 172.21.15.5:0/4057623558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e802b720 con 0x55d0ac405290 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/4057623558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 msgr2=0x55d0ac426040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac426040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8014990 tx=0x7fa7e8001e50 comp rx=0 tx=0).stop 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/4057623558 shutdown_connections 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac405290 0x55d0ac426040 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac4048e0 0x55d0ac419c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/4057623558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac40d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/4057623558 >> 172.21.15.5:0/4057623558 conn(0x55d0ac4059e0 msgr2=0x55d0ac400c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/4057623558 shutdown_connections 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.373+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/4057623558 wait complete. 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 -- start start 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 Processor -- start 2024-04-22T13:27:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac4163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 0x55d0ac4127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac405290 0x55d0ac412d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41b12a0 con 0x55d0ac4048e0 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41b0bd0 con 0x55d0ac405290 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41ade10 con 0x55d0ac3fefb0 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7fd3ca740 10 client.0 _ll_register_callbacks cb 0x7fa7e4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 0x55d0ac4127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 0x55d0ac4127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40594/0 (socket says 172.21.15.5:40594) 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7f3fff640 1 -- 172.21.15.5:0/3483070582 learned_addr learned my addr 172.21.15.5:0/3483070582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac405290 0x55d0ac412d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.374+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac4163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f3fff640 1 -- 172.21.15.5:0/3483070582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 msgr2=0x55d0ac4163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac4163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f3fff640 1 -- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac405290 msgr2=0x55d0ac412d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac405290 0x55d0ac412d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f3fff640 1 -- 172.21.15.5:0/3483070582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ac4133c0 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac405290 0x55d0ac412d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 0x55d0ac4127c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4003ca0 tx=0x7fa7e402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18cd69c448ad1b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7d37fe640 1 -- 172.21.15.5:0/3483070582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40035f0 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7d37fe640 1 -- 172.21.15.5:0/3483070582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa7e4003790 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e8005500 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7d37fe640 1 -- 172.21.15.5:0/3483070582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7e4062050 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0ac485800 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7d37fe640 1 -- 172.21.15.5:0/3483070582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e8005500 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7d37fe640 1 -- 172.21.15.5:0/3483070582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa7e4040420 con 0x55d0ac4048e0 2024-04-22T13:27:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 20 client.15996 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac40de10 0x55d0ac487fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 4 client.15996 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.375+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d0ac370790 con 0x55d0ac40de10 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.376+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac40de10 0x55d0ac487fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.376+0000 7fa7f37fe640 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac40de10 0x55d0ac487fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a403c2ed55f8a4b server_cookie=d65c16dd965560e5 in_seq=0 out_seq=0 2024-04-22T13:27:49.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.376+0000 7fa7d37fe640 10 client.15996 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7d37fe640 1 -- 172.21.15.5:0/3483070582 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0ac370790 con 0x55d0ac40de10 2024-04-22T13:27:49.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7d37fe640 10 client.15996 handle_command_reply: tid=0 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 client.15996 shutdown 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 msgr2=0x55d0ac4127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 0x55d0ac4127c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4003ca0 tx=0x7fa7e402c740 comp rx=0 tx=0).stop 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 shutdown_connections 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d0ac40de10 0x55d0ac487fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7d37fe640 0 client.15996 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0ac4048e0 0x55d0ac4127c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0ac405290 0x55d0ac412d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 --2- 172.21.15.5:0/3483070582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0ac3fefb0 0x55d0ac4163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 >> 172.21.15.5:0/3483070582 conn(0x55d0ac4059e0 msgr2=0x55d0ac40adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 shutdown_connections 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 1 -- 172.21.15.5:0/3483070582 wait complete. 2024-04-22T13:27:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.378+0000 7fa7fd3ca740 20 client.15996 trim_cache size 0 max 16384 2024-04-22T13:27:49.381 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:27:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a968198a-9549-4cae-b1a0-f2f6abcd7104", 2024-04-22T13:27:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:27:49.381 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7dc0a3e70 msgr2=0x7fa7dc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7dc0a3e70 0x7fa7dc0a6330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ce220 tx=0x7fa7ec0bd040 comp rx=0 tx=0).stop 2024-04-22T13:27:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 msgr2=0x7fa7f41a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 0x7fa7f41a6970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00eaa0 tx=0x7fa7ec00ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 shutdown_connections 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f4074cd0 0x7fa7f41a6970 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7dc0a3e70 0x7fa7dc0a6330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7f4107250 0x7fa7f4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 --2- 172.21.15.5:0/2307498842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7f4102180 0x7fa7f4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 >> 172.21.15.5:0/2307498842 conn(0x7fa7f40fda00 msgr2=0x7fa7f40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 shutdown_connections 2024-04-22T13:27:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.379+0000 7fa7fa6fd640 1 -- 172.21.15.5:0/2307498842 wait complete. 2024-04-22T13:27:49.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:27:49.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:27:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 -- 172.21.15.5:0/2777576947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 msgr2=0x7fbd0010ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 --2- 172.21.15.5:0/2777576947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd0010ee20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009ca0 tx=0x7fbcf802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 -- 172.21.15.5:0/2777576947 shutdown_connections 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 --2- 172.21.15.5:0/2777576947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd0010ee20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 --2- 172.21.15.5:0/2777576947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102b30 0x7fbd00106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 --2- 172.21.15.5:0/2777576947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd00102180 0x7fbd00102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 -- 172.21.15.5:0/2777576947 >> 172.21.15.5:0/2777576947 conn(0x7fbd000fda00 msgr2=0x7fbd000ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 -- 172.21.15.5:0/2777576947 shutdown_connections 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.535+0000 7fbd05406640 1 -- 172.21.15.5:0/2777576947 wait complete. 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 Processor -- start 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 -- start start 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102180 0x7fbd00198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd00198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0019cfc0 0x7fbd0019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001121c0 con 0x7fbd00107250 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00112040 con 0x7fbd0019cfc0 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbd05406640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00112340 con 0x7fbd00102180 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd00198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd00198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40604/0 (socket says 172.21.15.5:40604) 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 -- 172.21.15.5:0/332034168 learned_addr learned my addr 172.21.15.5:0/332034168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102180 0x7fbd00198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 -- 172.21.15.5:0/332034168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102180 msgr2=0x7fbd00198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102180 0x7fbd00198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 -- 172.21.15.5:0/332034168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0019cfc0 msgr2=0x7fbd0019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0019cfc0 0x7fbd0019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcf7fff640 1 -- 172.21.15.5:0/332034168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0019dab0 con 0x7fbd00107250 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.536+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102180 0x7fbd00198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.537+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd00198c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbce800d740 tx=0x7fbce800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.537+0000 7fbcfcff9640 1 -- 172.21.15.5:0/332034168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8017070 con 0x7fbd00107250 2024-04-22T13:27:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.537+0000 7fbcfcff9640 1 -- 172.21.15.5:0/332034168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce800de50 con 0x7fbd00107250 2024-04-22T13:27:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.537+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0019dda0 con 0x7fbd00107250 2024-04-22T13:27:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.537+0000 7fbcfcff9640 1 -- 172.21.15.5:0/332034168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8004ce0 con 0x7fbd00107250 2024-04-22T13:27:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.537+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd001a5540 con 0x7fbd00107250 2024-04-22T13:27:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.539+0000 7fbcfcff9640 1 -- 172.21.15.5:0/332034168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fbce8004e80 con 0x7fbd00107250 2024-04-22T13:27:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.539+0000 7fbd07ff5740 1 Processor -- start 2024-04-22T13:27:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.539+0000 7fbd07ff5740 1 -- start start 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81cff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d0c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561f81c641e0 con 0x561f81d0ce90 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561f81c01390 con 0x561f81cffbd0 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561f81c601a0 con 0x561f81cff2b0 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcfcff9640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcec0a3e40 0x7fbcec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcfeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d0c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81cff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40608/0 (socket says 172.21.15.5:40608) 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 -- 172.21.15.5:0/1914536409 learned_addr learned my addr 172.21.15.5:0/1914536409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcec0a3e40 0x7fbcec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcfcff9640 1 -- 172.21.15.5:0/332034168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbce80d0460 con 0x7fbd00107250 2024-04-22T13:27:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 -- 172.21.15.5:0/1914536409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 msgr2=0x561f81cff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81cff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 -- 172.21.15.5:0/1914536409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 msgr2=0x561f81d0c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d0c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 -- 172.21.15.5:0/1914536409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f81a407d0 con 0x561f81d0ce90 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d0c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0f280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbce8098bb0 tx=0x7fbce806ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6763c9b3f54f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.540+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcec0a3e40 0x7fbcec0a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8005cc0 tx=0x7fbcf8005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbcf57fa640 1 -- 172.21.15.5:0/1914536409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8020070 con 0x561f81d0ce90 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbcf57fa640 1 -- 172.21.15.5:0/1914536409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce8066cf0 con 0x561f81d0ce90 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbcf57fa640 1 -- 172.21.15.5:0/1914536409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8066990 con 0x561f81d0ce90 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1914536409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 msgr2=0x561f81d0f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0f280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbce8098bb0 tx=0x7fbce806ecd0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1914536409 shutdown_connections 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0f280 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d0c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1914536409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81cff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1914536409 >> 172.21.15.5:0/1914536409 conn(0x561f81cf9cb0 msgr2=0x561f81cfa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1914536409 shutdown_connections 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.541+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1914536409 wait complete. 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 -- start start 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 Processor -- start 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d09ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d0a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561f81d0fb90 con 0x561f81d0ce90 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561f81c1d2f0 con 0x561f81cff2b0 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561f81d0fd10 con 0x561f81cffbd0 2024-04-22T13:27:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbd07ff5740 10 client.0 _ll_register_callbacks cb 0x561f81c2a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40614/0 (socket says 172.21.15.5:40614) 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbcff7fe640 1 -- 172.21.15.5:0/1905525934 learned_addr learned my addr 172.21.15.5:0/1905525934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d09ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d0a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.542+0000 7fbcff7fe640 1 -- 172.21.15.5:0/1905525934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 msgr2=0x561f81d0a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d0a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbcff7fe640 1 -- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 msgr2=0x561f81d09ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d09ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbcff7fe640 1 -- 172.21.15.5:0/1905525934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f819763f0 con 0x561f81d0ce90 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0a960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80329c0 tx=0x7fbcf80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae39964fd4bd316 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbce67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbce67fc640 1 -- 172.21.15.5:0/1905525934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8050020 con 0x561f81d0ce90 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce8066cf0 con 0x561f81d0ce90 2024-04-22T13:27:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbce67fc640 1 -- 172.21.15.5:0/1905525934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbcf8004450 con 0x561f81d0ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbce67fc640 1 -- 172.21.15.5:0/1905525934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbcf805b050 con 0x561f81d0ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbd07ff5740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f819360f0 con 0x561f81d0ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbce67fc640 1 -- 172.21.15.5:0/1905525934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce8066cf0 con 0x561f81d0ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbce67fc640 1 -- 172.21.15.5:0/1905525934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbcf803f040 con 0x561f81d0ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbd07ff5740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.543+0000 7fbd07ff5740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbd07ff5740 20 client.16008 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d1ce90 0x561f81d1f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbd07ff5740 4 client.16008 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f81c71c00 con 0x561f81d1ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d1ce90 0x561f81d1f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbce67fc640 10 client.16008 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.544+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d1ce90 0x561f81d1f300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b2d6fea1b8c20f server_cookie=4151be4736009f1f in_seq=0 out_seq=0 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbce67fc640 1 -- 172.21.15.5:0/1905525934 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f81c71c00 con 0x561f81d1ce90 2024-04-22T13:27:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbce67fc640 10 client.16008 handle_command_reply: tid=0 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 client.16008 shutdown 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 msgr2=0x561f81d0a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0a960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80329c0 tx=0x7fbcf80095c0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 shutdown_connections 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d1ce90 0x561f81d1f300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d0a960 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d0a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbce67fc640 0 client.16008 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/1905525934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d09ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 >> 172.21.15.5:0/1905525934 conn(0x561f81cf9cb0 msgr2=0x561f81cfba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 shutdown_connections 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 1 -- 172.21.15.5:0/1905525934 wait complete. 2024-04-22T13:27:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.545+0000 7fbd07ff5740 20 client.16008 trim_cache size 0 max 16384 2024-04-22T13:27:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 Processor -- start 2024-04-22T13:27:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 -- start start 2024-04-22T13:27:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81d0c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d18a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d24bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001ab960 con 0x561f81d0ce90 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001ad4a0 con 0x561f81cffbd0 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001ac030 con 0x561f81cff2b0 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d24bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d24bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40616/0 (socket says 172.21.15.5:40616) 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3817001202 learned_addr learned my addr 172.21.15.5:0/3817001202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d18a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3817001202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 msgr2=0x561f81d0c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81d0c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.563+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81d0c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3817001202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 msgr2=0x561f81d18a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d18a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3817001202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f81cffa00 con 0x561f81d0ce90 2024-04-22T13:27:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81d0c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d24bd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8032af0 tx=0x7fbcf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee033c0174a7e336 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcf4ff9640 1 -- 172.21.15.5:0/3817001202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8064070 con 0x561f81d0ce90 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcf4ff9640 1 -- 172.21.15.5:0/3817001202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf8034b10 con 0x561f81d0ce90 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.564+0000 7fbcf4ff9640 1 -- 172.21.15.5:0/3817001202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8050070 con 0x561f81d0ce90 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3817001202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 msgr2=0x561f81d24bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d24bd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8032af0 tx=0x7fbcf8009bd0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3817001202 shutdown_connections 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d24bd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cffbd0 0x561f81d18a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3817001202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cff2b0 0x561f81d0c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3817001202 >> 172.21.15.5:0/3817001202 conn(0x561f81cf9cb0 msgr2=0x561f81cfb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3817001202 shutdown_connections 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3817001202 wait complete. 2024-04-22T13:27:49.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- start start 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 Processor -- start 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d06bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d07670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001acdd0 con 0x561f81d0ce90 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001ac700 con 0x561f81cff2b0 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001a9960 con 0x561f81cffbd0 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbd07ff5740 10 client.0 _ll_register_callbacks cb 0x7fbcf8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.565+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d07670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d07670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40618/0 (socket says 172.21.15.5:40618) 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcfeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d06bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3697879385 learned_addr learned my addr 172.21.15.5:0/3697879385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3697879385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 msgr2=0x561f81d07130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d07130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 msgr2=0x561f81d06bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d06bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 -- 172.21.15.5:0/3697879385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f81d82440 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcf7fff640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbcff7fe640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d07670 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80329c0 tx=0x7fbcf8032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ec4f91af06caa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbce67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbce67fc640 1 -- 172.21.15.5:0/3697879385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8064070 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbce67fc640 1 -- 172.21.15.5:0/3697879385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbcf8034440 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf8034b10 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbce67fc640 1 -- 172.21.15.5:0/3697879385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbcf8070050 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbd07ff5740 10 client.16014 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f81d836a0 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.566+0000 7fbce67fc640 1 -- 172.21.15.5:0/3697879385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf8034b10 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbce67fc640 1 -- 172.21.15.5:0/3697879385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbcf803f040 con 0x561f81d0ce90 2024-04-22T13:27:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 20 client.16014 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d83ca0 0x561f81d860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 4 client.16014 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f81c6ba70 con 0x561f81d83ca0 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d83ca0 0x561f81d860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbcfeffd640 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d83ca0 0x561f81d860c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c29a00cd100eab server_cookie=e1ad61a3f5189d62 in_seq=0 out_seq=0 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbce67fc640 10 client.16014 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbce67fc640 1 -- 172.21.15.5:0/3697879385 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561f81c6ba70 con 0x561f81d83ca0 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbce67fc640 10 client.16014 handle_command_reply: tid=0 2024-04-22T13:27:49.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.567+0000 7fbd07ff5740 1 client.16014 shutdown 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 msgr2=0x561f81d07670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d07670 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80329c0 tx=0x7fbcf8032ef0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 shutdown_connections 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561f81d83ca0 0x561f81d860c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f81d0ce90 0x561f81d07670 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561f81cffbd0 0x561f81d07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbce67fc640 0 client.16014 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 --2- 172.21.15.5:0/3697879385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561f81cff2b0 0x561f81d06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 >> 172.21.15.5:0/3697879385 conn(0x561f81cf9cb0 msgr2=0x561f81d09ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 shutdown_connections 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 1 -- 172.21.15.5:0/3697879385 wait complete. 2024-04-22T13:27:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd07ff5740 20 client.16014 trim_cache size 0 max 16384 2024-04-22T13:27:49.572 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:49.572 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: "a968198a-9549-4cae-b1a0-f2f6abcd7104": { 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a968198a-9549-4cae-b1a0-f2f6abcd7104", 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:49.573 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcec0a3e40 msgr2=0x7fbcec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcec0a3e40 0x7fbcec0a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8005cc0 tx=0x7fbcf8005c50 comp rx=0 tx=0).stop 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 msgr2=0x7fbd00198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd00198c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbce800d740 tx=0x7fbce800dc00 comp rx=0 tx=0).stop 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 shutdown_connections 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd00107250 0x7fbd00198c40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbcec0a3e40 0x7fbcec0a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0019cfc0 0x7fbd0019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 --2- 172.21.15.5:0/332034168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd00102180 0x7fbd00198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 >> 172.21.15.5:0/332034168 conn(0x7fbd000fda00 msgr2=0x7fbd000ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 shutdown_connections 2024-04-22T13:27:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:49.568+0000 7fbd05406640 1 -- 172.21.15.5:0/332034168 wait complete. 2024-04-22T13:27:49.576 INFO:tasks.cephfs.filesystem:scrub status for tag:a968198a-9549-4cae-b1a0-f2f6abcd7104 - {'path': '/', 'tag': 'a968198a-9549-4cae-b1a0-f2f6abcd7104', 'options': 'recursive,force'} 2024-04-22T13:27:50.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:49 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:27:50.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:49 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:50.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:49 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:27:50.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:49 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:50.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:49 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:27:50.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:49 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:50.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:49 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:27:50.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:49 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:50.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:49 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:27:50.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:49 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:50.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:49 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:27:50.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:49 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:50.865 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:50.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 -- 172.21.15.5:0/556511621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be006da20 msgr2=0x7f7be006de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:50.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 --2- 172.21.15.5:0/556511621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be006da20 0x7f7be006de00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009900 tx=0x7f7bd002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 -- 172.21.15.5:0/556511621 shutdown_connections 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 --2- 172.21.15.5:0/556511621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be011b5f0 0x7f7be011d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 --2- 172.21.15.5:0/556511621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be006e340 0x7f7be011b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 --2- 172.21.15.5:0/556511621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be006da20 0x7f7be006de00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 -- 172.21.15.5:0/556511621 >> 172.21.15.5:0/556511621 conn(0x7f7be006d090 msgr2=0x7f7be010af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 -- 172.21.15.5:0/556511621 shutdown_connections 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.988+0000 7f7be8062640 1 -- 172.21.15.5:0/556511621 wait complete. 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 Processor -- start 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 -- start start 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be006da20 0x7f7be01a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:50.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be006e340 0x7f7be01a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 0x7f7be01abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0120f70 con 0x7f7be011b5f0 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0120df0 con 0x7f7be006da20 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be8062640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01210f0 con 0x7f7be006e340 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 0x7f7be01abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 0x7f7be01abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40644/0 (socket says 172.21.15.5:40644) 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 -- 172.21.15.5:0/2389098568 learned_addr learned my addr 172.21.15.5:0/2389098568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be55fd640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be006e340 0x7f7be01a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be5dfe640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be006da20 0x7f7be01a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 -- 172.21.15.5:0/2389098568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be006e340 msgr2=0x7f7be01a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be006e340 0x7f7be01a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 -- 172.21.15.5:0/2389098568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be006da20 msgr2=0x7f7be01a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.989+0000 7f7be65ff640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be006da20 0x7f7be01a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:50.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7be65ff640 1 -- 172.21.15.5:0/2389098568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01ac4c0 con 0x7f7be011b5f0 2024-04-22T13:27:50.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7be65ff640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 0x7f7be01abd90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800eaa0 tx=0x7f7bd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:50.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800cdb0 con 0x7f7be011b5f0 2024-04-22T13:27:50.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01ac7b0 con 0x7f7be011b5f0 2024-04-22T13:27:50.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd8005020 con 0x7f7be011b5f0 2024-04-22T13:27:50.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80116d0 con 0x7f7be011b5f0 2024-04-22T13:27:50.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.990+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01b4040 con 0x7f7be011b5f0 2024-04-22T13:27:50.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.991+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be0068e00 con 0x7f7be011b5f0 2024-04-22T13:27:50.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.992+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f7bd80051d0 con 0x7f7be011b5f0 2024-04-22T13:27:50.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.993+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7bcc0a3e40 0x7f7bcc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:50.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.993+0000 7f7be5dfe640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7bcc0a3e40 0x7f7bcc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:50.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.993+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7bd80d0b30 con 0x7f7be011b5f0 2024-04-22T13:27:50.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.993+0000 7f7be5dfe640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7bcc0a3e40 0x7f7bcc0a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006010 tx=0x7f7bd0005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:50.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:50.998+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bd8016030 con 0x7f7be011b5f0 2024-04-22T13:27:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.223+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7be002d5c0 con 0x7f7be011b5f0 2024-04-22T13:27:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.224+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/2389098568 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f7bd809c960 con 0x7f7be011b5f0 2024-04-22T13:27:51.221 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:27:51.221 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:27:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.225+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7bcc0a3e40 msgr2=0x7f7bcc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.225+0000 7f7be8062640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7bcc0a3e40 0x7f7bcc0a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006010 tx=0x7f7bd0005e40 comp rx=0 tx=0).stop 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.225+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 msgr2=0x7f7be01abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.225+0000 7f7be8062640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 0x7f7be01abd90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800eaa0 tx=0x7f7bd800ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.225+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 shutdown_connections 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.225+0000 7f7be8062640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be011b5f0 0x7f7be01abd90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.226+0000 7f7be8062640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7bcc0a3e40 0x7f7bcc0a6300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.226+0000 7f7be8062640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be006e340 0x7f7be01a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.226+0000 7f7be8062640 1 --2- 172.21.15.5:0/2389098568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7be006da20 0x7f7be01a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.226+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 >> 172.21.15.5:0/2389098568 conn(0x7f7be006d090 msgr2=0x7f7be011c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.226+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 shutdown_connections 2024-04-22T13:27:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.226+0000 7f7be8062640 1 -- 172.21.15.5:0/2389098568 wait complete. 2024-04-22T13:27:51.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:50 smithi005 ceph-mon[28006]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-22T13:27:51.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:50 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:27:51.234 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db --set-id 4a1cd248 --release --await' 2024-04-22T13:27:51.234 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db --set-id 4a1cd248 --release --await 2024-04-22T13:27:51.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:50 smithi049 ceph-mon[31379]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-22T13:27:51.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:50 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:27:51.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:50 smithi107 ceph-mon[31859]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-22T13:27:51.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:50 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:27:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 -- 172.21.15.5:0/3391553118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 msgr2=0x7fded810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 --2- 172.21.15.5:0/3391553118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded810bfc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009900 tx=0x7fdec002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 -- 172.21.15.5:0/3391553118 shutdown_connections 2024-04-22T13:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 --2- 172.21.15.5:0/3391553118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded810bfc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 --2- 172.21.15.5:0/3391553118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fded810c500 0x7fded810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 --2- 172.21.15.5:0/3391553118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded81031c0 0x7fded81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 -- 172.21.15.5:0/3391553118 >> 172.21.15.5:0/3391553118 conn(0x7fded8078000 msgr2=0x7fded8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 -- 172.21.15.5:0/3391553118 shutdown_connections 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 -- 172.21.15.5:0/3391553118 wait complete. 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 Processor -- start 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.346+0000 7fdede64e640 1 -- start start 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fded81031c0 0x7fded81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded810c500 0x7fded819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fded81121c0 con 0x7fded8103ae0 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fded8112040 con 0x7fded810c500 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fded8112340 con 0x7fded81031c0 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded81a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40668/0 (socket says 172.21.15.5:40668) 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 -- 172.21.15.5:0/1321104155 learned_addr learned my addr 172.21.15.5:0/1321104155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded7fff640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fded81031c0 0x7fded81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 -- 172.21.15.5:0/1321104155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fded81031c0 msgr2=0x7fded81a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fded81031c0 0x7fded81a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 -- 172.21.15.5:0/1321104155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded810c500 msgr2=0x7fded819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded810c500 0x7fded819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 -- 172.21.15.5:0/1321104155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded819b000 con 0x7fded8103ae0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded77fe640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded81a0bc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec002c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded57fa640 1 -- 172.21.15.5:0/1321104155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0043020 con 0x7fded8103ae0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded819b290 con 0x7fded8103ae0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded57fa640 1 -- 172.21.15.5:0/1321104155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec002dc60 con 0x7fded8103ae0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fded57fa640 1 -- 172.21.15.5:0/1321104155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00039e0 con 0x7fded8103ae0 2024-04-22T13:27:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.347+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded81a7600 con 0x7fded8103ae0 2024-04-22T13:27:51.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.349+0000 7fded57fa640 1 -- 172.21.15.5:0/1321104155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fdec0046020 con 0x7fded8103ae0 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 Processor -- start 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 -- start start 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42be9f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42bea82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beaac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b42be04500 con 0x55b42be9fc30 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b42bdaa6a0 con 0x55b42bea8810 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdee1240740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b42be004c0 con 0x55b42be9f310 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42bea82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42bea82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40680/0 (socket says 172.21.15.5:40680) 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 -- 172.21.15.5:0/1108770751 learned_addr learned my addr 172.21.15.5:0/1108770751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded57fa640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdec40a3e40 0x7fdec40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42be9f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdec40a3e40 0x7fdec40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded57fa640 1 -- 172.21.15.5:0/1321104155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fdec00f29b0 con 0x7fded8103ae0 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 -- 172.21.15.5:0/1108770751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 msgr2=0x55b42be9f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42be9f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded77fe640 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beaac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 -- 172.21.15.5:0/1108770751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 msgr2=0x55b42beaac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beaac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.350+0000 7fded7fff640 1 -- 172.21.15.5:0/1108770751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b42bdc95e0 con 0x55b42be9fc30 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fded7fff640 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42bea82d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009cf0 tx=0x7fdec8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972f4926661c3f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdec40a3e40 0x7fdec40a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fded0005cc0 tx=0x7fded0005c10 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fdece7fc640 1 -- 172.21.15.5:0/1108770751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8004230 con 0x55b42be9fc30 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fdece7fc640 1 -- 172.21.15.5:0/1108770751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec80043d0 con 0x55b42be9fc30 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fdece7fc640 1 -- 172.21.15.5:0/1108770751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8019880 con 0x55b42be9fc30 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fdee1240740 1 -- 172.21.15.5:0/1108770751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 msgr2=0x55b42bea82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.351+0000 7fdee1240740 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42bea82d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009cf0 tx=0x7fdec8011810 comp rx=0 tx=0).stop 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- 172.21.15.5:0/1108770751 shutdown_connections 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42bea82d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beaac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 --2- 172.21.15.5:0/1108770751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42be9f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- 172.21.15.5:0/1108770751 >> 172.21.15.5:0/1108770751 conn(0x55b42be99a40 msgr2=0x55b42be99e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- 172.21.15.5:0/1108770751 shutdown_connections 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- 172.21.15.5:0/1108770751 wait complete. 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- start start 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 Processor -- start 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42bea8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42bea55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bea5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b42beab600 con 0x55b42bea8810 2024-04-22T13:27:51.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b42bdbda60 con 0x55b42be9fc30 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b42beab780 con 0x55b42be9f310 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.352+0000 7fdee1240740 10 client.0 _ll_register_callbacks cb 0x55b42bb16110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bea5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bea5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40682/0 (socket says 172.21.15.5:40682) 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 -- 172.21.15.5:0/1609186946 learned_addr learned my addr 172.21.15.5:0/1609186946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fded77fe640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42bea55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 -- 172.21.15.5:0/1609186946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 msgr2=0x55b42bea8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fded7fff640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42bea8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42bea8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 -- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 msgr2=0x55b42bea55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42bea55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 -- 172.21.15.5:0/1609186946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b42bb16920 con 0x55b42bea8810 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bea5d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fded0012810 tx=0x7fded0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4050fcde0c5392 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdeb77fe640 1 -- 172.21.15.5:0/1609186946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0003b70 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec80043d0 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdeb77fe640 1 -- 172.21.15.5:0/1609186946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fded0003d10 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdeb77fe640 1 -- 172.21.15.5:0/1609186946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fded001b870 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdee1240740 10 client.16032 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b42bbe2d30 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.353+0000 7fdeb77fe640 1 -- 172.21.15.5:0/1609186946 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec80043d0 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdeb77fe640 1 -- 172.21.15.5:0/1609186946 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fded000f040 con 0x55b42bea8810 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdee1240740 10 client.16032 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdee1240740 10 client.16032 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdee1240740 20 client.16032 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdee1240740 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42beb8920 0x55b42bebad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdee1240740 4 client.16032 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b42bdd7a10 con 0x55b42beb8920 2024-04-22T13:27:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fded7fff640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42beb8920 0x55b42bebad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fded7fff640 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42beb8920 0x55b42bebad50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7fa5356473e5aa server_cookie=63ea341f3a609cb1 in_seq=0 out_seq=0 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.354+0000 7fdeb77fe640 10 client.16032 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdeb77fe640 1 -- 172.21.15.5:0/1609186946 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b42bdd7a10 con 0x55b42beb8920 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdeb77fe640 10 client.16032 handle_command_reply: tid=0 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 client.16032 shutdown 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 msgr2=0x55b42bea5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bea5d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fded0012810 tx=0x7fded0004a40 comp rx=0 tx=0).stop 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 shutdown_connections 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42beb8920 0x55b42bebad50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bea5d20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdeb77fe640 0 client.16032 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42bea55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 --2- 172.21.15.5:0/1609186946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42bea8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 >> 172.21.15.5:0/1609186946 conn(0x55b42be99a40 msgr2=0x55b42be9b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 shutdown_connections 2024-04-22T13:27:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.355+0000 7fdee1240740 1 -- 172.21.15.5:0/1609186946 wait complete. 2024-04-22T13:27:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.356+0000 7fdee1240740 20 client.16032 trim_cache size 0 max 16384 2024-04-22T13:27:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.373+0000 7fdee1240740 1 Processor -- start 2024-04-22T13:27:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 -- start start 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42be9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beb09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fded81afd10 con 0x55b42be9fc30 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b1850 con 0x55b42bea8810 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdee1240740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b03e0 con 0x55b42be9f310 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42be9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42be9b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40684/0 (socket says 172.21.15.5:40684) 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 -- 172.21.15.5:0/3205576741 learned_addr learned my addr 172.21.15.5:0/3205576741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded7fff640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 -- 172.21.15.5:0/3205576741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 msgr2=0x55b42beb44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beb09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beb44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 -- 172.21.15.5:0/3205576741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 msgr2=0x55b42beb09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beb09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 -- 172.21.15.5:0/3205576741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b42beb7570 con 0x55b42be9fc30 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded7fff640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beb44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fded77fe640 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42be9b4b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdec00f0730 tx=0x7fdec00a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202569d1eff76f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdecdffb640 1 -- 172.21.15.5:0/3205576741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00f2e60 con 0x55b42be9fc30 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.374+0000 7fdecdffb640 1 -- 172.21.15.5:0/3205576741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec002de10 con 0x55b42be9fc30 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdecdffb640 1 -- 172.21.15.5:0/3205576741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00baeb0 con 0x55b42be9fc30 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 -- 172.21.15.5:0/3205576741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 msgr2=0x55b42be9b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42be9b4b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdec00f0730 tx=0x7fdec00a9040 comp rx=0 tx=0).stop 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 -- 172.21.15.5:0/3205576741 shutdown_connections 2024-04-22T13:27:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42be9fc30 0x55b42be9b4b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42bea8810 0x55b42beb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 --2- 172.21.15.5:0/3205576741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beb44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 -- 172.21.15.5:0/3205576741 >> 172.21.15.5:0/3205576741 conn(0x55b42be99a40 msgr2=0x55b42be9ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 -- 172.21.15.5:0/3205576741 shutdown_connections 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.375+0000 7fdee1240740 1 -- 172.21.15.5:0/3205576741 wait complete. 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 -- start start 2024-04-22T13:27:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 Processor -- start 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42beabed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bf1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b1180 con 0x55b42bea8810 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b0ab0 con 0x55b42be9fc30 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fded81adcf0 con 0x55b42be9f310 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdee1240740 10 client.0 _ll_register_callbacks cb 0x7fded806a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bf1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bf1ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40686/0 (socket says 172.21.15.5:40686) 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 -- 172.21.15.5:0/4160521115 learned_addr learned my addr 172.21.15.5:0/4160521115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fded7fff640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 -- 172.21.15.5:0/4160521115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 msgr2=0x55b42beab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fded77fe640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42beabed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 -- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 msgr2=0x55b42beabed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42beabed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fdedcbeb640 1 -- 172.21.15.5:0/4160521115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b42bf1f3d0 con 0x55b42bea8810 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fded7fff640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.376+0000 7fded77fe640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42beabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdedcbeb640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bf1ee90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fded0011fc0 tx=0x7fded0012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db8e874566887f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdeb77fe640 1 -- 172.21.15.5:0/4160521115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded003b020 con 0x55b42bea8810 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdeb77fe640 1 -- 172.21.15.5:0/4160521115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fded0004d20 con 0x55b42bea8810 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b42be11f20 con 0x55b42bea8810 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdeb77fe640 1 -- 172.21.15.5:0/4160521115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fded0032040 con 0x55b42bea8810 2024-04-22T13:27:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 10 client.16038 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b42bf20570 con 0x55b42bea8810 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdeb77fe640 1 -- 172.21.15.5:0/4160521115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b42be11f20 con 0x55b42bea8810 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdeb77fe640 1 -- 172.21.15.5:0/4160521115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fded0005560 con 0x55b42bea8810 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 10 client.16038 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 10 client.16038 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 20 client.16038 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42bf20d40 0x55b42bf23160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 4 client.16038 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4a1cd248", "release": true, "await": true}] 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.377+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4a1cd248", "release": true, "await": true}) -- 0x55b42be0bd90 con 0x55b42bf20d40 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.378+0000 7fded7fff640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42bf20d40 0x55b42bf23160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.378+0000 7fdeb77fe640 10 client.16038 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.378+0000 7fded7fff640 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42bf20d40 0x55b42bf23160 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51555d6f317e291c server_cookie=6bfe393ecec93e95 in_seq=0 out_seq=0 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdeb77fe640 1 -- 172.21.15.5:0/4160521115 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55b42be0bd90 con 0x55b42bf20d40 2024-04-22T13:27:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdeb77fe640 10 client.16038 handle_command_reply: tid=0 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 client.16038 shutdown 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 msgr2=0x55b42bf1ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bf1ee90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fded0011fc0 tx=0x7fded0012500 comp rx=0 tx=0).stop 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 shutdown_connections 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b42bf20d40 0x55b42bf23160 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b42bea8810 0x55b42bf1ee90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b42be9fc30 0x55b42beabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 --2- 172.21.15.5:0/4160521115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b42be9f310 0x55b42beab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdeb77fe640 0 client.16038 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 >> 172.21.15.5:0/4160521115 conn(0x55b42be99a40 msgr2=0x55b42be9b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 shutdown_connections 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.381+0000 7fdee1240740 1 -- 172.21.15.5:0/4160521115 wait complete. 2024-04-22T13:27:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdee1240740 20 client.16038 trim_cache size 0 max 16384 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 4, 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "4a1cd248": { 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4, 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:27:51.383 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 70.0, 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:27:51.384 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:27:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdec40a3e40 msgr2=0x7fdec40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdec40a3e40 0x7fdec40a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fded0005cc0 tx=0x7fded0005c10 comp rx=0 tx=0).stop 2024-04-22T13:27:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 msgr2=0x7fded81a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded81a0bc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec002c760 comp rx=0 tx=0).stop 2024-04-22T13:27:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 shutdown_connections 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded8103ae0 0x7fded81a0bc0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdec40a3e40 0x7fdec40a6300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fded810c500 0x7fded819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 --2- 172.21.15.5:0/1321104155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fded81031c0 0x7fded81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 >> 172.21.15.5:0/1321104155 conn(0x7fded8078000 msgr2=0x7fded8108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 shutdown_connections 2024-04-22T13:27:51.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:27:51.382+0000 7fdede64e640 1 -- 172.21.15.5:0/1321104155 wait complete. 2024-04-22T13:27:51.390 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: 4a1cd248, seconds elapsed: 11.1 2024-04-22T13:27:52.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:51 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2389098568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:52.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:51 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2389098568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:52.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:51 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2389098568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:53.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:52 smithi049 ceph-mon[31379]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-22T13:27:53.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:52 smithi107 ceph-mon[31859]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-22T13:27:53.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:52 smithi005 ceph-mon[28006]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-22T13:27:55.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:54 smithi049 ceph-mon[31379]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-22T13:27:55.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:54 smithi107 ceph-mon[31859]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-22T13:27:55.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:54 smithi005 ceph-mon[28006]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-22T13:27:57.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:56 smithi049 ceph-mon[31379]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-04-22T13:27:57.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:56 smithi107 ceph-mon[31859]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-04-22T13:27:57.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:56 smithi005 ceph-mon[28006]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 -- 172.21.15.5:0/3541523975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441010a0 msgr2=0x7f9844101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 --2- 172.21.15.5:0/3541523975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441010a0 0x7f9844101480 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f982c009900 tx=0x7f982c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 -- 172.21.15.5:0/3541523975 shutdown_connections 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 --2- 172.21.15.5:0/3541523975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9844106170 0x7f984410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 --2- 172.21.15.5:0/3541523975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101a50 0x7f9844105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 --2- 172.21.15.5:0/3541523975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441010a0 0x7f9844101480 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 -- 172.21.15.5:0/3541523975 >> 172.21.15.5:0/3541523975 conn(0x7f9844077f70 msgr2=0x7f9844078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 -- 172.21.15.5:0/3541523975 shutdown_connections 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.711+0000 7f984ae99640 1 -- 172.21.15.5:0/3541523975 wait complete. 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 Processor -- start 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 -- start start 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98441010a0 0x7f984419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844106170 0x7f984419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 0x7f9844198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9844111130 con 0x7f98441985f0 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9844110fb0 con 0x7f9844106170 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f984ae99640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98441112b0 con 0x7f98441010a0 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 0x7f9844198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 0x7f9844198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40708/0 (socket says 172.21.15.5:40708) 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 -- 172.21.15.5:0/4132819048 learned_addr learned my addr 172.21.15.5:0/4132819048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9848c35640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98441010a0 0x7f984419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9843fff640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844106170 0x7f984419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 -- 172.21.15.5:0/4132819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98441010a0 msgr2=0x7f984419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98441010a0 0x7f984419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 -- 172.21.15.5:0/4132819048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844106170 msgr2=0x7f984419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844106170 0x7f984419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9849436640 1 -- 172.21.15.5:0/4132819048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9844199110 con 0x7f98441985f0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.712+0000 7f9848c35640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98441010a0 0x7f984419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.713+0000 7f9849436640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 0x7f9844198aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d750 tx=0x7f983c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.713+0000 7f9841ffb640 1 -- 172.21.15.5:0/4132819048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c017070 con 0x7f98441985f0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.713+0000 7f9841ffb640 1 -- 172.21.15.5:0/4132819048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c0054d0 con 0x7f98441985f0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.713+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9844199400 con 0x7f98441985f0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.713+0000 7f9841ffb640 1 -- 172.21.15.5:0/4132819048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c004c60 con 0x7f98441985f0 2024-04-22T13:27:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.713+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a5650 con 0x7f98441985f0 2024-04-22T13:27:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.715+0000 7f9841ffb640 1 -- 172.21.15.5:0/4132819048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f983c009bd0 con 0x7f98441985f0 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 Processor -- start 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 -- start start 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9841ffb640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98300a3eb0 0x7f98300a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 0x55dbc2217620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2217fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22210e0 0x55dbc22234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbc217c8d0 con 0x55dbc22188d0 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dbc2119e80 con 0x55dbc22210e0 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f984da85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dbc2178890 con 0x55dbc2217b60 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9841ffb640 1 -- 172.21.15.5:0/4132819048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f983c0d0400 con 0x7f98441985f0 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9843fff640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98300a3eb0 0x7f98300a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9848c35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 0x55dbc2217620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9848c35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 0x55dbc2217620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40710/0 (socket says 172.21.15.5:40710) 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9848c35640 1 -- 172.21.15.5:0/1060019586 learned_addr learned my addr 172.21.15.5:0/1060019586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9843fff640 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2217fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.716+0000 7f9849436640 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22210e0 0x55dbc22234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9848c35640 1 -- 172.21.15.5:0/1060019586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 msgr2=0x55dbc2217fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9848c35640 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2217fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9848c35640 1 -- 172.21.15.5:0/1060019586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22210e0 msgr2=0x55dbc22234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9848c35640 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22210e0 0x55dbc22234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9848c35640 1 -- 172.21.15.5:0/1060019586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc213b320 con 0x55dbc22188d0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9843fff640 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2217fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9843fff640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98300a3eb0 0x7f98300a6370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9844105cc0 tx=0x7f9834015ac0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f9848c35640 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 0x55dbc2217620 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1bdc9c4df2372c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f983affd640 1 -- 172.21.15.5:0/1060019586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c044020 con 0x55dbc22188d0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f983affd640 1 -- 172.21.15.5:0/1060019586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f982c02cc90 con 0x55dbc22188d0 2024-04-22T13:27:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.717+0000 7f983affd640 1 -- 172.21.15.5:0/1060019586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c0046c0 con 0x55dbc22188d0 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 -- 172.21.15.5:0/1060019586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 msgr2=0x55dbc2217620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 0x55dbc2217620 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c02c650 comp rx=0 tx=0).stop 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 -- 172.21.15.5:0/1060019586 shutdown_connections 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22210e0 0x55dbc22234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2217fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 --2- 172.21.15.5:0/1060019586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22188d0 0x55dbc2217620 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 -- 172.21.15.5:0/1060019586 >> 172.21.15.5:0/1060019586 conn(0x55dbc21dec90 msgr2=0x55dbc2212850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 -- 172.21.15.5:0/1060019586 shutdown_connections 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 -- 172.21.15.5:0/1060019586 wait complete. 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 -- start start 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.718+0000 7f984da85740 1 Processor -- start 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc22209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbc2223db0 con 0x55dbc22210e0 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dbc2104eb0 con 0x55dbc22188d0 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dbc2223f30 con 0x55dbc2217b60 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f984da85740 10 client.0 _ll_register_callbacks cb 0x55dbc2131b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40720/0 (socket says 172.21.15.5:40720) 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 -- 172.21.15.5:0/3271344481 learned_addr learned my addr 172.21.15.5:0/3271344481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9843fff640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9848c35640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc22209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 -- 172.21.15.5:0/3271344481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 msgr2=0x55dbc22209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc22209d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 -- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 msgr2=0x55dbc221ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 -- 172.21.15.5:0/3271344481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc1e8f210 con 0x55dbc22210e0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9849436640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221e330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f983c09c110 tx=0x7f983c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e711216d8f12714e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.719+0000 7f9823fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f9823fff640 1 -- 172.21.15.5:0/3271344481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0a0530 con 0x55dbc22210e0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f9823fff640 1 -- 172.21.15.5:0/3271344481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f983c018d20 con 0x55dbc22210e0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f982c02cc90 con 0x55dbc22210e0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f9823fff640 1 -- 172.21.15.5:0/3271344481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f983c0a4df0 con 0x55dbc22210e0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbc1f5b620 con 0x55dbc22210e0 2024-04-22T13:27:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f9823fff640 1 -- 172.21.15.5:0/3271344481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f982c02cc90 con 0x55dbc22210e0 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f9823fff640 1 -- 172.21.15.5:0/3271344481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f983c0a0160 con 0x55dbc22210e0 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 20 client.16050 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2231150 0x55dbc2233560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 4 client.16050 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbc2190880 con 0x55dbc2231150 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.720+0000 7f9848c35640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2231150 0x55dbc2233560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.721+0000 7f9848c35640 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2231150 0x55dbc2233560 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ccd99dd894d0b6 server_cookie=5008a91b910c957 in_seq=0 out_seq=0 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.721+0000 7f9823fff640 10 client.16050 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f9823fff640 1 -- 172.21.15.5:0/3271344481 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbc2190880 con 0x55dbc2231150 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f9823fff640 10 client.16050 handle_command_reply: tid=0 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 client.16050 shutdown 2024-04-22T13:27:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 msgr2=0x55dbc221e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221e330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f983c09c110 tx=0x7f983c005680 comp rx=0 tx=0).stop 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 shutdown_connections 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2231150 0x55dbc2233560 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221e330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f9823fff640 0 client.16050 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 --2- 172.21.15.5:0/3271344481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc22209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 >> 172.21.15.5:0/3271344481 conn(0x55dbc21dec90 msgr2=0x55dbc22145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 shutdown_connections 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 1 -- 172.21.15.5:0/3271344481 wait complete. 2024-04-22T13:27:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.722+0000 7f984da85740 20 client.16050 trim_cache size 0 max 16384 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 Processor -- start 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 -- start start 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 0x55dbc222ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc2217080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc22210e0 0x55dbc2238e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98441adb30 con 0x55dbc2217b60 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98441af670 con 0x55dbc22188d0 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f984da85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98441ae200 con 0x55dbc22210e0 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f9848c35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 0x55dbc222ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.737+0000 7f9848c35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 0x55dbc222ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40732/0 (socket says 172.21.15.5:40732) 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 -- 172.21.15.5:0/3242023611 learned_addr learned my addr 172.21.15.5:0/3242023611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9843fff640 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc2217080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9849436640 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc22210e0 0x55dbc2238e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 -- 172.21.15.5:0/3242023611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc22210e0 msgr2=0x55dbc2238e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc22210e0 0x55dbc2238e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 -- 172.21.15.5:0/3242023611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 msgr2=0x55dbc2217080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc2217080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 -- 172.21.15.5:0/3242023611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc2216e30 con 0x55dbc2217b60 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9849436640 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc22210e0 0x55dbc2238e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f9848c35640 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 0x55dbc222ccb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8ee2d83562ca6a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f983a7fc640 1 -- 172.21.15.5:0/3242023611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c044040 con 0x55dbc2217b60 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f983a7fc640 1 -- 172.21.15.5:0/3242023611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f982c004140 con 0x55dbc2217b60 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.738+0000 7f983a7fc640 1 -- 172.21.15.5:0/3242023611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c02cac0 con 0x55dbc2217b60 2024-04-22T13:27:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 -- 172.21.15.5:0/3242023611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 msgr2=0x55dbc222ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 0x55dbc222ccb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c02c650 comp rx=0 tx=0).stop 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 -- 172.21.15.5:0/3242023611 shutdown_connections 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc22210e0 0x55dbc2238e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc2217080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 --2- 172.21.15.5:0/3242023611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc2217b60 0x55dbc222ccb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 -- 172.21.15.5:0/3242023611 >> 172.21.15.5:0/3242023611 conn(0x55dbc21dec90 msgr2=0x55dbc2212380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 -- 172.21.15.5:0/3242023611 shutdown_connections 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 -- 172.21.15.5:0/3242023611 wait complete. 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 -- start start 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.739+0000 7f984da85740 1 Processor -- start 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2229570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98441aefa0 con 0x55dbc22210e0 2024-04-22T13:27:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98441ae8d0 con 0x55dbc22188d0 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98441abb30 con 0x55dbc2217b60 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 10 client.0 _ll_register_callbacks cb 0x7f983c09bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40734/0 (socket says 172.21.15.5:40734) 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 -- 172.21.15.5:0/281618397 learned_addr learned my addr 172.21.15.5:0/281618397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9843fff640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9848c35640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2229570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 -- 172.21.15.5:0/281618397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 msgr2=0x55dbc2229570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2229570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 -- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 msgr2=0x55dbc221a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 -- 172.21.15.5:0/281618397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc221b540 con 0x55dbc22210e0 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9843fff640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9849436640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221ad10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f983c0ce180 tx=0x7f983c098270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b282ee8099ef6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9823fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9823fff640 1 -- 172.21.15.5:0/281618397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c09bcc0 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f9823fff640 1 -- 172.21.15.5:0/281618397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f983c09b7d0 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.740+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f982c004140 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f9823fff640 1 -- 172.21.15.5:0/281618397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f983c0a02e0 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 10 client.16056 fetch_fsmap learned FSMap version 27 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbc2297640 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f9823fff640 1 -- 172.21.15.5:0/281618397 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f982c004140 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f9823fff640 1 -- 172.21.15.5:0/281618397 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f983c0a16c0 con 0x55dbc22210e0 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 10 client.16056 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 20 client.16056 populate_metadata read hostname 'smithi005' 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2297d90 0x55dbc229a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 4 client.16056 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:27:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbc211c3d0 con 0x55dbc2297d90 2024-04-22T13:27:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.741+0000 7f9848c35640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2297d90 0x55dbc229a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.742+0000 7f9848c35640 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2297d90 0x55dbc229a1b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5690c12b98b27877 server_cookie=2cb523359d613f1f in_seq=0 out_seq=0 2024-04-22T13:27:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.742+0000 7f9823fff640 10 client.16056 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.745+0000 7f9823fff640 1 -- 172.21.15.5:0/281618397 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dbc211c3d0 con 0x55dbc2297d90 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.745+0000 7f9823fff640 10 client.16056 handle_command_reply: tid=0 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.745+0000 7f984da85740 1 client.16056 shutdown 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 msgr2=0x55dbc221ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221ad10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f983c0ce180 tx=0x7f983c098270 comp rx=0 tx=0).stop 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 shutdown_connections 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbc2297d90 0x55dbc229a1b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbc22210e0 0x55dbc221ad10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbc22188d0 0x55dbc221a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 --2- 172.21.15.5:0/281618397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbc2217b60 0x55dbc2229570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f9823fff640 0 client.16056 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 >> 172.21.15.5:0/281618397 conn(0x55dbc21dec90 msgr2=0x55dbc221c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 shutdown_connections 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 1 -- 172.21.15.5:0/281618397 wait complete. 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984da85740 20 client.16056 trim_cache size 0 max 16384 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:27:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:27:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98300a3eb0 msgr2=0x7f98300a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984ae99640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98300a3eb0 0x7f98300a6370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9844105cc0 tx=0x7f9834015ac0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 msgr2=0x7f9844198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.746+0000 7f984ae99640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 0x7f9844198aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d750 tx=0x7f983c00dc10 comp rx=0 tx=0).stop 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 shutdown_connections 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441985f0 0x7f9844198aa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98300a3eb0 0x7f98300a6370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844106170 0x7f984419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 --2- 172.21.15.5:0/4132819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98441010a0 0x7f984419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 >> 172.21.15.5:0/4132819048 conn(0x7f9844077f70 msgr2=0x7f9844069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 shutdown_connections 2024-04-22T13:27:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:27:57.747+0000 7f984ae99640 1 -- 172.21.15.5:0/4132819048 wait complete. 2024-04-22T13:27:59.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:27:58 smithi049 ceph-mon[31379]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:27:59.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:27:58 smithi107 ceph-mon[31859]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:27:59.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:27:58 smithi005 ceph-mon[28006]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:28:00.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:28:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:28:00.527+0000 7f3c8742a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:28:00.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:28:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:28:00.527+0000 7f3c8742a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:28:00.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:28:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:28:00.527+0000 7f3c8742a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:28:00.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:28:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:28:00.527+0000 7f3c8742a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:28:00.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:28:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:28:00.527+0000 7f3c8742a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:28:01.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:00 smithi049 ceph-mon[31379]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:28:01.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:00 smithi107 ceph-mon[31859]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:28:01.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:00 smithi005 ceph-mon[28006]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:28:03.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:02 smithi049 ceph-mon[31379]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-22T13:28:03.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:02 smithi049 ceph-mon[31379]: mgrmap e25: x(active, since 52s), standbys: z, y 2024-04-22T13:28:03.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:02 smithi107 ceph-mon[31859]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-22T13:28:03.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:02 smithi107 ceph-mon[31859]: mgrmap e25: x(active, since 52s), standbys: z, y 2024-04-22T13:28:03.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:02 smithi005 ceph-mon[28006]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-22T13:28:03.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:02 smithi005 ceph-mon[28006]: mgrmap e25: x(active, since 52s), standbys: z, y 2024-04-22T13:28:05.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:04 smithi049 ceph-mon[31379]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-22T13:28:05.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:04 smithi107 ceph-mon[31859]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-22T13:28:05.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:04 smithi005 ceph-mon[28006]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-22T13:28:07.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:06 smithi049 ceph-mon[31379]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-22T13:28:07.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:06 smithi107 ceph-mon[31859]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-22T13:28:07.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:06 smithi005 ceph-mon[28006]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-22T13:28:09.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:08 smithi049 ceph-mon[31379]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:28:09.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:08 smithi107 ceph-mon[31859]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:28:09.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:08 smithi005 ceph-mon[28006]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:28:09.962 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-22T13:28:09.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.069+0000 7f8f189b2640 1 -- 172.21.15.5:0/2475440607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14101a20 msgr2=0x7f8f14105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.069+0000 7f8f189b2640 1 --2- 172.21.15.5:0/2475440607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14101a20 0x7f8f14105a60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009900 tx=0x7f8efc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.069+0000 7f8f189b2640 1 -- 172.21.15.5:0/2475440607 shutdown_connections 2024-04-22T13:28:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.069+0000 7f8f189b2640 1 --2- 172.21.15.5:0/2475440607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14101a20 0x7f8f14105a60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 --2- 172.21.15.5:0/2475440607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f14106140 0x7f8f1410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 --2- 172.21.15.5:0/2475440607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f14101070 0x7f8f14101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 -- 172.21.15.5:0/2475440607 >> 172.21.15.5:0/2475440607 conn(0x7f8f140771d0 msgr2=0x7f8f140775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 -- 172.21.15.5:0/2475440607 shutdown_connections 2024-04-22T13:28:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 -- 172.21.15.5:0/2475440607 wait complete. 2024-04-22T13:28:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 Processor -- start 2024-04-22T13:28:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.070+0000 7f8f189b2640 1 -- start start 2024-04-22T13:28:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f189b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f14101070 0x7f8f14199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f189b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 0x7f8f14199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f189b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f14199ff0 0x7f8f141a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f189b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14111060 con 0x7f8f14106140 2024-04-22T13:28:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f189b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14110ee0 con 0x7f8f14199ff0 2024-04-22T13:28:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f189b2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141111e0 con 0x7f8f14101070 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 0x7f8f14199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 0x7f8f14199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59304/0 (socket says 172.21.15.5:59304) 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 -- 172.21.15.5:0/247486930 learned_addr learned my addr 172.21.15.5:0/247486930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f12d9d640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f14199ff0 0x7f8f141a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f1259c640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f14101070 0x7f8f14199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 -- 172.21.15.5:0/247486930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f14101070 msgr2=0x7f8f14199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f14101070 0x7f8f14199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 -- 172.21.15.5:0/247486930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f14199ff0 msgr2=0x7f8f141a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f14199ff0 0x7f8f141a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 -- 172.21.15.5:0/247486930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f141a4d70 con 0x7f8f14106140 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f09d9b640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 0x7f8f14199a60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8efc036040 tx=0x7f8efc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc043020 con 0x7f8f14106140 2024-04-22T13:28:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.071+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8efc02cb70 con 0x7f8f14106140 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.072+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f141a5000 con 0x7f8f14106140 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.072+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003bf0 con 0x7f8f14106140 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.072+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a5550 con 0x7f8f14106140 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.072+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f14068e00 con 0x7f8f14106140 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.074+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f8efc046020 con 0x7f8f14106140 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.074+0000 7f8f0bfff640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f000a3ed0 0x7f8f000a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.074+0000 7f8f1259c640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f000a3ed0 0x7f8f000a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.074+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8efc0f2b50 con 0x7f8f14106140 2024-04-22T13:28:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.075+0000 7f8f1259c640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f000a3ed0 0x7f8f000a6390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f04005e10 tx=0x7f8f04005da0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.079+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8efc04d020 con 0x7f8f14106140 2024-04-22T13:28:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.305+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f14109370 con 0x7f8f14106140 2024-04-22T13:28:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.305+0000 7f8f0bfff640 1 -- 172.21.15.5:0/247486930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f8efc0bf0f0 con 0x7f8f14106140 2024-04-22T13:28:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:28:10.301 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:28:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:28:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f000a3ed0 msgr2=0x7f8f000a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f000a3ed0 0x7f8f000a6390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f04005e10 tx=0x7f8f04005da0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 msgr2=0x7f8f14199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 0x7f8f14199a60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8efc036040 tx=0x7f8efc0045e0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 shutdown_connections 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f14106140 0x7f8f14199a60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f000a3ed0 0x7f8f000a6390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f14199ff0 0x7f8f141a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 --2- 172.21.15.5:0/247486930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f14101070 0x7f8f14199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.307+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 >> 172.21.15.5:0/247486930 conn(0x7f8f140771d0 msgr2=0x7f8f14100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.308+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 shutdown_connections 2024-04-22T13:28:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.308+0000 7f8f189b2640 1 -- 172.21.15.5:0/247486930 wait complete. 2024-04-22T13:28:10.314 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db / --timeout 60 --expiration 70 --await' 2024-04-22T13:28:10.315 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db / --timeout 60 --expiration 70 --await 2024-04-22T13:28:10.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 -- 172.21.15.5:0/7796718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8108040 msgr2=0x7f44c8108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 --2- 172.21.15.5:0/7796718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8108040 0x7f44c8108420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009900 tx=0x7f44bc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:10.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 -- 172.21.15.5:0/7796718 shutdown_connections 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 --2- 172.21.15.5:0/7796718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c8069bf0 0x7f44c810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 --2- 172.21.15.5:0/7796718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c8069250 0x7f44c80696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 --2- 172.21.15.5:0/7796718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8108040 0x7f44c8108420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 -- 172.21.15.5:0/7796718 >> 172.21.15.5:0/7796718 conn(0x7f44c8077c40 msgr2=0x7f44c8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 -- 172.21.15.5:0/7796718 shutdown_connections 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 -- 172.21.15.5:0/7796718 wait complete. 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 Processor -- start 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.432+0000 7f44cebb2640 1 -- start start 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cebb2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c8069250 0x7f44c81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cebb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 0x7f44c8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cebb2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c8108040 0x7f44c819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cebb2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81121f0 con 0x7f44c8069bf0 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cebb2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8112070 con 0x7f44c8108040 2024-04-22T13:28:10.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cebb2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8112370 con 0x7f44c8069250 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 0x7f44c8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 0x7f44c8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59324/0 (socket says 172.21.15.5:59324) 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 -- 172.21.15.5:0/1800508717 learned_addr learned my addr 172.21.15.5:0/1800508717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cc94e640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c8069250 0x7f44c81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44cd14f640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c8108040 0x7f44c819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 -- 172.21.15.5:0/1800508717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c8069250 msgr2=0x7f44c81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c8069250 0x7f44c81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 -- 172.21.15.5:0/1800508717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c8108040 msgr2=0x7f44c819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c8108040 0x7f44c819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 -- 172.21.15.5:0/1800508717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c819d6a0 con 0x7f44c8069bf0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c7fff640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 0x7f44c8198bf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f44b400d950 tx=0x7f44b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.433+0000 7f44c5ffb640 1 -- 172.21.15.5:0/1800508717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4017030 con 0x7f44c8069bf0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.434+0000 7f44c5ffb640 1 -- 172.21.15.5:0/1800508717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b4004020 con 0x7f44c8069bf0 2024-04-22T13:28:10.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.434+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c819d990 con 0x7f44c8069bf0 2024-04-22T13:28:10.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.434+0000 7f44c5ffb640 1 -- 172.21.15.5:0/1800508717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40047a0 con 0x7f44c8069bf0 2024-04-22T13:28:10.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.434+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c81a5220 con 0x7f44c8069bf0 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44c5ffb640 1 -- 172.21.15.5:0/1800508717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f44b4009a30 con 0x7f44c8069bf0 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 Processor -- start 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 -- start start 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdb6f0 0x562480fdbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44c5ffb640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f44b80a3f50 0x7f44b80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480feb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562480f3c0b0 con 0x562480fdc010 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562480eed680 con 0x562480fdb6f0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.436+0000 7f44d179f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562480ef9c60 con 0x562480fe92d0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44cc94e640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f44b80a3f50 0x7f44b80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59338/0 (socket says 172.21.15.5:59338) 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 -- 172.21.15.5:0/3541592733 learned_addr learned my addr 172.21.15.5:0/3541592733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44cd14f640 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480feb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c5ffb640 1 -- 172.21.15.5:0/1800508717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f44b40d0450 con 0x7f44c8069bf0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44cd14f640 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdb6f0 0x562480fdbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 -- 172.21.15.5:0/3541592733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 msgr2=0x562480feb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480feb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 -- 172.21.15.5:0/3541592733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdb6f0 msgr2=0x562480fdbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdb6f0 0x562480fdbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 -- 172.21.15.5:0/3541592733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562480d1ec90 con 0x562480fdc010 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44c7fff640 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe8d90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f44b409c170 tx=0x7f44b40207f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba4d6bf978c1a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.437+0000 7f44cc94e640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f44b80a3f50 0x7f44b80a6410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005ab0 tx=0x7f44bc005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44b2ffd640 1 -- 172.21.15.5:0/3541592733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40af480 con 0x562480fdc010 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44b2ffd640 1 -- 172.21.15.5:0/3541592733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b409c8e0 con 0x562480fdc010 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44b2ffd640 1 -- 172.21.15.5:0/3541592733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40af2a0 con 0x562480fdc010 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 -- 172.21.15.5:0/3541592733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 msgr2=0x562480fe8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe8d90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f44b409c170 tx=0x7f44b40207f0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 -- 172.21.15.5:0/3541592733 shutdown_connections 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe8d90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480feb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 --2- 172.21.15.5:0/3541592733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdb6f0 0x562480fdbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.438+0000 7f44d179f740 1 -- 172.21.15.5:0/3541592733 >> 172.21.15.5:0/3541592733 conn(0x562480fd60f0 msgr2=0x562480fd64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 -- 172.21.15.5:0/3541592733 shutdown_connections 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 -- 172.21.15.5:0/3541592733 wait complete. 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 -- start start 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 Processor -- start 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fdb6f0 0x562480fe62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fe92d0 0x562480fe6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562480f3f400 con 0x562480fdc010 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562480efb840 con 0x562480fe92d0 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562480f44720 con 0x562480fdb6f0 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44d179f740 10 client.0 _ll_register_callbacks cb 0x562480f4cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59348/0 (socket says 172.21.15.5:59348) 2024-04-22T13:28:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.439+0000 7f44c7fff640 1 -- 172.21.15.5:0/13351745 learned_addr learned my addr 172.21.15.5:0/13351745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44cd14f640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fe92d0 0x562480fe6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44cc94e640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fdb6f0 0x562480fe62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44c7fff640 1 -- 172.21.15.5:0/13351745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fdb6f0 msgr2=0x562480fe62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44c7fff640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fdb6f0 0x562480fe62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44c7fff640 1 -- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fe92d0 msgr2=0x562480fe6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44c7fff640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fe92d0 0x562480fe6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44c7fff640 1 -- 172.21.15.5:0/13351745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562480d7f420 con 0x562480fdc010 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44cd14f640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fe92d0 0x562480fe6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44c7fff640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe6810 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f44b40cda80 tx=0x7f44b40b3120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a554a447d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44a7fff640 1 -- 172.21.15.5:0/13351745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40a4e50 con 0x562480fdc010 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44a7fff640 1 -- 172.21.15.5:0/13351745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f44b40041d0 con 0x562480fdc010 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b409c8e0 con 0x562480fdc010 2024-04-22T13:28:10.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44a7fff640 1 -- 172.21.15.5:0/13351745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44b4098850 con 0x562480fdc010 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44d179f740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562480c12580 con 0x562480fdc010 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.440+0000 7f44a7fff640 1 -- 172.21.15.5:0/13351745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b409c8e0 con 0x562480fdc010 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44a7fff640 1 -- 172.21.15.5:0/13351745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f44b40a0170 con 0x562480fdc010 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44d179f740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44d179f740 10 client.16080 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44d179f740 20 client.16080 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44d179f740 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562480ff8e30 0x562480ffb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44d179f740 4 client.16080 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562480f4db10 con 0x562480ff8e30 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44cc94e640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562480ff8e30 0x562480ffb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44cc94e640 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562480ff8e30 0x562480ffb220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee814a581a6e4c38 server_cookie=649ccfef53bd1788 in_seq=0 out_seq=0 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.441+0000 7f44a7fff640 10 client.16080 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44a7fff640 1 -- 172.21.15.5:0/13351745 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562480f4db10 con 0x562480ff8e30 2024-04-22T13:28:10.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44a7fff640 10 client.16080 handle_command_reply: tid=0 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 client.16080 shutdown 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 msgr2=0x562480fe6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe6810 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f44b40cda80 tx=0x7f44b40b3120 comp rx=0 tx=0).stop 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 shutdown_connections 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562480ff8e30 0x562480ffb220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdc010 0x562480fe6810 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fe92d0 0x562480fe6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44a7fff640 0 client.16080 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 --2- 172.21.15.5:0/13351745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fdb6f0 0x562480fe62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 >> 172.21.15.5:0/13351745 conn(0x562480fd60f0 msgr2=0x562480fd7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.442+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 shutdown_connections 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.443+0000 7f44d179f740 1 -- 172.21.15.5:0/13351745 wait complete. 2024-04-22T13:28:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.443+0000 7f44d179f740 20 client.16080 trim_cache size 0 max 16384 2024-04-22T13:28:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 Processor -- start 2024-04-22T13:28:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 -- start start 2024-04-22T13:28:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562480ff48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x562480ffe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480ffe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81ad480 con 0x562480fdb6f0 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81ac010 con 0x562480fdc010 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.470+0000 7f44d179f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81ac6e0 con 0x562480fe92d0 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562480ff48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562480ff48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59362/0 (socket says 172.21.15.5:59362) 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 -- 172.21.15.5:0/1727788710 learned_addr learned my addr 172.21.15.5:0/1727788710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cd14f640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480ffe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44c7fff640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x562480ffe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 -- 172.21.15.5:0/1727788710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 msgr2=0x562480ffe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480ffe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 -- 172.21.15.5:0/1727788710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 msgr2=0x562480ffe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x562480ffe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 -- 172.21.15.5:0/1727788710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562480fdb320 con 0x562480fdb6f0 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cd14f640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480ffe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44c7fff640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x562480ffe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44cc94e640 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562480ff48b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8072362612bb6848 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44b27fc640 1 -- 172.21.15.5:0/1727788710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc005100 con 0x562480fdb6f0 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44b27fc640 1 -- 172.21.15.5:0/1727788710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc058070 con 0x562480fdb6f0 2024-04-22T13:28:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.471+0000 7f44b27fc640 1 -- 172.21.15.5:0/1727788710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc004920 con 0x562480fdb6f0 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 -- 172.21.15.5:0/1727788710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 msgr2=0x562480ff48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562480ff48b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc03e000 comp rx=0 tx=0).stop 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 -- 172.21.15.5:0/1727788710 shutdown_connections 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x562480ffe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x562480ffe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 --2- 172.21.15.5:0/1727788710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562480ff48b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 -- 172.21.15.5:0/1727788710 >> 172.21.15.5:0/1727788710 conn(0x562480fd60f0 msgr2=0x562480ff41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 -- 172.21.15.5:0/1727788710 shutdown_connections 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 -- 172.21.15.5:0/1727788710 wait complete. 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.472+0000 7f44d179f740 1 -- start start 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 Processor -- start 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562481053190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x5624810536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x56248105e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81a9920 con 0x562480fdb6f0 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81acdb0 con 0x562480fdc010 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81a9f90 con 0x562480fe92d0 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44d179f740 10 client.0 _ll_register_callbacks cb 0x7f44c0009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562481053190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562481053190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59370/0 (socket says 172.21.15.5:59370) 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 -- 172.21.15.5:0/2419063488 learned_addr learned my addr 172.21.15.5:0/2419063488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44c7fff640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x5624810536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cd14f640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x56248105e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 -- 172.21.15.5:0/2419063488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 msgr2=0x56248105e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x56248105e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 -- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 msgr2=0x5624810536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x5624810536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44cc94e640 1 -- 172.21.15.5:0/2419063488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56248105e590 con 0x562480fdb6f0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.473+0000 7f44c7fff640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x5624810536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44cc94e640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562481053190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f44bc000c00 tx=0x7f44bc052980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308b456bccc90a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44a7fff640 1 -- 172.21.15.5:0/2419063488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc063060 con 0x562480fdb6f0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44a7fff640 1 -- 172.21.15.5:0/2419063488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f44bc005100 con 0x562480fdb6f0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44bc058070 con 0x562480fdb6f0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44a7fff640 1 -- 172.21.15.5:0/2419063488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44bc005300 con 0x562480fdb6f0 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 10 client.16086 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56248105f870 con 0x562480fdb6f0 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44a7fff640 1 -- 172.21.15.5:0/2419063488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc058070 con 0x562480fdb6f0 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44a7fff640 1 -- 172.21.15.5:0/2419063488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f44bc0035f0 con 0x562480fdb6f0 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 10 client.16086 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 10 client.16086 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 20 client.16086 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56248105ffd0 0x5624810623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 4 client.16086 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await": true}] 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.474+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await": true}) -- 0x562480f47980 con 0x56248105ffd0 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.475+0000 7f44c7fff640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56248105ffd0 0x5624810623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.475+0000 7f44c7fff640 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56248105ffd0 0x5624810623f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a608e1008c6f3f server_cookie=191b1de0a2212172 in_seq=0 out_seq=0 2024-04-22T13:28:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.475+0000 7f44a7fff640 10 client.16086 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44a7fff640 1 -- 172.21.15.5:0/2419063488 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x562480f47980 con 0x56248105ffd0 2024-04-22T13:28:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44a7fff640 10 client.16086 handle_command_reply: tid=0 2024-04-22T13:28:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 client.16086 shutdown 2024-04-22T13:28:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 msgr2=0x562481053190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562481053190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f44bc000c00 tx=0x7f44bc052980 comp rx=0 tx=0).stop 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 shutdown_connections 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56248105ffd0 0x5624810623f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562480fe92d0 0x56248105e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562480fdc010 0x5624810536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 --2- 172.21.15.5:0/2419063488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562480fdb6f0 0x562481053190 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44a7fff640 0 client.16086 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 >> 172.21.15.5:0/2419063488 conn(0x562480fd60f0 msgr2=0x562480ff4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 shutdown_connections 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 1 -- 172.21.15.5:0/2419063488 wait complete. 2024-04-22T13:28:10.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.485+0000 7f44d179f740 20 client.16086 trim_cache size 0 max 16384 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 6, 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "3bd67737": { 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 6, 2024-04-22T13:28:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:28:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:28:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:28:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:28:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:28:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 70.0, 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:10.485 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f44b80a3f50 msgr2=0x7f44b80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f44b80a3f50 0x7f44b80a6410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005ab0 tx=0x7f44bc005a20 comp rx=0 tx=0).stop 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 msgr2=0x7f44c8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 0x7f44c8198bf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f44b400d950 tx=0x7f44b400de10 comp rx=0 tx=0).stop 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 shutdown_connections 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c8069bf0 0x7f44c8198bf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f44b80a3f50 0x7f44b80a6410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c8108040 0x7f44c819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 --2- 172.21.15.5:0/1800508717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c8069250 0x7f44c81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 >> 172.21.15.5:0/1800508717 conn(0x7f44c8077c40 msgr2=0x7f44c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 shutdown_connections 2024-04-22T13:28:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:10.486+0000 7f44cebb2640 1 -- 172.21.15.5:0/1800508717 wait complete. 2024-04-22T13:28:10.493 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 3bd67737, quiesce took 0.5 seconds. Will release after: 10 2024-04-22T13:28:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 -- 172.21.15.5:0/2902929830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c1010a0 msgr2=0x7fa03c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 --2- 172.21.15.5:0/2902929830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c1010a0 0x7fa03c101480 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa030009900 tx=0x7fa03002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 -- 172.21.15.5:0/2902929830 shutdown_connections 2024-04-22T13:28:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 --2- 172.21.15.5:0/2902929830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c106170 0x7fa03c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 --2- 172.21.15.5:0/2902929830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c101a50 0x7fa03c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 --2- 172.21.15.5:0/2902929830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c1010a0 0x7fa03c101480 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 -- 172.21.15.5:0/2902929830 >> 172.21.15.5:0/2902929830 conn(0x7fa03c077f70 msgr2=0x7fa03c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 -- 172.21.15.5:0/2902929830 shutdown_connections 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.841+0000 7fa043150640 1 -- 172.21.15.5:0/2902929830 wait complete. 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 Processor -- start 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 -- start start 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c1010a0 0x7fa03c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 0x7fa03c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c19a750 0x7fa03c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1110b0 con 0x7fa03c106170 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c110f30 con 0x7fa03c19a750 2024-04-22T13:28:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa043150640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c111230 con 0x7fa03c1010a0 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 0x7fa03c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 0x7fa03c1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59376/0 (socket says 172.21.15.5:59376) 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 -- 172.21.15.5:0/1153781089 learned_addr learned my addr 172.21.15.5:0/1153781089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa0416ed640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c19a750 0x7fa03c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa040eec640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c1010a0 0x7fa03c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 -- 172.21.15.5:0/1153781089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c1010a0 msgr2=0x7fa03c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c1010a0 0x7fa03c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 -- 172.21.15.5:0/1153781089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c19a750 msgr2=0x7fa03c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c19a750 0x7fa03c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa03bfff640 1 -- 172.21.15.5:0/1153781089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c19b350 con 0x7fa03c106170 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.842+0000 7fa040eec640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c1010a0 0x7fa03c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.843+0000 7fa03bfff640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 0x7fa03c1a0bd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa028005fd0 tx=0x7fa02800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.843+0000 7fa039ffb640 1 -- 172.21.15.5:0/1153781089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028017070 con 0x7fa03c106170 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.843+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03c1a7430 con 0x7fa03c106170 2024-04-22T13:28:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.843+0000 7fa039ffb640 1 -- 172.21.15.5:0/1153781089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028009c30 con 0x7fa03c106170 2024-04-22T13:28:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.843+0000 7fa039ffb640 1 -- 172.21.15.5:0/1153781089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028003d30 con 0x7fa03c106170 2024-04-22T13:28:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.843+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03c1a7830 con 0x7fa03c106170 2024-04-22T13:28:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 Processor -- start 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 -- start start 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x55629406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x55629406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x55629407b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556293fd48d0 con 0x5562940790e0 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556293f71e80 con 0x55629406fb60 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa045e1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556293fd0890 con 0x5562940708d0 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa039ffb640 1 -- 172.21.15.5:0/1153781089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7fa028003ed0 con 0x7fa03c106170 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa0416ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x55629406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa0416ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x55629406f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:52428/0 (socket says 172.21.15.5:52428) 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa0416ed640 1 -- 172.21.15.5:0/358347046 learned_addr learned my addr 172.21.15.5:0/358347046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa040eec640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x55629406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.846+0000 7fa03bfff640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x55629407b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa0416ed640 1 -- 172.21.15.5:0/358347046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 msgr2=0x55629406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa0416ed640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x55629406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa0416ed640 1 -- 172.21.15.5:0/358347046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 msgr2=0x55629407b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa0416ed640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x55629407b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa0416ed640 1 -- 172.21.15.5:0/358347046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556293f93320 con 0x5562940708d0 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa03bfff640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x55629407b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa040eec640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x55629406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa039ffb640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa02c0a3ec0 0x7fa02c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa0416ed640 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x55629406f620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa034009f90 tx=0x7fa034010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537d44c33c674740 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa040eec640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa02c0a3ec0 0x7fa02c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa039ffb640 1 -- 172.21.15.5:0/1153781089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa0280d0750 con 0x7fa03c106170 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa026ffd640 1 -- 172.21.15.5:0/358347046 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034011700 con 0x5562940708d0 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa026ffd640 1 -- 172.21.15.5:0/358347046 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa034011d50 con 0x5562940708d0 2024-04-22T13:28:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.847+0000 7fa026ffd640 1 -- 172.21.15.5:0/358347046 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034019d10 con 0x5562940708d0 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa040eec640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa02c0a3ec0 0x7fa02c0a6380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030005b50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 -- 172.21.15.5:0/358347046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 msgr2=0x55629406f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x55629406f620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa034009f90 tx=0x7fa034010860 comp rx=0 tx=0).stop 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 -- 172.21.15.5:0/358347046 shutdown_connections 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x55629407b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x55629406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 --2- 172.21.15.5:0/358347046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x55629406f620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 -- 172.21.15.5:0/358347046 >> 172.21.15.5:0/358347046 conn(0x556294036c90 msgr2=0x55629406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 -- 172.21.15.5:0/358347046 shutdown_connections 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.848+0000 7fa045e1c740 1 -- 172.21.15.5:0/358347046 wait complete. 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 -- start start 2024-04-22T13:28:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 Processor -- start 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294078a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 0x556294075da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940790e0 0x5562940762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55629407be10 con 0x5562940708d0 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556293f5ceb0 con 0x5562940790e0 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55629407bf90 con 0x55629406fb60 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa045e1c740 10 client.0 _ll_register_callbacks cb 0x556293f89b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 0x556294075da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa040eec640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294078a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 0x556294075da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59390/0 (socket says 172.21.15.5:59390) 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.849+0000 7fa03bfff640 1 -- 172.21.15.5:0/1150252596 learned_addr learned my addr 172.21.15.5:0/1150252596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa0416ed640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940790e0 0x5562940762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa040eec640 1 -- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940790e0 msgr2=0x5562940762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa040eec640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940790e0 0x5562940762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa040eec640 1 -- 172.21.15.5:0/1150252596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 msgr2=0x556294075da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa040eec640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 0x556294075da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa040eec640 1 -- 172.21.15.5:0/1150252596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556293ce7210 con 0x55629406fb60 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa03bfff640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 0x556294075da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa040eec640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294078a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa030009230 tx=0x7fa03002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9fdd0b5852ef71e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa01bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa01bfff640 1 -- 172.21.15.5:0/1150252596 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03004e030 con 0x55629406fb60 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa01bfff640 1 -- 172.21.15.5:0/1150252596 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa030005310 con 0x55629406fb60 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa034011d50 con 0x55629406fb60 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa01bfff640 1 -- 172.21.15.5:0/1150252596 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa030005510 con 0x55629406fb60 2024-04-22T13:28:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa045e1c740 10 client.25696 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.850+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556293db3620 con 0x55629406fb60 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa01bfff640 1 -- 172.21.15.5:0/1150252596 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa034011d50 con 0x55629406fb60 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa01bfff640 1 -- 172.21.15.5:0/1150252596 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa030003890 con 0x55629406fb60 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa045e1c740 10 client.25696 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa045e1c740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa045e1c740 20 client.25696 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556294089040 0x55629408b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa045e1c740 4 client.25696 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556293fe8880 con 0x556294089040 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.851+0000 7fa03bfff640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556294089040 0x55629408b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa03bfff640 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556294089040 0x55629408b450 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be67f6863d39b91 server_cookie=bf3ab334d0aebdaa in_seq=0 out_seq=0 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa01bfff640 10 client.25696 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa01bfff640 1 -- 172.21.15.5:0/1150252596 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556293fe8880 con 0x556294089040 2024-04-22T13:28:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa01bfff640 10 client.25696 handle_command_reply: tid=0 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 client.25696 shutdown 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 msgr2=0x556294078a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294078a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa030009230 tx=0x7fa03002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 shutdown_connections 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556294089040 0x55629408b450 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940708d0 0x556294075da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa01bfff640 0 client.25696 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940790e0 0x5562940762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1150252596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294078a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.852+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 >> 172.21.15.5:0/1150252596 conn(0x556294036c90 msgr2=0x55629406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.853+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 shutdown_connections 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.853+0000 7fa045e1c740 1 -- 172.21.15.5:0/1150252596 wait complete. 2024-04-22T13:28:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.853+0000 7fa045e1c740 20 client.25696 trim_cache size 0 max 16384 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 Processor -- start 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 -- start start 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294084d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940708d0 0x5562940784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294078a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1afd90 con 0x5562940790e0 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1b18d0 con 0x5562940708d0 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa045e1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1b0460 con 0x55629406fb60 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294078a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294078a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59396/0 (socket says 172.21.15.5:59396) 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 -- 172.21.15.5:0/1825790964 learned_addr learned my addr 172.21.15.5:0/1825790964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa040eec640 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294084d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 -- 172.21.15.5:0/1825790964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 msgr2=0x556294084d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294084d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 -- 172.21.15.5:0/1825790964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940708d0 msgr2=0x5562940784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940708d0 0x5562940784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.868+0000 7fa0416ed640 1 -- 172.21.15.5:0/1825790964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556294070220 con 0x5562940790e0 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.869+0000 7fa0416ed640 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294078a10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa034010d60 tx=0x7fa034009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40e3c4514f824f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.869+0000 7fa0267fc640 1 -- 172.21.15.5:0/1825790964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034003970 con 0x5562940790e0 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.869+0000 7fa0267fc640 1 -- 172.21.15.5:0/1825790964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa034011500 con 0x5562940790e0 2024-04-22T13:28:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.869+0000 7fa0267fc640 1 -- 172.21.15.5:0/1825790964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034005280 con 0x5562940790e0 2024-04-22T13:28:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.869+0000 7fa045e1c740 1 -- 172.21.15.5:0/1825790964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 msgr2=0x556294078a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.869+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294078a10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa034010d60 tx=0x7fa034009ac0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- 172.21.15.5:0/1825790964 shutdown_connections 2024-04-22T13:28:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294078a10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562940708d0 0x5562940784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 --2- 172.21.15.5:0/1825790964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55629406fb60 0x556294084d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- 172.21.15.5:0/1825790964 >> 172.21.15.5:0/1825790964 conn(0x556294036c90 msgr2=0x55629406a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- 172.21.15.5:0/1825790964 shutdown_connections 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- 172.21.15.5:0/1825790964 wait complete. 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- start start 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 Processor -- start 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x5562940815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x5562940727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1b1200 con 0x5562940790e0 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1b0b30 con 0x55629406fb60 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1add90 con 0x5562940708d0 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa045e1c740 10 client.0 _ll_register_callbacks cb 0x7fa030043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.870+0000 7fa0416ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59398/0 (socket says 172.21.15.5:59398) 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 -- 172.21.15.5:0/2409351449 learned_addr learned my addr 172.21.15.5:0/2409351449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa040eec640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x5562940815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa03bfff640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x5562940727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 -- 172.21.15.5:0/2409351449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 msgr2=0x5562940727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x5562940727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 -- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 msgr2=0x5562940815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x5562940815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 -- 172.21.15.5:0/2409351449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556294073520 con 0x5562940790e0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa0416ed640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294072cf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa0340108a0 tx=0x7fa034003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d8856592da3958 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa01bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa01bfff640 1 -- 172.21.15.5:0/2409351449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034031020 con 0x5562940790e0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa01bfff640 1 -- 172.21.15.5:0/2409351449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa034003d20 con 0x5562940790e0 2024-04-22T13:28:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa034011500 con 0x5562940790e0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa01bfff640 1 -- 172.21.15.5:0/2409351449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0340116b0 con 0x5562940790e0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa045e1c740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.871+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562940ef5e0 con 0x5562940790e0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa01bfff640 1 -- 172.21.15.5:0/2409351449 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa034011500 con 0x5562940790e0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa01bfff640 1 -- 172.21.15.5:0/2409351449 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa034024470 con 0x5562940790e0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa045e1c740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa045e1c740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa045e1c740 20 client.16104 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa045e1c740 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562940efaf0 0x5562940f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa045e1c740 4 client.16104 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556293f743d0 con 0x5562940efaf0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa040eec640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562940efaf0 0x5562940f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa040eec640 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562940efaf0 0x5562940f1ee0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77716f8012da7dc3 server_cookie=5769a5e1546db7f2 in_seq=0 out_seq=0 2024-04-22T13:28:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.872+0000 7fa01bfff640 10 client.16104 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa01bfff640 1 -- 172.21.15.5:0/2409351449 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556293f743d0 con 0x5562940efaf0 2024-04-22T13:28:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa01bfff640 10 client.16104 handle_command_reply: tid=0 2024-04-22T13:28:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 client.16104 shutdown 2024-04-22T13:28:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 msgr2=0x556294072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294072cf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa0340108a0 tx=0x7fa034003970 comp rx=0 tx=0).stop 2024-04-22T13:28:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 shutdown_connections 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562940efaf0 0x5562940f1ee0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562940790e0 0x556294072cf0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562940708d0 0x5562940727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 --2- 172.21.15.5:0/2409351449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55629406fb60 0x5562940815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 >> 172.21.15.5:0/2409351449 conn(0x556294036c90 msgr2=0x556294070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa01bfff640 0 client.16104 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 shutdown_connections 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 1 -- 172.21.15.5:0/2409351449 wait complete. 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.879+0000 7fa045e1c740 20 client.16104 trim_cache size 0 max 16384 2024-04-22T13:28:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:28:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:28:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:28:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:28:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa02c0a3ec0 msgr2=0x7fa02c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa02c0a3ec0 0x7fa02c0a6380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030005b50 comp rx=0 tx=0).stop 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 msgr2=0x7fa03c1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 0x7fa03c1a0bd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa028005fd0 tx=0x7fa02800dde0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 shutdown_connections 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa03c106170 0x7fa03c1a0bd0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa02c0a3ec0 0x7fa02c0a6380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c19a750 0x7fa03c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 --2- 172.21.15.5:0/1153781089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c1010a0 0x7fa03c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 >> 172.21.15.5:0/1153781089 conn(0x7fa03c077f70 msgr2=0x7fa03c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 shutdown_connections 2024-04-22T13:28:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:10.880+0000 7fa043150640 1 -- 172.21.15.5:0/1153781089 wait complete. 2024-04-22T13:28:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:10 smithi049 ceph-mon[31379]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:28:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:10 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/247486930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:11.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:10 smithi107 ceph-mon[31859]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:28:11.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:10 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/247486930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:11.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:10 smithi005 ceph-mon[28006]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:28:11.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:10 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/247486930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:12 smithi049 ceph-mon[31379]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:28:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:12 smithi107 ceph-mon[31859]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:28:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:12 smithi005 ceph-mon[28006]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:28:15.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:14 smithi049 ceph-mon[31379]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:28:15.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:14 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:28:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:14 smithi107 ceph-mon[31859]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:28:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:14 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:28:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:14 smithi005 ceph-mon[28006]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:28:15.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:14 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:28:16.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:28:16.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:28:16.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:28:16.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:28:16.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:16.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:28:16.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:28:16.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:28:17.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:17 smithi049 ceph-mon[31379]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:28:17.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:17 smithi107 ceph-mon[31859]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:28:17.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:17 smithi005 ceph-mon[28006]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:28:19.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:19 smithi049 ceph-mon[31379]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-22T13:28:19.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:19 smithi107 ceph-mon[31859]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-22T13:28:19.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:19 smithi005 ceph-mon[28006]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-22T13:28:19.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:19.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 -- 172.21.15.5:0/1410274131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069f10 msgr2=0x7f9c0c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 --2- 172.21.15.5:0/1410274131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069f10 0x7f9c0c106bc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009900 tx=0x7f9c0002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 -- 172.21.15.5:0/1410274131 shutdown_connections 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 --2- 172.21.15.5:0/1410274131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069f10 0x7f9c0c106bc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 --2- 172.21.15.5:0/1410274131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c1072a0 0x7f9c0c10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 --2- 172.21.15.5:0/1410274131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0c069560 0x7f9c0c069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 -- 172.21.15.5:0/1410274131 >> 172.21.15.5:0/1410274131 conn(0x7f9c0c077fa0 msgr2=0x7f9c0c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 -- 172.21.15.5:0/1410274131 shutdown_connections 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.720+0000 7f9c125b2640 1 -- 172.21.15.5:0/1410274131 wait complete. 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.721+0000 7f9c125b2640 1 Processor -- start 2024-04-22T13:28:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.721+0000 7f9c125b2640 1 -- start start 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.721+0000 7f9c125b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 0x7f9c0c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.721+0000 7f9c125b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0c1072a0 0x7f9c0c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c125b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c101890 0x7f9c0c1a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 0x7f9c0c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 0x7f9c0c100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55794/0 (socket says 172.21.15.5:55794) 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2919609003 learned_addr learned my addr 172.21.15.5:0/2919609003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1121b0 con 0x7f9c0c069560 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c112030 con 0x7f9c0c1072a0 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c112330 con 0x7f9c0c101890 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0b7fe640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0c1072a0 0x7f9c0c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c101890 0x7f9c0c1a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2919609003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c101890 msgr2=0x7f9c0c1a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c101890 0x7f9c0c1a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2919609003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0c1072a0 msgr2=0x7f9c0c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0c1072a0 0x7f9c0c101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2919609003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1a7040 con 0x7f9c0c069560 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c101890 0x7f9c0c1a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 0x7f9c0c100e10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00e9f0 tx=0x7f9bfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c097fa640 1 -- 172.21.15.5:0/2919609003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00cd80 con 0x7f9c0c069560 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c097fa640 1 -- 172.21.15.5:0/2919609003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0108e0 con 0x7f9c0c069560 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c097fa640 1 -- 172.21.15.5:0/2919609003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc018980 con 0x7f9c0c069560 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.722+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1a7330 con 0x7f9c0c069560 2024-04-22T13:28:19.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.723+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1a7800 con 0x7f9c0c069560 2024-04-22T13:28:19.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 Processor -- start 2024-04-22T13:28:19.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 -- start start 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c097fa640 1 -- 172.21.15.5:0/2919609003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f9bfc010430 con 0x7f9c0c069560 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f5f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f5fea0 0x561687f68540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f68a80 0x561687f6ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561687ec41e0 con 0x561687f5fea0 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561687e61390 con 0x561687f5f580 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.725+0000 7f9c151a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561687ec01a0 con 0x561687f68a80 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f5fea0 0x561687f68540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f5f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f68a80 0x561687f6ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f5f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49690/0 (socket says 172.21.15.5:49690) 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2122459984 learned_addr learned my addr 172.21.15.5:0/2122459984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c097fa640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9bf40a3f80 0x7f9bf40a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2122459984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f68a80 msgr2=0x561687f6ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f68a80 0x561687f6ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2122459984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f5fea0 msgr2=0x561687f68540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f5fea0 0x561687f68540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2122459984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561687ca07d0 con 0x561687f5f580 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c097fa640 1 -- 172.21.15.5:0/2919609003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9bfc0d0df0 con 0x7f9c0c069560 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f5fea0 0x561687f68540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9bf40a3f80 0x7f9bf40a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.726+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f5f960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009e40 tx=0x7f9c0400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed91e4c55440f76a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.727+0000 7f9bf27fc640 1 -- 172.21.15.5:0/2122459984 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04021070 con 0x561687f5f580 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.727+0000 7f9bf27fc640 1 -- 172.21.15.5:0/2122459984 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0401bdb0 con 0x561687f5f580 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.727+0000 7f9bf27fc640 1 -- 172.21.15.5:0/2122459984 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04010a90 con 0x561687f5f580 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.727+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9bf40a3f80 0x7f9bf40a6440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9c0401e000 tx=0x7f9c04021000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.727+0000 7f9c151a5740 1 -- 172.21.15.5:0/2122459984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 msgr2=0x561687f5f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.727+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f5f960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009e40 tx=0x7f9c0400f730 comp rx=0 tx=0).stop 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 -- 172.21.15.5:0/2122459984 shutdown_connections 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f5fea0 0x561687f68540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f68a80 0x561687f6ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2122459984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f5f960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 -- 172.21.15.5:0/2122459984 >> 172.21.15.5:0/2122459984 conn(0x561687f59cb0 msgr2=0x561687f5a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 -- 172.21.15.5:0/2122459984 shutdown_connections 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 -- 172.21.15.5:0/2122459984 wait complete. 2024-04-22T13:28:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 -- start start 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 Processor -- start 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f65a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f65fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.728+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c151a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561687f6b870 con 0x561687f68a80 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c151a5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561687e7d2f0 con 0x561687f5fea0 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c151a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561687f6b9f0 con 0x561687f5f580 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55804/0 (socket says 172.21.15.5:55804) 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c151a5740 10 client.0 _ll_register_callbacks cb 0x561687e8a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 -- 172.21.15.5:0/3173788766 learned_addr learned my addr 172.21.15.5:0/3173788766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f65a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c0b7fe640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f65fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 -- 172.21.15.5:0/3173788766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 msgr2=0x561687f65a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f65a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 -- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 msgr2=0x561687f65fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f65fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 -- 172.21.15.5:0/3173788766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561687bd63f0 con 0x561687f68a80 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f664f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9c04010710 tx=0x7f9c0400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b963f865bf29b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9beb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.729+0000 7f9beb7fe640 1 -- 172.21.15.5:0/3173788766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04033060 con 0x561687f68a80 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9beb7fe640 1 -- 172.21.15.5:0/3173788766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9c04010850 con 0x561687f68a80 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0401bdb0 con 0x561687f68a80 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9beb7fe640 1 -- 172.21.15.5:0/3173788766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c04010a50 con 0x561687f68a80 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9c151a5740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561687b960f0 con 0x561687f68a80 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9beb7fe640 1 -- 172.21.15.5:0/3173788766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0401bdb0 con 0x561687f68a80 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.730+0000 7f9beb7fe640 1 -- 172.21.15.5:0/3173788766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9c0402d030 con 0x561687f68a80 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c151a5740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c151a5740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c151a5740 20 client.16116 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c151a5740 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687f789c0 0x561687f7adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c151a5740 4 client.16116 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561687ed1c00 con 0x561687f789c0 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687f789c0 0x561687f7adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687f789c0 0x561687f7adb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488b580546535174 server_cookie=fbafa350215b3b13 in_seq=0 out_seq=0 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.731+0000 7f9beb7fe640 10 client.16116 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9beb7fe640 1 -- 172.21.15.5:0/3173788766 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561687ed1c00 con 0x561687f789c0 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9beb7fe640 10 client.16116 handle_command_reply: tid=0 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 client.16116 shutdown 2024-04-22T13:28:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 msgr2=0x561687f664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f664f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9c04010710 tx=0x7f9c0400ff00 comp rx=0 tx=0).stop 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 shutdown_connections 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687f789c0 0x561687f7adb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f664f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f65fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9beb7fe640 0 client.16116 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 --2- 172.21.15.5:0/3173788766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f65a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 >> 172.21.15.5:0/3173788766 conn(0x561687f59cb0 msgr2=0x561687f5ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 shutdown_connections 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.732+0000 7f9c151a5740 1 -- 172.21.15.5:0/3173788766 wait complete. 2024-04-22T13:28:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.733+0000 7f9c151a5740 20 client.16116 trim_cache size 0 max 16384 2024-04-22T13:28:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 Processor -- start 2024-04-22T13:28:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 -- start start 2024-04-22T13:28:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f68130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5fea0 0x561687f74730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f80570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1afdd0 con 0x561687f68a80 2024-04-22T13:28:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b1910 con 0x561687f5f580 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.750+0000 7f9c151a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b04a0 con 0x561687f5fea0 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f80570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f80570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55810/0 (socket says 172.21.15.5:55810) 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2244427622 learned_addr learned my addr 172.21.15.5:0/2244427622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f68130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c0b7fe640 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5fea0 0x561687f74730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2244427622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5fea0 msgr2=0x561687f74730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5fea0 0x561687f74730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2244427622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 msgr2=0x561687f68130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f68130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2244427622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561687f7b340 con 0x561687f68a80 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f80570 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9c04010010 tx=0x7f9c04009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ca5aace2e67f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9bf1ffb640 1 -- 172.21.15.5:0/2244427622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04033040 con 0x561687f68a80 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9bf1ffb640 1 -- 172.21.15.5:0/2244427622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0401b800 con 0x561687f68a80 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.751+0000 7f9bf1ffb640 1 -- 172.21.15.5:0/2244427622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040268b0 con 0x561687f68a80 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 -- 172.21.15.5:0/2244427622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 msgr2=0x561687f80570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f80570 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9c04010010 tx=0x7f9c04009fd0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 -- 172.21.15.5:0/2244427622 shutdown_connections 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f80570 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5fea0 0x561687f74730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2244427622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5f580 0x561687f68130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 -- 172.21.15.5:0/2244427622 >> 172.21.15.5:0/2244427622 conn(0x561687f59cb0 msgr2=0x561687f60d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 -- 172.21.15.5:0/2244427622 shutdown_connections 2024-04-22T13:28:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.752+0000 7f9c151a5740 1 -- 172.21.15.5:0/2244427622 wait complete. 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.753+0000 7f9c151a5740 1 -- start start 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.753+0000 7f9c151a5740 1 Processor -- start 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.753+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f70e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.753+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f6c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.753+0000 7f9c151a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f6c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f6c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c10b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f6c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55826/0 (socket says 172.21.15.5:55826) 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c10b4f640 1 -- 172.21.15.5:0/2476228409 learned_addr learned my addr 172.21.15.5:0/2476228409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f70e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b1240 con 0x561687f68a80 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b0b70 con 0x561687f5fea0 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1addb0 con 0x561687f5f580 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0b7fe640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f6c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 msgr2=0x561687f6c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f6c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2476228409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 msgr2=0x561687f6c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f6c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561687f6cdb0 con 0x561687f5f580 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c10b4f640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f6c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c151a5740 10 client.0 _ll_register_callbacks cb 0x7f9c0401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0401b800 con 0x561687f5f580 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9c0bfff640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f70e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0ce330 tx=0x7f9bfc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8dc54522f118b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.754+0000 7f9beb7fe640 10 client.25717 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9beb7fe640 1 -- 172.21.15.5:0/2476228409 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0a57f0 con 0x561687f5f580 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9beb7fe640 1 -- 172.21.15.5:0/2476228409 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9bfc010a90 con 0x561687f5f580 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9beb7fe640 1 -- 172.21.15.5:0/2476228409 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9bfc0bd0b0 con 0x561687f5f580 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 10 client.25717 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=27}) -- 0x561687fde9c0 con 0x561687f5f580 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9beb7fe640 1 -- 172.21.15.5:0/2476228409 <== mon.1 v2:172.21.15.107:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9bfc0991f0 con 0x561687f5f580 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 10 client.25717 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 10 client.25717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 20 client.25717 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687fdf0c0 0x561687fe1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 4 client.25717 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.755+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561687ecba70 con 0x561687fdf0c0 2024-04-22T13:28:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9c0b7fe640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687fdf0c0 0x561687fe1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9beb7fe640 10 client.25717 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9c0b7fe640 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687fdf0c0 0x561687fe1580 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c569f98dcb58e663 server_cookie=1a461f617a545349 in_seq=0 out_seq=0 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9beb7fe640 1 -- 172.21.15.5:0/2476228409 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561687ecba70 con 0x561687fdf0c0 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9beb7fe640 10 client.25717 handle_command_reply: tid=0 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9c151a5740 1 client.25717 shutdown 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 msgr2=0x561687f70e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f70e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0ce330 tx=0x7f9bfc000bc0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.756+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 shutdown_connections 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561687fdf0c0 0x561687fe1580 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561687f68a80 0x561687f6c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561687f5fea0 0x561687f6c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9beb7fe640 0 client.25717 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 --2- 172.21.15.5:0/2476228409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687f5f580 0x561687f70e90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 >> 172.21.15.5:0/2476228409 conn(0x561687f59cb0 msgr2=0x561687f584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 shutdown_connections 2024-04-22T13:28:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 1 -- 172.21.15.5:0/2476228409 wait complete. 2024-04-22T13:28:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c151a5740 20 client.25717 trim_cache size 0 max 16384 2024-04-22T13:28:19.764 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:19.764 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:28:19.764 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:28:19.764 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9bf40a3f80 msgr2=0x7f9bf40a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c125b2640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9bf40a3f80 0x7f9bf40a6440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9c0401e000 tx=0x7f9c04021000 comp rx=0 tx=0).stop 2024-04-22T13:28:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 msgr2=0x7f9c0c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.757+0000 7f9c125b2640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 0x7f9c0c100e10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00e9f0 tx=0x7f9bfc00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 shutdown_connections 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9bf40a3f80 0x7f9bf40a6440 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0c101890 0x7f9c0c1a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0c1072a0 0x7f9c0c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 --2- 172.21.15.5:0/2919609003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c0c069560 0x7f9c0c100e10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 >> 172.21.15.5:0/2919609003 conn(0x7f9c0c077fa0 msgr2=0x7f9c0c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 shutdown_connections 2024-04-22T13:28:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.758+0000 7f9c125b2640 1 -- 172.21.15.5:0/2919609003 wait complete. 2024-04-22T13:28:19.767 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:28:19.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:19.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 -- 172.21.15.5:0/2736786051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0101070 msgr2=0x7f95c0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 --2- 172.21.15.5:0/2736786051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0101070 0x7f95c0101450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009900 tx=0x7f95bc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 -- 172.21.15.5:0/2736786051 shutdown_connections 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 --2- 172.21.15.5:0/2736786051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c0106140 0x7f95c010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 --2- 172.21.15.5:0/2736786051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95c0101a20 0x7f95c0105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 --2- 172.21.15.5:0/2736786051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0101070 0x7f95c0101450 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 -- 172.21.15.5:0/2736786051 >> 172.21.15.5:0/2736786051 conn(0x7f95c00771d0 msgr2=0x7f95c00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 -- 172.21.15.5:0/2736786051 shutdown_connections 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.896+0000 7f95c758f640 1 -- 172.21.15.5:0/2736786051 wait complete. 2024-04-22T13:28:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 Processor -- start 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 -- start start 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95c0101070 0x7f95c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c0106140 0x7f95c0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 0x7f95c019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111080 con 0x7f95c019cf80 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0110f00 con 0x7f95c0106140 2024-04-22T13:28:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c758f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111200 con 0x7f95c0101070 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c5b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 0x7f95c019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c5b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 0x7f95c019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55844/0 (socket says 172.21.15.5:55844) 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c5b2c640 1 -- 172.21.15.5:0/3388539513 learned_addr learned my addr 172.21.15.5:0/3388539513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.897+0000 7f95c532b640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95c0101070 0x7f95c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c4b2a640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c0106140 0x7f95c0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c5b2c640 1 -- 172.21.15.5:0/3388539513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95c0101070 msgr2=0x7f95c01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c5b2c640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95c0101070 0x7f95c01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c5b2c640 1 -- 172.21.15.5:0/3388539513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c0106140 msgr2=0x7f95c0198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c5b2c640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c0106140 0x7f95c0198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c5b2c640 1 -- 172.21.15.5:0/3388539513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c019da70 con 0x7f95c019cf80 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c5b2c640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 0x7f95c019d430 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f95b800eb10 tx=0x7f95b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8004b20 con 0x7f95c019cf80 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c019dd60 con 0x7f95c019cf80 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b8004cc0 con 0x7f95c019cf80 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8010670 con 0x7f95c019cf80 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.898+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c01a5550 con 0x7f95c019cf80 2024-04-22T13:28:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.899+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c0068e00 con 0x7f95c019cf80 2024-04-22T13:28:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.900+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f95b8005020 con 0x7f95c019cf80 2024-04-22T13:28:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.901+0000 7f95b67fc640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f95ac0a3ed0 0x7f95ac0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.901+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f95b80d0fb0 con 0x7f95c019cf80 2024-04-22T13:28:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.901+0000 7f95c532b640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f95ac0a3ed0 0x7f95ac0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.902+0000 7f95c532b640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f95ac0a3ed0 0x7f95ac0a6390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95bc006010 tx=0x7f95bc005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:19.906+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95b809d440 con 0x7f95c019cf80 2024-04-22T13:28:20.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.167+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95c0101af0 con 0x7f95c019cf80 2024-04-22T13:28:20.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.168+0000 7f95b67fc640 1 -- 172.21.15.5:0/3388539513 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f95b809cd40 con 0x7f95c019cf80 2024-04-22T13:28:20.164 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:28:20.164 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:28:20.165 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:28:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f95ac0a3ed0 msgr2=0x7f95ac0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f95ac0a3ed0 0x7f95ac0a6390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95bc006010 tx=0x7f95bc005e40 comp rx=0 tx=0).stop 2024-04-22T13:28:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 msgr2=0x7f95c019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 0x7f95c019d430 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f95b800eb10 tx=0x7f95b800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 shutdown_connections 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c019cf80 0x7f95c019d430 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f95ac0a3ed0 0x7f95ac0a6390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c0106140 0x7f95c0198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 --2- 172.21.15.5:0/3388539513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95c0101070 0x7f95c01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.170+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 >> 172.21.15.5:0/3388539513 conn(0x7f95c00771d0 msgr2=0x7f95c0107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.171+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 shutdown_connections 2024-04-22T13:28:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.171+0000 7f95c758f640 1 -- 172.21.15.5:0/3388539513 wait complete. 2024-04-22T13:28:20.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 -- 172.21.15.5:0/1470240821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 msgr2=0x7f126410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 --2- 172.21.15.5:0/1470240821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f126410e9a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f125c009ca0 tx=0x7f125c02c180 comp rx=0 tx=0).stop 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 -- 172.21.15.5:0/1470240821 shutdown_connections 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 --2- 172.21.15.5:0/1470240821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f126410e9a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 --2- 172.21.15.5:0/1470240821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264103ae0 0x7f126410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 --2- 172.21.15.5:0/1470240821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12641031c0 0x7f12641035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 -- 172.21.15.5:0/1470240821 >> 172.21.15.5:0/1470240821 conn(0x7f1264078000 msgr2=0x7f1264078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 -- 172.21.15.5:0/1470240821 shutdown_connections 2024-04-22T13:28:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.290+0000 7f126bef1640 1 -- 172.21.15.5:0/1470240821 wait complete. 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 Processor -- start 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 -- start start 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 0x7f1264100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1264103ae0 0x7f1264101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f1264101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1264112190 con 0x7f126410c500 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1264112010 con 0x7f1264103ae0 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126bef1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1264112310 con 0x7f12641031c0 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126a48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f1264101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f1269c8d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 0x7f1264100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f1269c8d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 0x7f1264100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:53628/0 (socket says 172.21.15.5:53628) 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f1269c8d640 1 -- 172.21.15.5:0/1503113903 learned_addr learned my addr 172.21.15.5:0/1503113903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.291+0000 7f126948c640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1264103ae0 0x7f1264101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f1269c8d640 1 -- 172.21.15.5:0/1503113903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1264103ae0 msgr2=0x7f1264101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f1269c8d640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1264103ae0 0x7f1264101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f1269c8d640 1 -- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 msgr2=0x7f1264101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f1269c8d640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f1264101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f1269c8d640 1 -- 172.21.15.5:0/1503113903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641a6bf0 con 0x7f12641031c0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f126a48e640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f1264101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f1269c8d640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 0x7f1264100e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f126000ea60 tx=0x7f126000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f125affd640 1 -- 172.21.15.5:0/1503113903 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126000cde0 con 0x7f12641031c0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f125affd640 1 -- 172.21.15.5:0/1503113903 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12600035f0 con 0x7f12641031c0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12641a6ee0 con 0x7f12641031c0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f125affd640 1 -- 172.21.15.5:0/1503113903 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260010690 con 0x7f12641031c0 2024-04-22T13:28:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.292+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12641a73a0 con 0x7f12641031c0 2024-04-22T13:28:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.295+0000 7f125affd640 1 -- 172.21.15.5:0/1503113903 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f12600037a0 con 0x7f12641031c0 2024-04-22T13:28:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 Processor -- start 2024-04-22T13:28:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 -- start start 2024-04-22T13:28:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d34d0 0x5556de3d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de3dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556de3381f0 con 0x5556de3d3e80 2024-04-22T13:28:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5556de2d54b0 con 0x5556de3d34d0 2024-04-22T13:28:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126ebb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5556de3341b0 con 0x5556de3dca00 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f125affd640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f12500a3ed0 0x7f12500a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126948c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de3dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126a48e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d34d0 0x5556de3d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126948c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de3dee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:53632/0 (socket says 172.21.15.5:53632) 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126948c640 1 -- 172.21.15.5:0/2659508896 learned_addr learned my addr 172.21.15.5:0/2659508896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f126a48e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d34d0 0x5556de3d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49746/0 (socket says 172.21.15.5:49746) 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f125affd640 1 -- 172.21.15.5:0/1503113903 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f12600d0e10 con 0x7f12641031c0 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.296+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126a48e640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f12500a3ed0 0x7f12500a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126948c640 1 -- 172.21.15.5:0/2659508896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d34d0 msgr2=0x5556de3d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126948c640 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d34d0 0x5556de3d38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126948c640 1 -- 172.21.15.5:0/2659508896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 msgr2=0x5556de3dc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126948c640 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3dc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126948c640 1 -- 172.21.15.5:0/2659508896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556de2c71b0 con 0x5556de3dca00 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126948c640 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de3dee50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f12540096a0 tx=0x7f1254009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf9caffff190eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f126a48e640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f12500a3ed0 0x7f12500a6390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f125c004af0 tx=0x7f125c005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f124ffff640 1 -- 172.21.15.5:0/2659508896 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254009db0 con 0x5556de3dca00 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f124ffff640 1 -- 172.21.15.5:0/2659508896 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125400c8b0 con 0x5556de3dca00 2024-04-22T13:28:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.297+0000 7f124ffff640 1 -- 172.21.15.5:0/2659508896 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12540175b0 con 0x5556de3dca00 2024-04-22T13:28:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 -- 172.21.15.5:0/2659508896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 msgr2=0x5556de3dee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de3dee50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f12540096a0 tx=0x7f1254009b60 comp rx=0 tx=0).stop 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 -- 172.21.15.5:0/2659508896 shutdown_connections 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de3dee50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2659508896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d34d0 0x5556de3d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.298+0000 7f126ebb6740 1 -- 172.21.15.5:0/2659508896 >> 172.21.15.5:0/2659508896 conn(0x5556de3cdba0 msgr2=0x5556de3cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.299+0000 7f126ebb6740 1 -- 172.21.15.5:0/2659508896 shutdown_connections 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.299+0000 7f126ebb6740 1 -- 172.21.15.5:0/2659508896 wait complete. 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.299+0000 7f126ebb6740 1 -- start start 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.299+0000 7f126ebb6740 1 Processor -- start 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3dca00 0x5556de3d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556de3df970 con 0x5556de3d3e80 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5556de2f1500 con 0x5556de3dca00 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5556de3dfaf0 con 0x5556de3d34d0 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126ebb6740 10 client.0 _ll_register_callbacks cb 0x5556de2bfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3d96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55870/0 (socket says 172.21.15.5:55870) 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 -- 172.21.15.5:0/2095559973 learned_addr learned my addr 172.21.15.5:0/2095559973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126a48e640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3dca00 0x5556de3d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 -- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 msgr2=0x5556de3dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 -- 172.21.15.5:0/2095559973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3dca00 msgr2=0x5556de3d9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3dca00 0x5556de3d9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 -- 172.21.15.5:0/2095559973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556de04a3f0 con 0x5556de3d3e80 2024-04-22T13:28:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126a48e640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3dca00 0x5556de3d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f126948c640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3d96d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f125400c6a0 tx=0x7f125400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3ea31455927961 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.300+0000 7f124d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f124d7fa640 1 -- 172.21.15.5:0/2095559973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254009e20 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f124d7fa640 1 -- 172.21.15.5:0/2095559973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1254026040 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125400c8b0 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f124d7fa640 1 -- 172.21.15.5:0/2095559973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1254017df0 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 10 client.16140 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556de1147d0 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f124d7fa640 1 -- 172.21.15.5:0/2095559973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125400c8b0 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f124d7fa640 1 -- 172.21.15.5:0/2095559973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f12540035f0 con 0x5556de3d3e80 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 10 client.16140 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 10 client.16140 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 20 client.16140 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de3ecb20 0x5556de3eef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 4 client.16140 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.301+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556de2d7930 con 0x5556de3ecb20 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.302+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de3ecb20 0x5556de3eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.302+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de3ecb20 0x5556de3eef10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d3f2702c5382f85 server_cookie=1266e037f67da58 in_seq=0 out_seq=0 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.302+0000 7f124d7fa640 10 client.16140 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f124d7fa640 1 -- 172.21.15.5:0/2095559973 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556de2d7930 con 0x5556de3ecb20 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f124d7fa640 10 client.16140 handle_command_reply: tid=0 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 client.16140 shutdown 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 msgr2=0x5556de3d96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3d96d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f125400c6a0 tx=0x7f125400cc70 comp rx=0 tx=0).stop 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 shutdown_connections 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de3ecb20 0x5556de3eef10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d3e80 0x5556de3d96d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3dca00 0x5556de3d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2095559973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 >> 172.21.15.5:0/2095559973 conn(0x5556de3cdba0 msgr2=0x5556de3cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f124d7fa640 0 client.16140 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 shutdown_connections 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.303+0000 7f126ebb6740 1 -- 172.21.15.5:0/2095559973 wait complete. 2024-04-22T13:28:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.304+0000 7f126ebb6740 20 client.16140 trim_cache size 0 max 16384 2024-04-22T13:28:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 Processor -- start 2024-04-22T13:28:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 -- start start 2024-04-22T13:28:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 0x5556de3dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12641ab910 con 0x5556de3dca00 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12641ad450 con 0x5556de3d3e80 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126ebb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12641abfe0 con 0x5556de3d34d0 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126a48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 0x5556de3dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126a48e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 0x5556de3dc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55876/0 (socket says 172.21.15.5:55876) 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f126a48e640 1 -- 172.21.15.5:0/523127668 learned_addr learned my addr 172.21.15.5:0/523127668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.317+0000 7f1269c8d640 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126948c640 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126a48e640 1 -- 172.21.15.5:0/523127668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 msgr2=0x5556de3e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126a48e640 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126a48e640 1 -- 172.21.15.5:0/523127668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 msgr2=0x5556de3dbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126a48e640 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3dbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126a48e640 1 -- 172.21.15.5:0/523127668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556de3dfaa0 con 0x5556de3dca00 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126948c640 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3dbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f126a48e640 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 0x5556de3dc340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f125c009ca0 tx=0x7f125c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508c0815f09ef194 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f124f7fe640 1 -- 172.21.15.5:0/523127668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c039040 con 0x5556de3dca00 2024-04-22T13:28:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f124f7fe640 1 -- 172.21.15.5:0/523127668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125c052070 con 0x5556de3dca00 2024-04-22T13:28:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.318+0000 7f124f7fe640 1 -- 172.21.15.5:0/523127668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c042a50 con 0x5556de3dca00 2024-04-22T13:28:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 -- 172.21.15.5:0/523127668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 msgr2=0x5556de3dc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 0x5556de3dc340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f125c009ca0 tx=0x7f125c03b000 comp rx=0 tx=0).stop 2024-04-22T13:28:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 -- 172.21.15.5:0/523127668 shutdown_connections 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3dca00 0x5556de3dc340 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3dbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 --2- 172.21.15.5:0/523127668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3d34d0 0x5556de3e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 -- 172.21.15.5:0/523127668 >> 172.21.15.5:0/523127668 conn(0x5556de3cdba0 msgr2=0x5556de3ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 -- 172.21.15.5:0/523127668 shutdown_connections 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 -- 172.21.15.5:0/523127668 wait complete. 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 -- start start 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.319+0000 7f126ebb6740 1 Processor -- start 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 0x5556de3dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de450dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12641acd80 con 0x5556de3d34d0 2024-04-22T13:28:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12641ac6b0 con 0x5556de3d3e80 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12641a9910 con 0x5556de3dca00 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126ebb6740 10 client.0 _ll_register_callbacks cb 0x7f1254015f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 0x5556de3dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 0x5556de3dfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55884/0 (socket says 172.21.15.5:55884) 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 -- 172.21.15.5:0/2803340598 learned_addr learned my addr 172.21.15.5:0/2803340598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126a48e640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de450dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 -- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 msgr2=0x5556de450dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de450dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f126948c640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 -- 172.21.15.5:0/2803340598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 msgr2=0x5556de3e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.320+0000 7f1269c8d640 1 -- 172.21.15.5:0/2803340598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556de451310 con 0x5556de3d34d0 2024-04-22T13:28:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f124d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f1269c8d640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 0x5556de3dfef0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f126009c8a0 tx=0x7f12600bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dad94ba87491f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f124d7fa640 1 -- 172.21.15.5:0/2803340598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600a2460 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f124d7fa640 1 -- 172.21.15.5:0/2803340598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f12600bd0b0 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c052070 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f124d7fa640 1 -- 172.21.15.5:0/2803340598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12600a5810 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f126ebb6740 10 client.16146 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556de452650 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f124d7fa640 1 -- 172.21.15.5:0/2803340598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125c052070 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.321+0000 7f124d7fa640 1 -- 172.21.15.5:0/2803340598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f12600cece0 con 0x5556de3d34d0 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126ebb6740 10 client.16146 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126ebb6740 10 client.16146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126ebb6740 20 client.16146 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de452d20 0x5556de455140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126ebb6740 4 client.16146 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556de345c10 con 0x5556de452d20 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126948c640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de452d20 0x5556de455140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f126948c640 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de452d20 0x5556de455140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a20f796cc3ae7a3 server_cookie=31386a6498c0e456 in_seq=0 out_seq=0 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.322+0000 7f124d7fa640 10 client.16146 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f124d7fa640 1 -- 172.21.15.5:0/2803340598 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556de345c10 con 0x5556de452d20 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f124d7fa640 10 client.16146 handle_command_reply: tid=0 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 client.16146 shutdown 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 msgr2=0x5556de3dfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 0x5556de3dfef0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f126009c8a0 tx=0x7f12600bd040 comp rx=0 tx=0).stop 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 shutdown_connections 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5556de3dca00 0x5556de450dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556de3d3e80 0x5556de3e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5556de452d20 0x5556de455140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 --2- 172.21.15.5:0/2803340598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556de3d34d0 0x5556de3dfef0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f124d7fa640 0 client.16146 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 >> 172.21.15.5:0/2803340598 conn(0x5556de3cdba0 msgr2=0x5556de3d7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 shutdown_connections 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 1 -- 172.21.15.5:0/2803340598 wait complete. 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.323+0000 7f126ebb6740 20 client.16146 trim_cache size 0 max 16384 2024-04-22T13:28:20.325 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f12500a3ed0 msgr2=0x7f12500a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f12500a3ed0 0x7f12500a6390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f125c004af0 tx=0x7f125c005c50 comp rx=0 tx=0).stop 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 msgr2=0x7f1264100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 0x7f1264100e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f126000ea60 tx=0x7f126000ef20 comp rx=0 tx=0).stop 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 shutdown_connections 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126410c500 0x7f1264101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f12500a3ed0 0x7f12500a6390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1264103ae0 0x7f1264101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 --2- 172.21.15.5:0/1503113903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12641031c0 0x7f1264100e10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 >> 172.21.15.5:0/1503113903 conn(0x7f1264078000 msgr2=0x7f12640fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 shutdown_connections 2024-04-22T13:28:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.324+0000 7f126bef1640 1 -- 172.21.15.5:0/1503113903 wait complete. 2024-04-22T13:28:20.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 -- 172.21.15.5:0/2912985334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c8103ae0 msgr2=0x7f06c810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/2912985334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c8103ae0 0x7f06c810bfc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009900 tx=0x7f06b802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 -- 172.21.15.5:0/2912985334 shutdown_connections 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/2912985334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c8103ae0 0x7f06c810bfc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/2912985334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c810c500 0x7f06c810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/2912985334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06c81031c0 0x7f06c81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 -- 172.21.15.5:0/2912985334 >> 172.21.15.5:0/2912985334 conn(0x7f06c8078000 msgr2=0x7f06c8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 -- 172.21.15.5:0/2912985334 shutdown_connections 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.440+0000 7f06cfcc8640 1 -- 172.21.15.5:0/2912985334 wait complete. 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 Processor -- start 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 -- start start 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 0x7f06c8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06c8103ae0 0x7f06c8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c810c500 0x7f06c8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8112190 con 0x7f06c81031c0 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8112010 con 0x7f06c8103ae0 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cfcc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8112310 con 0x7f06c810c500 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 0x7f06c8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 0x7f06c8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55904/0 (socket says 172.21.15.5:55904) 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 -- 172.21.15.5:0/3047372379 learned_addr learned my addr 172.21.15.5:0/3047372379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cd263640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06c8103ae0 0x7f06c8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 -- 172.21.15.5:0/3047372379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c810c500 msgr2=0x7f06c8101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c810c500 0x7f06c8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 -- 172.21.15.5:0/3047372379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06c8103ae0 msgr2=0x7f06c8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06c8103ae0 0x7f06c8101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.441+0000 7f06cda64640 1 -- 172.21.15.5:0/3047372379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c81a6cd0 con 0x7f06c81031c0 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.442+0000 7f06cda64640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 0x7f06c8100e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f06c400d950 tx=0x7f06c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.442+0000 7f06beffd640 1 -- 172.21.15.5:0/3047372379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4017030 con 0x7f06c81031c0 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.442+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c81a6fc0 con 0x7f06c81031c0 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.442+0000 7f06beffd640 1 -- 172.21.15.5:0/3047372379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c4004020 con 0x7f06c81031c0 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.442+0000 7f06beffd640 1 -- 172.21.15.5:0/3047372379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c40047a0 con 0x7f06c81031c0 2024-04-22T13:28:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.442+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c81a7480 con 0x7f06c81031c0 2024-04-22T13:28:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.445+0000 7f06beffd640 1 -- 172.21.15.5:0/3047372379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f06c4009a30 con 0x7f06c81031c0 2024-04-22T13:28:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06d2999740 1 Processor -- start 2024-04-22T13:28:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06d2999740 1 -- start start 2024-04-22T13:28:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06beffd640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f06b40a3ed0 0x7f06b40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06d2999740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbaff58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06d2999740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbaffe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06beffd640 1 -- 172.21.15.5:0/3047372379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f06c40d05c0 con 0x7f06c81031c0 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06ce265640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f06b40a3ed0 0x7f06b40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06cd263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbaffe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06cd263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbaffe430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55910/0 (socket says 172.21.15.5:55910) 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.446+0000 7f06cda64640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbaff58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cd263640 1 -- 172.21.15.5:0/3269899900 learned_addr learned my addr 172.21.15.5:0/3269899900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06d2999740 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaffea00 0x560bbb000e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bbaf5a1f0 con 0x560bbaff5e80 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560bbaef74b0 con 0x560bbaffea00 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560bbaf561b0 con 0x560bbaff54d0 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cda64640 1 -- 172.21.15.5:0/3269899900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaffea00 msgr2=0x560bbb000e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06ce265640 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaffea00 0x560bbb000e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cda64640 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaffea00 0x560bbb000e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cda64640 1 -- 172.21.15.5:0/3269899900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 msgr2=0x560bbaffe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cda64640 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbaffe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cda64640 1 -- 172.21.15.5:0/3269899900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bbaee91b0 con 0x560bbaff54d0 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cd263640 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbaffe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06ce265640 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaffea00 0x560bbb000e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06ce265640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f06b40a3ed0 0x7f06b40a6390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c00099e0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06cda64640 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbaff58b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06c40cdb00 tx=0x7f06c40af890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d94e3cc9d9cd4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.447+0000 7f06b3fff640 1 -- 172.21.15.5:0/3269899900 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c40af1e0 con 0x560bbaff54d0 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.448+0000 7f06b3fff640 1 -- 172.21.15.5:0/3269899900 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c409c350 con 0x560bbaff54d0 2024-04-22T13:28:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.448+0000 7f06b3fff640 1 -- 172.21.15.5:0/3269899900 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c40989c0 con 0x560bbaff54d0 2024-04-22T13:28:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 msgr2=0x560bbaff58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbaff58b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06c40cdb00 tx=0x7f06c40af890 comp rx=0 tx=0).stop 2024-04-22T13:28:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 shutdown_connections 2024-04-22T13:28:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbaffe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaffea00 0x560bbb000e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 --2- 172.21.15.5:0/3269899900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbaff58b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 >> 172.21.15.5:0/3269899900 conn(0x560bbafefba0 msgr2=0x560bbafeff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 shutdown_connections 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 -- 172.21.15.5:0/3269899900 wait complete. 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 -- start start 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.449+0000 7f06d2999740 1 Processor -- start 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 0x560bbaffe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbaffb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbaffbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bbb001970 con 0x560bbaff54d0 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560bbaf13500 con 0x560bbaff5e80 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560bbb001af0 con 0x560bbaffea00 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 0x560bbaffe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 0x560bbaffe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55912/0 (socket says 172.21.15.5:55912) 2024-04-22T13:28:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 -- 172.21.15.5:0/2519746477 learned_addr learned my addr 172.21.15.5:0/2519746477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06d2999740 10 client.0 _ll_register_callbacks cb 0x560bbaee1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cd263640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbaffb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06ce265640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbaffbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 -- 172.21.15.5:0/2519746477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 msgr2=0x560bbaffbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbaffbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 -- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 msgr2=0x560bbaffb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbaffb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06cda64640 1 -- 172.21.15.5:0/2519746477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bbac6c3f0 con 0x560bbaff54d0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.450+0000 7f06ce265640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbaffbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06cda64640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 0x560bbaffe310 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f06c40a4a90 tx=0x7f06c409a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4238bb7a2c905b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06b17fa640 1 -- 172.21.15.5:0/2519746477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c40a4fc0 con 0x560bbaff54d0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c409c350 con 0x560bbaff54d0 2024-04-22T13:28:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06b17fa640 1 -- 172.21.15.5:0/2519746477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f06c40989c0 con 0x560bbaff54d0 2024-04-22T13:28:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06b17fa640 1 -- 172.21.15.5:0/2519746477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06c40ac150 con 0x560bbaff54d0 2024-04-22T13:28:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06d2999740 10 client.16158 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bbad367d0 con 0x560bbaff54d0 2024-04-22T13:28:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.451+0000 7f06b17fa640 1 -- 172.21.15.5:0/2519746477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c409c350 con 0x560bbaff54d0 2024-04-22T13:28:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06b17fa640 1 -- 172.21.15.5:0/2519746477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06c40a9b90 con 0x560bbaff54d0 2024-04-22T13:28:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06d2999740 10 client.16158 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06d2999740 10 client.16158 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06d2999740 20 client.16158 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06d2999740 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb00ec10 0x560bbb011060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06d2999740 4 client.16158 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bbaef9930 con 0x560bbb00ec10 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.452+0000 7f06cd263640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb00ec10 0x560bbb011060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.453+0000 7f06cd263640 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb00ec10 0x560bbb011060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42867ab50bab6228 server_cookie=fc3dae30f9fb4209 in_seq=0 out_seq=0 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.453+0000 7f06b17fa640 10 client.16158 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06b17fa640 1 -- 172.21.15.5:0/2519746477 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bbaef9930 con 0x560bbb00ec10 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06b17fa640 10 client.16158 handle_command_reply: tid=0 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 client.16158 shutdown 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 msgr2=0x560bbaffe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 0x560bbaffe310 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f06c40a4a90 tx=0x7f06c409a650 comp rx=0 tx=0).stop 2024-04-22T13:28:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 shutdown_connections 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb00ec10 0x560bbb011060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbaffbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbaffb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 --2- 172.21.15.5:0/2519746477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff54d0 0x560bbaffe310 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 >> 172.21.15.5:0/2519746477 conn(0x560bbafefba0 msgr2=0x560bbaff18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06b17fa640 0 client.16158 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 shutdown_connections 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 1 -- 172.21.15.5:0/2519746477 wait complete. 2024-04-22T13:28:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.454+0000 7f06d2999740 20 client.16158 trim_cache size 0 max 16384 2024-04-22T13:28:20.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 Processor -- start 2024-04-22T13:28:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 -- start start 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff54d0 0x560bbafeaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbb00a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbb006d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ab880 con 0x560bbaff5e80 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ad3c0 con 0x560bbaff54d0 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06d2999740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81abf50 con 0x560bbaffea00 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06cd263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbb00a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06cd263640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbb00a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55916/0 (socket says 172.21.15.5:55916) 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06cd263640 1 -- 172.21.15.5:0/2297276970 learned_addr learned my addr 172.21.15.5:0/2297276970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06ce265640 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbb006d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.468+0000 7f06cda64640 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff54d0 0x560bbafeaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06cd263640 1 -- 172.21.15.5:0/2297276970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 msgr2=0x560bbb006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06cd263640 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbb006d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06cd263640 1 -- 172.21.15.5:0/2297276970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff54d0 msgr2=0x560bbafeaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06cd263640 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff54d0 0x560bbafeaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06cd263640 1 -- 172.21.15.5:0/2297276970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bbaffbfb0 con 0x560bbaff5e80 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06cd263640 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbb00a830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005940 tx=0x7f06b802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06aafcb96cfbc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06b37fe640 1 -- 172.21.15.5:0/2297276970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004180 con 0x560bbaff5e80 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06b37fe640 1 -- 172.21.15.5:0/2297276970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b804a070 con 0x560bbaff5e80 2024-04-22T13:28:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.469+0000 7f06b37fe640 1 -- 172.21.15.5:0/2297276970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8005500 con 0x560bbaff5e80 2024-04-22T13:28:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 -- 172.21.15.5:0/2297276970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 msgr2=0x560bbb00a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbb00a830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005940 tx=0x7f06b802c350 comp rx=0 tx=0).stop 2024-04-22T13:28:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 -- 172.21.15.5:0/2297276970 shutdown_connections 2024-04-22T13:28:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaff5e80 0x560bbb00a830 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaffea00 0x560bbb006d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 --2- 172.21.15.5:0/2297276970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff54d0 0x560bbafeaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 -- 172.21.15.5:0/2297276970 >> 172.21.15.5:0/2297276970 conn(0x560bbafefba0 msgr2=0x560bbaff6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 -- 172.21.15.5:0/2297276970 shutdown_connections 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.470+0000 7f06d2999740 1 -- 172.21.15.5:0/2297276970 wait complete. 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 -- start start 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 Processor -- start 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbb001d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbb002240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaffea00 0x560bbb073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81accf0 con 0x560bbaffea00 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ac620 con 0x560bbaff5e80 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81a9910 con 0x560bbaff54d0 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06d2999740 10 client.0 _ll_register_callbacks cb 0x7f06c40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06ce265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaffea00 0x560bbb073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06cda64640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbb001d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06ce265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaffea00 0x560bbb073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55924/0 (socket says 172.21.15.5:55924) 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06ce265640 1 -- 172.21.15.5:0/92850978 learned_addr learned my addr 172.21.15.5:0/92850978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.471+0000 7f06cd263640 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbb002240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06cda64640 1 -- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 msgr2=0x560bbb002240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06cda64640 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbb002240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06cda64640 1 -- 172.21.15.5:0/92850978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaffea00 msgr2=0x560bbb073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06cda64640 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaffea00 0x560bbb073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06cda64640 1 -- 172.21.15.5:0/92850978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bbb073670 con 0x560bbaff54d0 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06cda64640 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbb001d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06c40987b0 tx=0x7f06c4005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96cf11c78df6a067 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06b17fa640 1 -- 172.21.15.5:0/92850978 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c409bcb0 con 0x560bbaff54d0 2024-04-22T13:28:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06b17fa640 1 -- 172.21.15.5:0/92850978 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f06c40d0320 con 0x560bbaff54d0 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.472+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b804a070 con 0x560bbaff54d0 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06b17fa640 1 -- 172.21.15.5:0/92850978 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06c40a4fc0 con 0x560bbaff54d0 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 10 client.25756 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bbb074780 con 0x560bbaff54d0 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06b17fa640 1 -- 172.21.15.5:0/92850978 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b804a070 con 0x560bbaff54d0 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06b17fa640 1 -- 172.21.15.5:0/92850978 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06c409b210 con 0x560bbaff54d0 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 10 client.25756 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 10 client.25756 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 20 client.25756 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb074d00 0x560bbb0770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 4 client.25756 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bbb0113f0 con 0x560bbb074d00 2024-04-22T13:28:20.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.473+0000 7f06cd263640 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb074d00 0x560bbb0770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06cd263640 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb074d00 0x560bbb0770f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c9b8baadd6f3f6d server_cookie=b8e301a51b78b388 in_seq=0 out_seq=0 2024-04-22T13:28:20.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06b17fa640 10 client.25756 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06b17fa640 1 -- 172.21.15.5:0/92850978 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bbb0113f0 con 0x560bbb074d00 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06b17fa640 10 client.25756 handle_command_reply: tid=0 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 client.25756 shutdown 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 msgr2=0x560bbb001d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbb001d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06c40987b0 tx=0x7f06c4005d70 comp rx=0 tx=0).stop 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 shutdown_connections 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560bbb074d00 0x560bbb0770f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bbaffea00 0x560bbb073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560bbaff5e80 0x560bbb002240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 --2- 172.21.15.5:0/92850978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560bbaff54d0 0x560bbb001d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 >> 172.21.15.5:0/92850978 conn(0x560bbafefba0 msgr2=0x560bbaff6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06b17fa640 0 client.25756 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 shutdown_connections 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.474+0000 7f06d2999740 1 -- 172.21.15.5:0/92850978 wait complete. 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06d2999740 20 client.25756 trim_cache size 0 max 16384 2024-04-22T13:28:20.475 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f06b40a3ed0 msgr2=0x7f06b40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f06b40a3ed0 0x7f06b40a6390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c00099e0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 msgr2=0x7f06c8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 0x7f06c8100e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f06c400d950 tx=0x7f06c400de10 comp rx=0 tx=0).stop 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 shutdown_connections 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f06b40a3ed0 0x7f06b40a6390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c810c500 0x7f06c8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06c8103ae0 0x7f06c8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 --2- 172.21.15.5:0/3047372379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c81031c0 0x7f06c8100e10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 >> 172.21.15.5:0/3047372379 conn(0x7f06c8078000 msgr2=0x7f06c80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 shutdown_connections 2024-04-22T13:28:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.475+0000 7f06cfcc8640 1 -- 172.21.15.5:0/3047372379 wait complete. 2024-04-22T13:28:20.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:28:20.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 -- 172.21.15.5:0/719450205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e01031e0 msgr2=0x7f82e01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 --2- 172.21.15.5:0/719450205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e01031e0 0x7f82e01035c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009900 tx=0x7f82dc02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 -- 172.21.15.5:0/719450205 shutdown_connections 2024-04-22T13:28:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 --2- 172.21.15.5:0/719450205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82e010c520 0x7f82e010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 --2- 172.21.15.5:0/719450205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82e0103b00 0x7f82e010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 --2- 172.21.15.5:0/719450205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e01031e0 0x7f82e01035c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 -- 172.21.15.5:0/719450205 >> 172.21.15.5:0/719450205 conn(0x7f82e0078000 msgr2=0x7f82e0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 -- 172.21.15.5:0/719450205 shutdown_connections 2024-04-22T13:28:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.626+0000 7f82e7d50640 1 -- 172.21.15.5:0/719450205 wait complete. 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.627+0000 7f82e7d50640 1 Processor -- start 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.627+0000 7f82e7d50640 1 -- start start 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.627+0000 7f82e7d50640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82e0103b00 0x7f82e019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.627+0000 7f82e7d50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82e010c520 0x7f82e019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.627+0000 7f82e7d50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 0x7f82e0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 0x7f82e0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 0x7f82e0198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55934/0 (socket says 172.21.15.5:55934) 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82e01121d0 con 0x7f82e0198770 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 -- 172.21.15.5:0/1094427749 learned_addr learned my addr 172.21.15.5:0/1094427749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82e0112050 con 0x7f82e010c520 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82e0112350 con 0x7f82e0103b00 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82e0103b00 0x7f82e019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82e010c520 0x7f82e019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 -- 172.21.15.5:0/1094427749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82e0103b00 msgr2=0x7f82e019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82e0103b00 0x7f82e019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 -- 172.21.15.5:0/1094427749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82e010c520 msgr2=0x7f82e019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82e010c520 0x7f82e019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 -- 172.21.15.5:0/1094427749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e0199370 con 0x7f82e0198770 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.628+0000 7f82e62ed640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 0x7f82e0198bf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f82d8007440 tx=0x7f82d8007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.629+0000 7f82d6ffd640 1 -- 172.21.15.5:0/1094427749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8012c70 con 0x7f82e0198770 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.629+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e0199600 con 0x7f82e0198770 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.629+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e01010e0 con 0x7f82e0198770 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.629+0000 7f82d6ffd640 1 -- 172.21.15.5:0/1094427749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d8012e10 con 0x7f82e0198770 2024-04-22T13:28:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.629+0000 7f82d6ffd640 1 -- 172.21.15.5:0/1094427749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8005280 con 0x7f82e0198770 2024-04-22T13:28:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.632+0000 7f82d6ffd640 1 -- 172.21.15.5:0/1094427749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f82d8004a40 con 0x7f82e0198770 2024-04-22T13:28:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.632+0000 7f82eaa1b740 1 Processor -- start 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 -- start start 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8dae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8db7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8db9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ba8d131f0 con 0x559ba8dae4d0 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559ba8cb04b0 con 0x559ba8db7a00 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559ba8d0f1b0 con 0x559ba8daee80 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82e52eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8dae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82e52eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8dae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55944/0 (socket says 172.21.15.5:55944) 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82e52eb640 1 -- 172.21.15.5:0/3394684047 learned_addr learned my addr 172.21.15.5:0/3394684047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82e5aec640 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8db7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.633+0000 7f82e52eb640 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8db9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82d6ffd640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82cc0a3f50 0x7f82cc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e52eb640 1 -- 172.21.15.5:0/3394684047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 msgr2=0x559ba8db7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e52eb640 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8db7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e52eb640 1 -- 172.21.15.5:0/3394684047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 msgr2=0x559ba8db9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e52eb640 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8db9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e52eb640 1 -- 172.21.15.5:0/3394684047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba8ca21b0 con 0x559ba8dae4d0 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e62ed640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82cc0a3f50 0x7f82cc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82e52eb640 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8dae8b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f82d0011680 tx=0x7f82d0009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5d93e873bf29b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82d6ffd640 1 -- 172.21.15.5:0/1094427749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82d80d82c0 con 0x7f82e0198770 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82cbfff640 1 -- 172.21.15.5:0/3394684047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d001a030 con 0x559ba8dae4d0 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82cbfff640 1 -- 172.21.15.5:0/3394684047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d000fa30 con 0x559ba8dae4d0 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.634+0000 7f82cbfff640 1 -- 172.21.15.5:0/3394684047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d001eb10 con 0x559ba8dae4d0 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 -- 172.21.15.5:0/3394684047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 msgr2=0x559ba8dae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8dae8b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f82d0011680 tx=0x7f82d0009950 comp rx=0 tx=0).stop 2024-04-22T13:28:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82e62ed640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82cc0a3f50 0x7f82cc0a6410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82d80d5860 tx=0x7f82d80092b0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 -- 172.21.15.5:0/3394684047 shutdown_connections 2024-04-22T13:28:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8db9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8db7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/3394684047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8dae8b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 -- 172.21.15.5:0/3394684047 >> 172.21.15.5:0/3394684047 conn(0x559ba8da8ba0 msgr2=0x559ba8da8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 -- 172.21.15.5:0/3394684047 shutdown_connections 2024-04-22T13:28:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.635+0000 7f82eaa1b740 1 -- 172.21.15.5:0/3394684047 wait complete. 2024-04-22T13:28:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.636+0000 7f82eaa1b740 1 -- start start 2024-04-22T13:28:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.636+0000 7f82eaa1b740 1 Processor -- start 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.636+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8db72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8daee80 0x559ba8db4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8db7a00 0x559ba8db4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ba8dba970 con 0x559ba8dae4d0 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559ba8ccc500 con 0x559ba8daee80 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559ba8dbaaf0 con 0x559ba8db7a00 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8db72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82eaa1b740 10 client.0 _ll_register_callbacks cb 0x559ba8c9ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8db72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55950/0 (socket says 172.21.15.5:55950) 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 -- 172.21.15.5:0/1658106006 learned_addr learned my addr 172.21.15.5:0/1658106006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8daee80 0x559ba8db4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 -- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8db7a00 msgr2=0x559ba8db4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8db7a00 0x559ba8db4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 -- 172.21.15.5:0/1658106006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8daee80 msgr2=0x559ba8db4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8daee80 0x559ba8db4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.637+0000 7f82e52eb640 1 -- 172.21.15.5:0/1658106006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba8a253f0 con 0x559ba8dae4d0 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8db72e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f82d0014040 tx=0x7f82d000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5819634476fbb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82c97fa640 1 -- 172.21.15.5:0/1658106006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d001a040 con 0x559ba8dae4d0 2024-04-22T13:28:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82c97fa640 1 -- 172.21.15.5:0/1658106006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82d0004220 con 0x559ba8dae4d0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d000fa30 con 0x559ba8dae4d0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82c97fa640 1 -- 172.21.15.5:0/1658106006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82d0004420 con 0x559ba8dae4d0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82eaa1b740 10 client.16176 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ba8aef7d0 con 0x559ba8dae4d0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.638+0000 7f82c97fa640 1 -- 172.21.15.5:0/1658106006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d000fa30 con 0x559ba8dae4d0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82c97fa640 1 -- 172.21.15.5:0/1658106006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f82d0026030 con 0x559ba8dae4d0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82eaa1b740 10 client.16176 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82eaa1b740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82eaa1b740 20 client.16176 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8dc9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82eaa1b740 4 client.16176 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.639+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ba8cb2930 con 0x559ba8dc7a60 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.640+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8dc9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.640+0000 7f82c97fa640 10 client.16176 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.640+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8dc9e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c12ac41738b7ca47 server_cookie=b77039ea741686a3 in_seq=0 out_seq=0 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82c97fa640 1 -- 172.21.15.5:0/1658106006 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ba8cb2930 con 0x559ba8dc7a60 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82c97fa640 10 client.16176 handle_command_reply: tid=0 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 client.16176 shutdown 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 msgr2=0x559ba8db72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8db72e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f82d0014040 tx=0x7f82d000f790 comp rx=0 tx=0).stop 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 shutdown_connections 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8dc9e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82c97fa640 0 client.16176 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8db7a00 0x559ba8db4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8daee80 0x559ba8db4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1658106006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8dae4d0 0x559ba8db72e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.641+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 >> 172.21.15.5:0/1658106006 conn(0x559ba8da8ba0 msgr2=0x559ba8daa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.642+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 shutdown_connections 2024-04-22T13:28:20.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.642+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1658106006 wait complete. 2024-04-22T13:28:20.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.642+0000 7f82eaa1b740 20 client.16176 trim_cache size 0 max 16384 2024-04-22T13:28:20.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.655+0000 7f82eaa1b740 1 Processor -- start 2024-04-22T13:28:20.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.655+0000 7f82eaa1b740 1 -- start start 2024-04-22T13:28:20.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8dae4d0 0x559ba8dc3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8daa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 0x559ba8db6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82c400e6d0 con 0x559ba8db7a00 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4010210 con 0x559ba8dae4d0 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82eaa1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82c400eda0 con 0x559ba8daee80 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 0x559ba8db6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 0x559ba8db6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55954/0 (socket says 172.21.15.5:55954) 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 -- 172.21.15.5:0/780648460 learned_addr learned my addr 172.21.15.5:0/780648460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8dae4d0 0x559ba8dc3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e5aec640 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8daa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 -- 172.21.15.5:0/780648460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 msgr2=0x559ba8daa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8daa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 -- 172.21.15.5:0/780648460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8dae4d0 msgr2=0x559ba8dc3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8dae4d0 0x559ba8dc3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 -- 172.21.15.5:0/780648460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba8daec20 con 0x559ba8db7a00 2024-04-22T13:28:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.656+0000 7f82e52eb640 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 0x559ba8db6da0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f82d0014010 tx=0x7f82d0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbc805cda2e062b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.657+0000 7f82cb7fe640 1 -- 172.21.15.5:0/780648460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d000fa30 con 0x559ba8db7a00 2024-04-22T13:28:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.657+0000 7f82cb7fe640 1 -- 172.21.15.5:0/780648460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d000f800 con 0x559ba8db7a00 2024-04-22T13:28:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.657+0000 7f82cb7fe640 1 -- 172.21.15.5:0/780648460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0004ac0 con 0x559ba8db7a00 2024-04-22T13:28:20.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 -- 172.21.15.5:0/780648460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 msgr2=0x559ba8db6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 0x559ba8db6da0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f82d0014010 tx=0x7f82d0005970 comp rx=0 tx=0).stop 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 -- 172.21.15.5:0/780648460 shutdown_connections 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8db7a00 0x559ba8db6da0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8daee80 0x559ba8daa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/780648460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8dae4d0 0x559ba8dc3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 -- 172.21.15.5:0/780648460 >> 172.21.15.5:0/780648460 conn(0x559ba8da8ba0 msgr2=0x559ba8da9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 -- 172.21.15.5:0/780648460 shutdown_connections 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 -- 172.21.15.5:0/780648460 wait complete. 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 -- start start 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.658+0000 7f82eaa1b740 1 Processor -- start 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8dae4d0 0x559ba8dbf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82eaa1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 0x559ba8dbff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82e5aec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 0x559ba8dbff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82e5aec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 0x559ba8dbff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55956/0 (socket says 172.21.15.5:55956) 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82e5aec640 1 -- 172.21.15.5:0/1300783621 learned_addr learned my addr 172.21.15.5:0/1300783621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8dae4d0 0x559ba8dbf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.659+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8dbae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82c400fb40 con 0x559ba8daee80 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82c400f470 con 0x559ba8db7a00 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82c400c6b0 con 0x559ba8dae4d0 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82e5aec640 1 -- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8dae4d0 msgr2=0x559ba8dbf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8dae4d0 0x559ba8dbf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82e5aec640 1 -- 172.21.15.5:0/1300783621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 msgr2=0x559ba8dbae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8dbae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82e5aec640 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba8dbb560 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82e5aec640 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 0x559ba8dbff20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0096c0 tx=0x7f82dc004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275cc0f1578bc456 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82eaa1b740 10 client.0 _ll_register_callbacks cb 0x7f82d001e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d000f800 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.660+0000 7f82c97fa640 1 -- 172.21.15.5:0/1300783621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc000e60 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82c97fa640 1 -- 172.21.15.5:0/1300783621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82dc02ca90 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82c97fa640 1 -- 172.21.15.5:0/1300783621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82dc02cc90 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=27}) -- 0x559ba8e2d770 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82c97fa640 1 -- 172.21.15.5:0/1300783621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f82dc049070 con 0x559ba8daee80 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 10 client.16182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 20 client.16182 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8e2fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 4 client.16182 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.661+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ba8d20c10 con 0x559ba8dc7a60 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.662+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8e2fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.662+0000 7f82e52eb640 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8e2fdc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7647a8f4dc659d2 server_cookie=9263ec6b72e03ea in_seq=0 out_seq=0 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.662+0000 7f82c97fa640 10 client.16182 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.662+0000 7f82c97fa640 1 -- 172.21.15.5:0/1300783621 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ba8d20c10 con 0x559ba8dc7a60 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.662+0000 7f82c97fa640 10 client.16182 handle_command_reply: tid=0 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.662+0000 7f82eaa1b740 1 client.16182 shutdown 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 msgr2=0x559ba8dbff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 0x559ba8dbff20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0096c0 tx=0x7f82dc004440 comp rx=0 tx=0).stop 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 shutdown_connections 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x559ba8dc7a60 0x559ba8e2fdc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82c97fa640 0 client.16182 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ba8daee80 0x559ba8dbff20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ba8db7a00 0x559ba8dbae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 --2- 172.21.15.5:0/1300783621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ba8dae4d0 0x559ba8dbf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 >> 172.21.15.5:0/1300783621 conn(0x559ba8da8ba0 msgr2=0x559ba8daa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 shutdown_connections 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 1 -- 172.21.15.5:0/1300783621 wait complete. 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.663+0000 7f82eaa1b740 20 client.16182 trim_cache size 0 max 16384 2024-04-22T13:28:20.664 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82cc0a3f50 msgr2=0x7f82cc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82cc0a3f50 0x7f82cc0a6410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82d80d5860 tx=0x7f82d80092b0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 msgr2=0x7f82e0198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 0x7f82e0198bf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f82d8007440 tx=0x7f82d8007900 comp rx=0 tx=0).stop 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 shutdown_connections 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82e0198770 0x7f82e0198bf0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82cc0a3f50 0x7f82cc0a6410 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82e010c520 0x7f82e019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 --2- 172.21.15.5:0/1094427749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82e0103b00 0x7f82e019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.664+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 >> 172.21.15.5:0/1094427749 conn(0x7f82e0078000 msgr2=0x7f82e00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.665+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 shutdown_connections 2024-04-22T13:28:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.665+0000 7f82e7d50640 1 -- 172.21.15.5:0/1094427749 wait complete. 2024-04-22T13:28:20.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 -- 172.21.15.5:0/2492071677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c101070 msgr2=0x7f6a9c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 --2- 172.21.15.5:0/2492071677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c101070 0x7f6a9c101450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009860 tx=0x7f6a9802bb70 comp rx=0 tx=0).stop 2024-04-22T13:28:20.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 -- 172.21.15.5:0/2492071677 shutdown_connections 2024-04-22T13:28:20.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 --2- 172.21.15.5:0/2492071677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c106140 0x7f6a9c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 --2- 172.21.15.5:0/2492071677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a9c101a20 0x7f6a9c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 --2- 172.21.15.5:0/2492071677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c101070 0x7f6a9c101450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.669+0000 7f6aa328c640 1 -- 172.21.15.5:0/2492071677 >> 172.21.15.5:0/2492071677 conn(0x7f6a9c0771d0 msgr2=0x7f6a9c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.670+0000 7f6aa328c640 1 -- 172.21.15.5:0/2492071677 shutdown_connections 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.670+0000 7f6aa328c640 1 -- 172.21.15.5:0/2492071677 wait complete. 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.670+0000 7f6aa328c640 1 Processor -- start 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 -- start start 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 0x7f6a9c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c106140 0x7f6a9c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a9c19a770 0x7f6a9c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c10ff40 con 0x7f6a9c101070 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c10fdc0 con 0x7f6a9c19a770 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa328c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1100c0 con 0x7f6a9c106140 2024-04-22T13:28:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 0x7f6a9c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 0x7f6a9c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55976/0 (socket says 172.21.15.5:55976) 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 -- 172.21.15.5:0/356237072 learned_addr learned my addr 172.21.15.5:0/356237072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa0827640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c106140 0x7f6a9c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1829640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a9c19a770 0x7f6a9c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 -- 172.21.15.5:0/356237072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c106140 msgr2=0x7f6a9c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c106140 0x7f6a9c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 -- 172.21.15.5:0/356237072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a9c19a770 msgr2=0x7f6a9c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a9c19a770 0x7f6a9c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa1028640 1 -- 172.21.15.5:0/356237072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c19b370 con 0x7f6a9c101070 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.671+0000 7f6aa0827640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c106140 0x7f6a9c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.672+0000 7f6aa1028640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 0x7f6a9c1a06b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6a98037040 tx=0x7f6a980359a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.672+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98041020 con 0x7f6a9c101070 2024-04-22T13:28:20.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.672+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c19b600 con 0x7f6a9c101070 2024-04-22T13:28:20.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.672+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a9802c9c0 con 0x7f6a9c101070 2024-04-22T13:28:20.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.672+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98039040 con 0x7f6a9c101070 2024-04-22T13:28:20.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.672+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c1a7740 con 0x7f6a9c101070 2024-04-22T13:28:20.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.673+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c102540 con 0x7f6a9c101070 2024-04-22T13:28:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.675+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f6a98044020 con 0x7f6a9c101070 2024-04-22T13:28:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.676+0000 7f6a927fc640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6a880a3ed0 0x7f6a880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.676+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6a980f2170 con 0x7f6a9c101070 2024-04-22T13:28:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.676+0000 7f6aa0827640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6a880a3ed0 0x7f6a880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.677+0000 7f6aa0827640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6a880a3ed0 0x7f6a880a6390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c19bfc0 tx=0x7f6a8c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:28:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.684+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a9804b020 con 0x7f6a9c101070 2024-04-22T13:28:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 -- 172.21.15.5:0/1404093884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d810c500 msgr2=0x7ff4d810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/1404093884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d810c500 0x7ff4d810e9a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009ca0 tx=0x7ff4d002c180 comp rx=0 tx=0).stop 2024-04-22T13:28:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 -- 172.21.15.5:0/1404093884 shutdown_connections 2024-04-22T13:28:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/1404093884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d810c500 0x7ff4d810e9a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/1404093884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d8103ae0 0x7ff4d810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/1404093884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d81031c0 0x7ff4d81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 -- 172.21.15.5:0/1404093884 >> 172.21.15.5:0/1404093884 conn(0x7ff4d8078000 msgr2=0x7ff4d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 -- 172.21.15.5:0/1404093884 shutdown_connections 2024-04-22T13:28:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.814+0000 7ff4dfe31640 1 -- 172.21.15.5:0/1404093884 wait complete. 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 Processor -- start 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 -- start start 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 0x7ff4d8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8103ae0 0x7ff4d8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d810c500 0x7ff4d8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8112170 con 0x7ff4d81031c0 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8111ff0 con 0x7ff4d8103ae0 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4dfe31640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81122f0 con 0x7ff4d810c500 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.815+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 0x7ff4d8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 0x7ff4d8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55996/0 (socket says 172.21.15.5:55996) 2024-04-22T13:28:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/2315553619 learned_addr learned my addr 172.21.15.5:0/2315553619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8103ae0 0x7ff4d8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d810c500 0x7ff4d8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/2315553619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d810c500 msgr2=0x7ff4d8101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d810c500 0x7ff4d8101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 msgr2=0x7ff4d8100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 0x7ff4d8100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/2315553619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d81a6bb0 con 0x7ff4d8103ae0 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 0x7ff4d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d810c500 0x7ff4d8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.816+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8103ae0 0x7ff4d8101350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800e9f0 tx=0x7ff4c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.817+0000 7ff4ceffd640 1 -- 172.21.15.5:0/2315553619 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800ccc0 con 0x7ff4d8103ae0 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.817+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d81a6ea0 con 0x7ff4d8103ae0 2024-04-22T13:28:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.817+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d81a73b0 con 0x7ff4d8103ae0 2024-04-22T13:28:20.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.817+0000 7ff4ceffd640 1 -- 172.21.15.5:0/2315553619 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c80108e0 con 0x7ff4d8103ae0 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.818+0000 7ff4ceffd640 1 -- 172.21.15.5:0/2315553619 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8018880 con 0x7ff4d8103ae0 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.820+0000 7ff4e2a1e740 1 Processor -- start 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.820+0000 7ff4e2a1e740 1 -- start start 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4ceffd640 1 -- 172.21.15.5:0/2315553619 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7ff4c8010430 con 0x7ff4d8103ae0 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a7e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a92620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576f79e31f0 con 0x5576f7a7e470 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5576f79804b0 con 0x5576f7a7ee20 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576f79df1b0 con 0x5576f7a90210 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4dd3cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a92620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4de3ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a7e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4de3ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a7e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56002/0 (socket says 172.21.15.5:56002) 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4de3ce640 1 -- 172.21.15.5:0/1303386313 learned_addr learned my addr 172.21.15.5:0/1303386313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4ceffd640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4c40a3db0 0x7ff4c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/1303386313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 msgr2=0x5576f7a92620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ceffd640 1 -- 172.21.15.5:0/2315553619 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff4c80d0c30 con 0x7ff4d8103ae0 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.821+0000 7ff4dd3cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a92620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:53828/0 (socket says 172.21.15.5:53828) 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a92620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/1303386313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 msgr2=0x5576f7a7e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a7e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/1303386313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f79721b0 con 0x5576f7a7ee20 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a7e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a92620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4c40a3db0 0x7ff4c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8fc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40058c0 tx=0x7ff4d40109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ccece4fa75af254 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4c3fff640 1 -- 172.21.15.5:0/1303386313 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40117c0 con 0x5576f7a7ee20 2024-04-22T13:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4c3fff640 1 -- 172.21.15.5:0/1303386313 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d4011e10 con 0x5576f7a7ee20 2024-04-22T13:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.822+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4c40a3db0 0x7ff4c40a6270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0037040 tx=0x7ff4d0032340 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.823+0000 7ff4c3fff640 1 -- 172.21.15.5:0/1303386313 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d401be60 con 0x5576f7a7ee20 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.823+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1303386313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 msgr2=0x5576f7a8fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.823+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8fc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40058c0 tx=0x7ff4d40109e0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.823+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1303386313 shutdown_connections 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.823+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a92620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.823+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8fc40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.824+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1303386313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a7e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.824+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1303386313 >> 172.21.15.5:0/1303386313 conn(0x5576f7a78ba0 msgr2=0x5576f7a78f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.824+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1303386313 shutdown_connections 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.824+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1303386313 wait complete. 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.824+0000 7ff4e2a1e740 1 -- start start 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.824+0000 7ff4e2a1e740 1 Processor -- start 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a8d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576f7a93140 con 0x5576f7a7e470 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5576f799c500 con 0x5576f7a7ee20 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576f7a932c0 con 0x5576f7a90210 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4e2a1e740 10 client.0 _ll_register_callbacks cb 0x5576f796ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56018/0 (socket says 172.21.15.5:56018) 2024-04-22T13:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/2530838529 learned_addr learned my addr 172.21.15.5:0/2530838529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a8d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/2530838529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 msgr2=0x5576f7a8d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a8d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 msgr2=0x5576f7a8cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/2530838529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f76f53f0 con 0x5576f7a7e470 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.825+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a8fb80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000c00 tx=0x7ff4d401b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7028733d1edcc02 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4c17fa640 1 -- 172.21.15.5:0/2530838529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d402a060 con 0x5576f7a7e470 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4c17fa640 1 -- 172.21.15.5:0/2530838529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff4d401b950 con 0x5576f7a7e470 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d4011e10 con 0x5576f7a7e470 2024-04-22T13:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4c17fa640 1 -- 172.21.15.5:0/2530838529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4d4004030 con 0x5576f7a7e470 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4e2a1e740 10 client.16200 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.826+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576f77bf7d0 con 0x5576f7a7e470 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4c17fa640 1 -- 172.21.15.5:0/2530838529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d4011e10 con 0x5576f7a7e470 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4c17fa640 1 -- 172.21.15.5:0/2530838529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4d40116d0 con 0x5576f7a7e470 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4e2a1e740 10 client.16200 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4e2a1e740 10 client.16200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4e2a1e740 20 client.16200 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7aa0320 0x5576f7aa2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4e2a1e740 4 client.16200 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f7982930 con 0x5576f7aa0320 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.827+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7aa0320 0x5576f7aa2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.828+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7aa0320 0x5576f7aa2710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f853aad1189c2445 server_cookie=51acebc198a184cf in_seq=0 out_seq=0 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.828+0000 7ff4c17fa640 10 client.16200 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.829+0000 7ff4c17fa640 1 -- 172.21.15.5:0/2530838529 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576f7982930 con 0x5576f7aa0320 2024-04-22T13:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.829+0000 7ff4c17fa640 10 client.16200 handle_command_reply: tid=0 2024-04-22T13:28:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.829+0000 7ff4e2a1e740 1 client.16200 shutdown 2024-04-22T13:28:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 msgr2=0x5576f7a8fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a8fb80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000c00 tx=0x7ff4d401b660 comp rx=0 tx=0).stop 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 shutdown_connections 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7aa0320 0x5576f7aa2710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a8d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7ee20 0x5576f7a8cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/2530838529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a8fb80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4c17fa640 0 client.16200 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 >> 172.21.15.5:0/2530838529 conn(0x5576f7a78ba0 msgr2=0x5576f7a7a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 shutdown_connections 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/2530838529 wait complete. 2024-04-22T13:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.832+0000 7ff4e2a1e740 20 client.16200 trim_cache size 0 max 16384 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.858+0000 7ff4e2a1e740 1 Processor -- start 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.858+0000 7ff4e2a1e740 1 -- start start 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a7ee20 0x5576f7a882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a90210 0x5576f7aa7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc00e6d0 con 0x5576f7a7e470 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc010210 con 0x5576f7a90210 2024-04-22T13:28:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc00eda0 con 0x5576f7a7ee20 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4dd3cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a7ee20 0x5576f7a882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4ddbcd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a9c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56022/0 (socket says 172.21.15.5:56022) 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4ddbcd640 1 -- 172.21.15.5:0/3425272958 learned_addr learned my addr 172.21.15.5:0/3425272958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.859+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a90210 0x5576f7aa7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4de3ce640 1 -- 172.21.15.5:0/3425272958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a7ee20 msgr2=0x5576f7a882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a7ee20 0x5576f7a882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4de3ce640 1 -- 172.21.15.5:0/3425272958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 msgr2=0x5576f7a9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4de3ce640 1 -- 172.21.15.5:0/3425272958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f7a8d890 con 0x5576f7a90210 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a90210 0x5576f7aa7d60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4d002c6c0 tx=0x7ff4d0047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b0c9953c41aabae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4c37fe640 1 -- 172.21.15.5:0/3425272958 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0051020 con 0x5576f7a90210 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4c37fe640 1 -- 172.21.15.5:0/3425272958 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0004610 con 0x5576f7a90210 2024-04-22T13:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.860+0000 7ff4c37fe640 1 -- 172.21.15.5:0/3425272958 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0039040 con 0x5576f7a90210 2024-04-22T13:28:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/3425272958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a90210 msgr2=0x5576f7aa7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a90210 0x5576f7aa7d60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4d002c6c0 tx=0x7ff4d0047000 comp rx=0 tx=0).stop 2024-04-22T13:28:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/3425272958 shutdown_connections 2024-04-22T13:28:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a90210 0x5576f7aa7d60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a7ee20 0x5576f7a882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/3425272958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7e470 0x5576f7a9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/3425272958 >> 172.21.15.5:0/3425272958 conn(0x5576f7a78ba0 msgr2=0x5576f7a87be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/3425272958 shutdown_connections 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.861+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/3425272958 wait complete. 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.862+0000 7ff4e2a1e740 1 -- start start 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.862+0000 7ff4e2a1e740 1 Processor -- start 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.862+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7e470 0x5576f7a986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.862+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 0x5576f7a93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.862+0000 7ff4e2a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.862+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc00fb40 con 0x5576f7a7ee20 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc00f470 con 0x5576f7a7e470 2024-04-22T13:28:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4e2a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc00c6b0 con 0x5576f7a90210 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 0x5576f7a93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 0x5576f7a93b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56034/0 (socket says 172.21.15.5:56034) 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/1554490337 learned_addr learned my addr 172.21.15.5:0/1554490337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4e2a1e740 10 client.0 _ll_register_callbacks cb 0x7ff4d400be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7e470 0x5576f7a986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4de3ce640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/1554490337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 msgr2=0x5576f7a94090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a94090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7e470 msgr2=0x5576f7a986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7e470 0x5576f7a986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 -- 172.21.15.5:0/1554490337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f7a94600 con 0x5576f7a7ee20 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4dd3cc640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 0x5576f7a93b50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8098ed0 tx=0x7ff4c80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8e98ee2f7d0e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.863+0000 7ff4c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4c17fa640 1 -- 172.21.15.5:0/1554490337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80a5630 con 0x5576f7a7ee20 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4c17fa640 1 -- 172.21.15.5:0/1554490337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff4c809bec0 con 0x5576f7a7ee20 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d0004610 con 0x5576f7a7ee20 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4c17fa640 1 -- 172.21.15.5:0/1554490337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4c809c0c0 con 0x5576f7a7ee20 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4e2a1e740 10 client.16206 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576f7b063b0 con 0x5576f7a7ee20 2024-04-22T13:28:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.864+0000 7ff4c17fa640 1 -- 172.21.15.5:0/1554490337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c80bbb80 con 0x5576f7a7ee20 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4c17fa640 1 -- 172.21.15.5:0/1554490337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4c809b930 con 0x5576f7a7ee20 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4e2a1e740 10 client.16206 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4e2a1e740 10 client.16206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4e2a1e740 20 client.16206 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7b06b30 0x5576f7b08f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4e2a1e740 4 client.16206 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff4d0004610 con 0x5576f7b06b30 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.865+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7b06b30 0x5576f7b08f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4c17fa640 10 client.16206 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4ddbcd640 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7b06b30 0x5576f7b08f50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=294e0e6ae1f19999 server_cookie=c0729708446c1634 in_seq=0 out_seq=0 2024-04-22T13:28:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4c17fa640 1 -- 172.21.15.5:0/1554490337 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff4d0004610 con 0x5576f7b06b30 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4c17fa640 10 client.16206 handle_command_reply: tid=0 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 client.16206 shutdown 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 msgr2=0x5576f7a93b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 0x5576f7a93b50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8098ed0 tx=0x7ff4c80bd040 comp rx=0 tx=0).stop 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 shutdown_connections 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5576f7b06b30 0x5576f7b08f50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576f7a7ee20 0x5576f7a93b50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4c17fa640 0 client.16206 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576f7a90210 0x5576f7a94090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 --2- 172.21.15.5:0/1554490337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f7a7e470 0x5576f7a986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.866+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 >> 172.21.15.5:0/1554490337 conn(0x5576f7a78ba0 msgr2=0x5576f7a77c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.867+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 shutdown_connections 2024-04-22T13:28:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.867+0000 7ff4e2a1e740 1 -- 172.21.15.5:0/1554490337 wait complete. 2024-04-22T13:28:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.867+0000 7ff4e2a1e740 20 client.16206 trim_cache size 0 max 16384 2024-04-22T13:28:20.867 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4c40a3db0 msgr2=0x7ff4c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4c40a3db0 0x7ff4c40a6270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0037040 tx=0x7ff4d0032340 comp rx=0 tx=0).stop 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8103ae0 msgr2=0x7ff4d8101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8103ae0 0x7ff4d8101350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800e9f0 tx=0x7ff4c800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 shutdown_connections 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4c40a3db0 0x7ff4c40a6270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4d810c500 0x7ff4d8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4d8103ae0 0x7ff4d8101350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 --2- 172.21.15.5:0/2315553619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4d81031c0 0x7ff4d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 >> 172.21.15.5:0/2315553619 conn(0x7ff4d8078000 msgr2=0x7ff4d80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 shutdown_connections 2024-04-22T13:28:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:20.868+0000 7ff4dfe31640 1 -- 172.21.15.5:0/2315553619 wait complete. 2024-04-22T13:28:20.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:28:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.019+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a9c1093a0 con 0x7f6a9c101070 2024-04-22T13:28:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.020+0000 7f6a927fc640 1 -- 172.21.15.5:0/356237072 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f6a980be710 con 0x7f6a9c101070 2024-04-22T13:28:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:28:21.016 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:28:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:28:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6a880a3ed0 msgr2=0x7f6a880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6a880a3ed0 0x7f6a880a6390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c19bfc0 tx=0x7f6a8c005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 msgr2=0x7f6a9c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 0x7f6a9c1a06b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6a98037040 tx=0x7f6a980359a0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 shutdown_connections 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6a880a3ed0 0x7f6a880a6390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a9c19a770 0x7f6a9c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a9c106140 0x7f6a9c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 --2- 172.21.15.5:0/356237072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a9c101070 0x7f6a9c1a06b0 secure :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6a98037040 tx=0x7f6a980359a0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 >> 172.21.15.5:0/356237072 conn(0x7f6a9c0771d0 msgr2=0x7f6a9c107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.022+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 shutdown_connections 2024-04-22T13:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.023+0000 7f6aa328c640 1 -- 172.21.15.5:0/356237072 wait complete. 2024-04-22T13:28:21.030 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db --set-id 3bd67737 --release --await' 2024-04-22T13:28:21.030 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db --set-id 3bd67737 --release --await 2024-04-22T13:28:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 -- 172.21.15.5:0/3349778858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c410c930 msgr2=0x7f90c410cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 --2- 172.21.15.5:0/3349778858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c410c930 0x7f90c410cdb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009930 tx=0x7f90b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 -- 172.21.15.5:0/3349778858 shutdown_connections 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 --2- 172.21.15.5:0/3349778858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c410c930 0x7f90c410cdb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 --2- 172.21.15.5:0/3349778858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c410d2f0 0x7f90c4119800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 --2- 172.21.15.5:0/3349778858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90c4112940 0x7f90c4112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 -- 172.21.15.5:0/3349778858 >> 172.21.15.5:0/3349778858 conn(0x7f90c406d100 msgr2=0x7f90c410af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 -- 172.21.15.5:0/3349778858 shutdown_connections 2024-04-22T13:28:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.065+0000 7f90c97f4640 1 -- 172.21.15.5:0/3349778858 wait complete. 2024-04-22T13:28:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.066+0000 7f90c97f4640 1 Processor -- start 2024-04-22T13:28:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.066+0000 7f90c97f4640 1 -- start start 2024-04-22T13:28:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c97f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90c410c930 0x7f90c41a3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c97f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c410d2f0 0x7f90c41a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c97f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 0x7f90c41a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c97f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90c411cc00 con 0x7f90c4112940 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c97f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90c411ca80 con 0x7f90c410d2f0 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c97f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90c411cd80 con 0x7f90c410c930 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 0x7f90c41a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 0x7f90c41a7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56048/0 (socket says 172.21.15.5:56048) 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 -- 172.21.15.5:0/2130224760 learned_addr learned my addr 172.21.15.5:0/2130224760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90c410c930 0x7f90c41a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c27fc640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c410d2f0 0x7f90c41a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 -- 172.21.15.5:0/2130224760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90c410c930 msgr2=0x7f90c41a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90c410c930 0x7f90c41a3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 -- 172.21.15.5:0/2130224760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c410d2f0 msgr2=0x7f90c41a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c410d2f0 0x7f90c41a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.067+0000 7f90c37fe640 1 -- 172.21.15.5:0/2130224760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c41a8170 con 0x7f90c4112940 2024-04-22T13:28:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.068+0000 7f90c37fe640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 0x7f90c41a7a40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00ea60 tx=0x7f90bc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.071+0000 7f90abfff640 1 -- 172.21.15.5:0/2130224760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00ce40 con 0x7f90c4112940 2024-04-22T13:28:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.071+0000 7f90abfff640 1 -- 172.21.15.5:0/2130224760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90bc005020 con 0x7f90c4112940 2024-04-22T13:28:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.071+0000 7f90abfff640 1 -- 172.21.15.5:0/2130224760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc010670 con 0x7f90c4112940 2024-04-22T13:28:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.071+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c41a8460 con 0x7f90c4112940 2024-04-22T13:28:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.072+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41afcf0 con 0x7f90c4112940 2024-04-22T13:28:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.075+0000 7f90cc3e4740 1 Processor -- start 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.075+0000 7f90abfff640 1 -- 172.21.15.5:0/2130224760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f90bc0054b0 con 0x7f90c4112940 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.075+0000 7f90cc3e4740 1 -- start start 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efb122d1f0 con 0x55efb12d0ce0 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56058/0 (socket says 172.21.15.5:56058) 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c27fc640 1 -- 172.21.15.5:0/1558782999 learned_addr learned my addr 172.21.15.5:0/1558782999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efb11ca4b0 con 0x55efb12da210 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1558782999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55efb12291b0 con 0x55efb12d1690 2024-04-22T13:28:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c2ffd640 1 -- 172.21.15.5:0/1558782999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 msgr2=0x55efb12dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c2ffd640 1 -- 172.21.15.5:0/1558782999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 msgr2=0x55efb12d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90abfff640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90ac0a40a0 0x7f90ac0a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c2ffd640 1 -- 172.21.15.5:0/1558782999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb11bc1b0 con 0x55efb12d1690 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.076+0000 7f90c27fc640 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.077+0000 7f90abfff640 1 -- 172.21.15.5:0/2130224760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f90bc0d0f20 con 0x7f90c4112940 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.077+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d9c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009c00 tx=0x7f90b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b510bef1e8ac878 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.077+0000 7f90a97fa640 1 -- 172.21.15.5:0/1558782999 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4004050 con 0x55efb12d1690 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.077+0000 7f90a97fa640 1 -- 172.21.15.5:0/1558782999 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b40054d0 con 0x55efb12d1690 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.077+0000 7f90a97fa640 1 -- 172.21.15.5:0/1558782999 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4004cd0 con 0x55efb12d1690 2024-04-22T13:28:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.077+0000 7f90c37fe640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90ac0a40a0 0x7f90ac0a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1558782999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 msgr2=0x55efb12d9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d9c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009c00 tx=0x7f90b4011810 comp rx=0 tx=0).stop 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90c37fe640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90ac0a40a0 0x7f90ac0a6560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0ce4c0 tx=0x7f90bc0096c0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1558782999 shutdown_connections 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d9c40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1558782999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1558782999 >> 172.21.15.5:0/1558782999 conn(0x55efb12cb3b0 msgr2=0x55efb12cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1558782999 shutdown_connections 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1558782999 wait complete. 2024-04-22T13:28:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 -- start start 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.078+0000 7f90cc3e4740 1 Processor -- start 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d0ce0 0x55efb12d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12d6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efb12dd180 con 0x55efb12d1690 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55efb11e6500 con 0x55efb12da210 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55efb12dd300 con 0x55efb12d0ce0 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90cc3e4740 10 client.0 _ll_register_callbacks cb 0x55efb11b4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12d6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12d6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56064/0 (socket says 172.21.15.5:56064) 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 -- 172.21.15.5:0/237893046 learned_addr learned my addr 172.21.15.5:0/237893046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c27fc640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d0ce0 0x55efb12d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c27fc640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 -- 172.21.15.5:0/237893046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d0ce0 msgr2=0x55efb12d9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d0ce0 0x55efb12d9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 -- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 msgr2=0x55efb12d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c2ffd640 1 -- 172.21.15.5:0/237893046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb0f3f3f0 con 0x55efb12d1690 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.079+0000 7f90c27fc640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12d6ec0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009bd0 tx=0x7f90b4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b629be3b046bf619 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90a6ff5640 1 -- 172.21.15.5:0/237893046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4029070 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b40054d0 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90a6ff5640 1 -- 172.21.15.5:0/237893046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f90b4003a00 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90a6ff5640 1 -- 172.21.15.5:0/237893046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90b4035050 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90cc3e4740 10 client.16218 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efb10097d0 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.080+0000 7f90a6ff5640 1 -- 172.21.15.5:0/237893046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b40054d0 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90a6ff5640 1 -- 172.21.15.5:0/237893046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90b40050c0 con 0x55efb12d1690 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90cc3e4740 10 client.16218 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90cc3e4740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90cc3e4740 20 client.16218 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb12ea740 0x55efb12ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90cc3e4740 4 client.16218 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efb11cc930 con 0x55efb12ea740 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90c27fc640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb12ea740 0x55efb12ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90c27fc640 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb12ea740 0x55efb12ecb80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ff5b30e77c8282 server_cookie=fcc72f2afeba374 in_seq=0 out_seq=0 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.081+0000 7f90a6ff5640 10 client.16218 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90a6ff5640 1 -- 172.21.15.5:0/237893046 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efb11cc930 con 0x55efb12ea740 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90a6ff5640 10 client.16218 handle_command_reply: tid=0 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 client.16218 shutdown 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 msgr2=0x55efb12d6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12d6ec0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009bd0 tx=0x7f90b4005da0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 shutdown_connections 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb12ea740 0x55efb12ecb80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12d6ec0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/237893046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d0ce0 0x55efb12d9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 >> 172.21.15.5:0/237893046 conn(0x55efb12cb3b0 msgr2=0x55efb12cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90a6ff5640 0 client.16218 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.082+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 shutdown_connections 2024-04-22T13:28:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.083+0000 7f90cc3e4740 1 -- 172.21.15.5:0/237893046 wait complete. 2024-04-22T13:28:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.083+0000 7f90cc3e4740 20 client.16218 trim_cache size 0 max 16384 2024-04-22T13:28:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.109+0000 7f90cc3e4740 1 Processor -- start 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.109+0000 7f90cc3e4740 1 -- start start 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.109+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.109+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.109+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.109+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b41c0 con 0x55efb12d0ce0 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56070/0 (socket says 172.21.15.5:56070) 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c27fc640 1 -- 172.21.15.5:0/2345725059 learned_addr learned my addr 172.21.15.5:0/2345725059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b5d00 con 0x55efb12da210 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b4890 con 0x55efb12d1690 2024-04-22T13:28:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c27fc640 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 -- 172.21.15.5:0/2345725059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 msgr2=0x55efb12d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 -- 172.21.15.5:0/2345725059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 msgr2=0x55efb12e6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12e6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 -- 172.21.15.5:0/2345725059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb12d5e70 con 0x55efb12d1690 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c27fc640 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.110+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d2470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55efb12d6720 tx=0x7f90b4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dfb2a21b12c016c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90a8ff9640 1 -- 172.21.15.5:0/2345725059 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4029030 con 0x55efb12d1690 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90a8ff9640 1 -- 172.21.15.5:0/2345725059 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b4004610 con 0x55efb12d1690 2024-04-22T13:28:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90a8ff9640 1 -- 172.21.15.5:0/2345725059 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4003dd0 con 0x55efb12d1690 2024-04-22T13:28:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 msgr2=0x55efb12d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d2470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55efb12d6720 tx=0x7f90b4005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 shutdown_connections 2024-04-22T13:28:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12da210 0x55efb12d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12d1690 0x55efb12d2470 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/2345725059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d0ce0 0x55efb12e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 >> 172.21.15.5:0/2345725059 conn(0x55efb12cb3b0 msgr2=0x55efb12cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 shutdown_connections 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.111+0000 7f90cc3e4740 1 -- 172.21.15.5:0/2345725059 wait complete. 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 -- start start 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 Processor -- start 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 0x55efb12dd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12da210 0x55efb134ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b5630 con 0x55efb12d1690 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b4f60 con 0x55efb12d0ce0 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.112+0000 7f90cc3e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b21a0 con 0x55efb12da210 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 0x55efb12dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 0x55efb12dd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:49974/0 (socket says 172.21.15.5:49974) 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 -- 172.21.15.5:0/1545849709 learned_addr learned my addr 172.21.15.5:0/1545849709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90cc3e4740 10 client.0 _ll_register_callbacks cb 0x7f90b4004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12da210 0x55efb134ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 -- 172.21.15.5:0/1545849709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12da210 msgr2=0x55efb134ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12da210 0x55efb134ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 -- 172.21.15.5:0/1545849709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 msgr2=0x55efb12ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 -- 172.21.15.5:0/1545849709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb12d1430 con 0x55efb12d0ce0 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90c27fc640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 0x55efb12dd700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90b8036040 tx=0x7f90b8005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f331cce2cc1693 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.113+0000 7f90a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:28:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90a6ff5640 1 -- 172.21.15.5:0/1545849709 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8005450 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90a6ff5640 1 -- 172.21.15.5:0/1545849709 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f90b80055f0 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b4004610 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90a6ff5640 1 -- 172.21.15.5:0/1545849709 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90b802cc70 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 10 client.25730 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efb1350390 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90a6ff5640 1 -- 172.21.15.5:0/1545849709 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b8004b70 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90a6ff5640 1 -- 172.21.15.5:0/1545849709 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90b8003a20 con 0x55efb12d0ce0 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 10 client.25730 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 10 client.25730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 20 client.25730 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb1350b10 0x55efb1352f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 4 client.25730 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.114+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f90b4004610 con 0x55efb1350b10 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb1350b10 0x55efb1352f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90a6ff5640 10 client.25730 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb1350b10 0x55efb1352f30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8795434c9297f03b server_cookie=ebd46171d949a2 in_seq=0 out_seq=0 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90a6ff5640 1 -- 172.21.15.5:0/1545849709 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f90b4004610 con 0x55efb1350b10 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90a6ff5640 10 client.25730 handle_command_reply: tid=0 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90cc3e4740 1 client.25730 shutdown 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 msgr2=0x55efb12dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.115+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 0x55efb12dd700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90b8036040 tx=0x7f90b8005a20 comp rx=0 tx=0).stop 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 shutdown_connections 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55efb1350b10 0x55efb1352f30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90a6ff5640 0 client.25730 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efb12d1690 0x55efb12ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efb12da210 0x55efb134ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 --2- 172.21.15.5:0/1545849709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55efb12d0ce0 0x55efb12dd700 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 >> 172.21.15.5:0/1545849709 conn(0x55efb12cb3b0 msgr2=0x55efb12c1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 shutdown_connections 2024-04-22T13:28:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 1 -- 172.21.15.5:0/1545849709 wait complete. 2024-04-22T13:28:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90cc3e4740 20 client.25730 trim_cache size 0 max 16384 2024-04-22T13:28:21.119 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90ac0a40a0 msgr2=0x7f90ac0a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90ac0a40a0 0x7f90ac0a6560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0ce4c0 tx=0x7f90bc0096c0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 msgr2=0x7f90c41a7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 0x7f90c41a7a40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00ea60 tx=0x7f90bc00ef20 comp rx=0 tx=0).stop 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 shutdown_connections 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4112940 0x7f90c41a7a40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90ac0a40a0 0x7f90ac0a6560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c410d2f0 0x7f90c41a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 --2- 172.21.15.5:0/2130224760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90c410c930 0x7f90c41a3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.116+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 >> 172.21.15.5:0/2130224760 conn(0x7f90c406d100 msgr2=0x7f90c410f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.117+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 shutdown_connections 2024-04-22T13:28:21.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.117+0000 7f90c97f4640 1 -- 172.21.15.5:0/2130224760 wait complete. 2024-04-22T13:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 -- 172.21.15.5:0/146398657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c106160 msgr2=0x7f907c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 --2- 172.21.15.5:0/146398657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c106160 0x7f907c10dd30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9074009ca0 tx=0x7f907402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 -- 172.21.15.5:0/146398657 shutdown_connections 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 --2- 172.21.15.5:0/146398657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c106160 0x7f907c10dd30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 --2- 172.21.15.5:0/146398657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c101a40 0x7f907c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 --2- 172.21.15.5:0/146398657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c101090 0x7f907c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 -- 172.21.15.5:0/146398657 >> 172.21.15.5:0/146398657 conn(0x7f907c078000 msgr2=0x7f907c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 -- 172.21.15.5:0/146398657 shutdown_connections 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.190+0000 7f90812d2640 1 -- 172.21.15.5:0/146398657 wait complete. 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 Processor -- start 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 -- start start 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 0x7f907c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c106160 0x7f907c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c19cfd0 0x7f907c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1110b0 con 0x7f907c101090 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f907c110f30 con 0x7f907c19cfd0 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f90812d2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f907c111230 con 0x7f907c106160 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f907affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 0x7f907c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f907affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 0x7f907c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56112/0 (socket says 172.21.15.5:56112) 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f907affd640 1 -- 172.21.15.5:0/3001447941 learned_addr learned my addr 172.21.15.5:0/3001447941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c106160 0x7f907c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.191+0000 7f907b7fe640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c19cfd0 0x7f907c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907affd640 1 -- 172.21.15.5:0/3001447941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c106160 msgr2=0x7f907c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907affd640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c106160 0x7f907c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907affd640 1 -- 172.21.15.5:0/3001447941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c19cfd0 msgr2=0x7f907c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907affd640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c19cfd0 0x7f907c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907affd640 1 -- 172.21.15.5:0/3001447941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c19dac0 con 0x7f907c101090 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c106160 0x7f907c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907b7fe640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c19cfd0 0x7f907c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f907affd640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 0x7f907c198760 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f906c00ea60 tx=0x7f906c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f9063fff640 1 -- 172.21.15.5:0/3001447941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00cde0 con 0x7f907c101090 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f9063fff640 1 -- 172.21.15.5:0/3001447941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c0035f0 con 0x7f907c101090 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c19ddb0 con 0x7f907c101090 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f9063fff640 1 -- 172.21.15.5:0/3001447941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c010690 con 0x7f907c101090 2024-04-22T13:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.192+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c1a55f0 con 0x7f907c101090 2024-04-22T13:28:21.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.195+0000 7f9063fff640 1 -- 172.21.15.5:0/3001447941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f906c004000 con 0x7f907c101090 2024-04-22T13:28:21.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.195+0000 7f9083fa4740 1 Processor -- start 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 -- start start 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9dda96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9c30 0x557e9ddb22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9063fff640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90640a3f50 0x7f90640a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 0x557e9ddb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557e9dd0e500 con 0x557e9ddb2810 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557e9dcb46a0 con 0x557e9dda9310 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9083fa4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557e9dd0a4c0 con 0x557e9dda9c30 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f907b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 0x557e9ddb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f907b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 0x557e9ddb4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56120/0 (socket says 172.21.15.5:56120) 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f907b7fe640 1 -- 172.21.15.5:0/633041573 learned_addr learned my addr 172.21.15.5:0/633041573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f9063fff640 1 -- 172.21.15.5:0/3001447941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f906c0d0e40 con 0x7f907c101090 2024-04-22T13:28:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f907affd640 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9c30 0x557e9ddb22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.196+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90640a3f50 0x7f90640a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907affd640 1 -- 172.21.15.5:0/633041573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 msgr2=0x557e9dda96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907affd640 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9dda96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907affd640 1 -- 172.21.15.5:0/633041573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 msgr2=0x557e9ddb4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907affd640 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 0x557e9ddb4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907affd640 1 -- 172.21.15.5:0/633041573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9dcd35e0 con 0x557e9dda9c30 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907b7fe640 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 0x557e9ddb4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907affd640 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9c30 0x557e9ddb22d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f906c0ce380 tx=0x7f906c09c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46a6bc57bebed6e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90640a3f50 0x7f90640a6410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f907c105cb0 tx=0x7f9070005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f90617fa640 1 -- 172.21.15.5:0/633041573 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c067970 con 0x557e9dda9c30 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f90617fa640 1 -- 172.21.15.5:0/633041573 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c067df0 con 0x557e9dda9c30 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.197+0000 7f90617fa640 1 -- 172.21.15.5:0/633041573 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c09aed0 con 0x557e9dda9c30 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 -- 172.21.15.5:0/633041573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9c30 msgr2=0x557e9ddb22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9c30 0x557e9ddb22d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f906c0ce380 tx=0x7f906c09c530 comp rx=0 tx=0).stop 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 -- 172.21.15.5:0/633041573 shutdown_connections 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9ddb2810 0x557e9ddb4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9c30 0x557e9ddb22d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 --2- 172.21.15.5:0/633041573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9dda96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 -- 172.21.15.5:0/633041573 >> 172.21.15.5:0/633041573 conn(0x557e9dda3a40 msgr2=0x557e9dda3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 -- 172.21.15.5:0/633041573 shutdown_connections 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.198+0000 7f9083fa4740 1 -- 172.21.15.5:0/633041573 wait complete. 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 -- start start 2024-04-22T13:28:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 Processor -- start 2024-04-22T13:28:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9ddaf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddafc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9ddb2810 0x557e9ddb01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557e9ddb5540 con 0x557e9dda9c30 2024-04-22T13:28:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557e9dcc7a60 con 0x557e9dda9310 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557e9ddb56c0 con 0x557e9ddb2810 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f9083fa4740 10 client.0 _ll_register_callbacks cb 0x557e9da20110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddafc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddafc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56134/0 (socket says 172.21.15.5:56134) 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.199+0000 7f907a7fc640 1 -- 172.21.15.5:0/372507727 learned_addr learned my addr 172.21.15.5:0/372507727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907affd640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9ddaf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907b7fe640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9ddb2810 0x557e9ddb01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907a7fc640 1 -- 172.21.15.5:0/372507727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9ddb2810 msgr2=0x557e9ddb01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907a7fc640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9ddb2810 0x557e9ddb01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907a7fc640 1 -- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 msgr2=0x557e9ddaf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907a7fc640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9ddaf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907a7fc640 1 -- 172.21.15.5:0/372507727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9da20920 con 0x557e9dda9c30 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907affd640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9ddaf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f907a7fc640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddafc80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9070011fe0 tx=0x7f9070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9654b8eed28de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f905eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f905eff5640 1 -- 172.21.15.5:0/372507727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070024020 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f905eff5640 1 -- 172.21.15.5:0/372507727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9070012860 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c067df0 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f905eff5640 1 -- 172.21.15.5:0/372507727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f907002f050 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f9083fa4740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.200+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e9daecd30 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f905eff5640 1 -- 172.21.15.5:0/372507727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c067df0 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f905eff5640 1 -- 172.21.15.5:0/372507727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9070020bd0 con 0x557e9dda9c30 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f9083fa4740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f9083fa4740 10 client.16236 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f9083fa4740 20 client.16236 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f9083fa4740 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9ddc4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f9083fa4740 4 client.16236 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9dce1a10 con 0x557e9ddc2a20 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.201+0000 7f907affd640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9ddc4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f907affd640 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9ddc4e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e6fec9dd2ad4d0 server_cookie=dcfd3a4ff60d3e4b in_seq=0 out_seq=0 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f905eff5640 10 client.16236 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f905eff5640 1 -- 172.21.15.5:0/372507727 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e9dce1a10 con 0x557e9ddc2a20 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f905eff5640 10 client.16236 handle_command_reply: tid=0 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f9083fa4740 1 client.16236 shutdown 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 msgr2=0x557e9ddafc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f9083fa4740 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddafc80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9070011fe0 tx=0x7f9070004730 comp rx=0 tx=0).stop 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 shutdown_connections 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.202+0000 7f9083fa4740 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9ddc4e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddafc80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9ddb2810 0x557e9ddb01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f905eff5640 0 client.16236 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 1 --2- 172.21.15.5:0/372507727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9dda9310 0x557e9ddaf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 >> 172.21.15.5:0/372507727 conn(0x557e9dda3a40 msgr2=0x557e9dda5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 shutdown_connections 2024-04-22T13:28:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 1 -- 172.21.15.5:0/372507727 wait complete. 2024-04-22T13:28:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.203+0000 7f9083fa4740 20 client.16236 trim_cache size 0 max 16384 2024-04-22T13:28:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 Processor -- start 2024-04-22T13:28:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 -- start start 2024-04-22T13:28:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9ddbe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddcac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1aba10 con 0x557e9dda9c30 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ad550 con 0x557e9ddb2810 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f9083fa4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ac0e0 con 0x557e9dda9310 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56144/0 (socket says 172.21.15.5:56144) 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.220+0000 7f907a7fc640 1 -- 172.21.15.5:0/3320745342 learned_addr learned my addr 172.21.15.5:0/3320745342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907affd640 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9ddbe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907b7fe640 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddcac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907a7fc640 1 -- 172.21.15.5:0/3320745342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 msgr2=0x557e9ddbe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9ddbe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907a7fc640 1 -- 172.21.15.5:0/3320745342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 msgr2=0x557e9ddcac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddcac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907a7fc640 1 -- 172.21.15.5:0/3320745342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9dda8ce0 con 0x557e9dda9c30 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907b7fe640 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddcac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddca700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f907001f3e0 tx=0x7f9070003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef053695c2740f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f9060ff9640 1 -- 172.21.15.5:0/3320745342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070024040 con 0x557e9dda9c30 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f9060ff9640 1 -- 172.21.15.5:0/3320745342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9070016070 con 0x557e9dda9c30 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.221+0000 7f9060ff9640 1 -- 172.21.15.5:0/3320745342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907001fc30 con 0x557e9dda9c30 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 -- 172.21.15.5:0/3320745342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 msgr2=0x557e9ddca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddca700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f907001f3e0 tx=0x7f9070003e50 comp rx=0 tx=0).stop 2024-04-22T13:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 -- 172.21.15.5:0/3320745342 shutdown_connections 2024-04-22T13:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9ddca700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddcac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3320745342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9ddbe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 -- 172.21.15.5:0/3320745342 >> 172.21.15.5:0/3320745342 conn(0x557e9dda3a40 msgr2=0x557e9dda2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 -- 172.21.15.5:0/3320745342 shutdown_connections 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 -- 172.21.15.5:0/3320745342 wait complete. 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 -- start start 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.222+0000 7f9083fa4740 1 Processor -- start 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9de212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9de217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9de217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddb6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9de217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56150/0 (socket says 172.21.15.5:56150) 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9de212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907a7fc640 1 -- 172.21.15.5:0/3202094176 learned_addr learned my addr 172.21.15.5:0/3202094176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ace80 con 0x557e9dda9c30 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ac7b0 con 0x557e9ddb2810 2024-04-22T13:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1a99f0 con 0x557e9dda9310 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f9083fa4740 10 client.0 _ll_register_callbacks cb 0x7f9070011bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907b7fe640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddb6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907affd640 1 -- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 msgr2=0x557e9ddb6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907affd640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddb6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907affd640 1 -- 172.21.15.5:0/3202094176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 msgr2=0x557e9de217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907affd640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9de217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907affd640 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9ddb7400 con 0x557e9dda9310 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.223+0000 7f907b7fe640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddb6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f907affd640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9de212b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f906c09ae80 tx=0x7f906c09c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2989360538811d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f905eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f905eff5640 1 -- 172.21.15.5:0/3202094176 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c06c030 con 0x557e9dda9310 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f905eff5640 1 -- 172.21.15.5:0/3202094176 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f906c09aed0 con 0x557e9dda9310 2024-04-22T13:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9070016070 con 0x557e9dda9310 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f905eff5640 1 -- 172.21.15.5:0/3202094176 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f906c0a5840 con 0x557e9dda9310 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 10 client.25828 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e9de29b80 con 0x557e9dda9310 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f905eff5640 1 -- 172.21.15.5:0/3202094176 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9070016070 con 0x557e9dda9310 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f905eff5640 1 -- 172.21.15.5:0/3202094176 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f906c0a53b0 con 0x557e9dda9310 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 10 client.25828 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 10 client.25828 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 20 client.25828 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9de2bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 4 client.25828 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3bd67737", "release": true, "await": true}] 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.224+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3bd67737", "release": true, "await": true}) -- 0x557e9dd1bf20 con 0x557e9ddc2a20 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.225+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9de2bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.225+0000 7f905eff5640 10 client.25828 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.225+0000 7f907a7fc640 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9de2bf70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6301d3748dbd6afa server_cookie=af95963ccd2165ee in_seq=0 out_seq=0 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f905eff5640 1 -- 172.21.15.5:0/3202094176 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x557e9dd1bf20 con 0x557e9ddc2a20 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f905eff5640 10 client.25828 handle_command_reply: tid=0 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 client.25828 shutdown 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 msgr2=0x557e9de212b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9de212b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f906c09ae80 tx=0x7f906c09c660 comp rx=0 tx=0).stop 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 shutdown_connections 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557e9ddc2a20 0x557e9de2bf70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e9dda9c30 0x557e9de217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e9ddb2810 0x557e9ddb6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f905eff5640 0 client.25828 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 --2- 172.21.15.5:0/3202094176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e9dda9310 0x557e9de212b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 >> 172.21.15.5:0/3202094176 conn(0x557e9dda3a40 msgr2=0x557e9ddb0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 shutdown_connections 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 1 -- 172.21.15.5:0/3202094176 wait complete. 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.228+0000 7f9083fa4740 20 client.25828 trim_cache size 0 max 16384 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 8, 2024-04-22T13:28:21.228 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "3bd67737": { 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 8, 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 70.0, 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:28:21.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:21.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:21.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:21.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:21.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:21.230 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90640a3f50 msgr2=0x7f90640a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90640a3f50 0x7f90640a6410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f907c105cb0 tx=0x7f9070005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 msgr2=0x7f907c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 0x7f907c198760 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f906c00ea60 tx=0x7f906c00ef20 comp rx=0 tx=0).stop 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 shutdown_connections 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f90640a3f50 0x7f90640a6410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f907c19cfd0 0x7f907c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f907c106160 0x7f907c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 --2- 172.21.15.5:0/3001447941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f907c101090 0x7f907c198760 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 >> 172.21.15.5:0/3001447941 conn(0x7f907c078000 msgr2=0x7f907c069650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 shutdown_connections 2024-04-22T13:28:21.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:21.229+0000 7f90812d2640 1 -- 172.21.15.5:0/3001447941 wait complete. 2024-04-22T13:28:21.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:21 smithi005 ceph-mon[28006]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:28:21.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:21 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3388539513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:21.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:21 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/356237072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:21.239 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: 3bd67737, seconds elapsed: 11.3 2024-04-22T13:28:21.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:21 smithi049 ceph-mon[31379]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:28:21.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:21 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3388539513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:21.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:21 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/356237072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:21 smithi107 ceph-mon[31859]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:21 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3388539513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:21 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/356237072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:22.128 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:28:22.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:28:22.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:28:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 -- 172.21.15.5:0/1608285666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 msgr2=0x7ff51410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 --2- 172.21.15.5:0/1608285666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff51410dd30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 -- 172.21.15.5:0/1608285666 shutdown_connections 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 --2- 172.21.15.5:0/1608285666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff51410dd30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 --2- 172.21.15.5:0/1608285666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff514101a40 0x7ff514105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 --2- 172.21.15.5:0/1608285666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff514101090 0x7ff514101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 -- 172.21.15.5:0/1608285666 >> 172.21.15.5:0/1608285666 conn(0x7ff514078000 msgr2=0x7ff514078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 -- 172.21.15.5:0/1608285666 shutdown_connections 2024-04-22T13:28:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.273+0000 7ff5188e4640 1 -- 172.21.15.5:0/1608285666 wait complete. 2024-04-22T13:28:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.274+0000 7ff5188e4640 1 Processor -- start 2024-04-22T13:28:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.274+0000 7ff5188e4640 1 -- start start 2024-04-22T13:28:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff5188e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff514101090 0x7ff514198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff5188e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff514198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff5188e4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff51419d000 0x7ff51419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff5188e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141110d0 con 0x7ff514106160 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff5188e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff514110f50 con 0x7ff514101090 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff5188e4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111250 con 0x7ff51419d000 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff514198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff514198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56170/0 (socket says 172.21.15.5:56170) 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 -- 172.21.15.5:0/3834695773 learned_addr learned my addr 172.21.15.5:0/3834695773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff512d9d640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff51419d000 0x7ff51419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff51259c640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff514101090 0x7ff514198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 -- 172.21.15.5:0/3834695773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff51419d000 msgr2=0x7ff51419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff51419d000 0x7ff51419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 -- 172.21.15.5:0/3834695773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff514101090 msgr2=0x7ff514198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff514101090 0x7ff514198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 -- 172.21.15.5:0/3834695773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419daf0 con 0x7ff514106160 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.275+0000 7ff511d9b640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff514198c80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9c0 tx=0x7ff50800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.276+0000 7ff5037fe640 1 -- 172.21.15.5:0/3834695773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800cd80 con 0x7ff514106160 2024-04-22T13:28:22.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.276+0000 7ff5037fe640 1 -- 172.21.15.5:0/3834695773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5080054d0 con 0x7ff514106160 2024-04-22T13:28:22.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.276+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419dde0 con 0x7ff514106160 2024-04-22T13:28:22.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.276+0000 7ff5037fe640 1 -- 172.21.15.5:0/3834695773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508010430 con 0x7ff514106160 2024-04-22T13:28:22.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.276+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff514102d70 con 0x7ff514106160 2024-04-22T13:28:22.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.278+0000 7ff5037fe640 1 -- 172.21.15.5:0/3834695773 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7ff508004fb0 con 0x7ff514106160 2024-04-22T13:28:22.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.279+0000 7ff5037fe640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4fc0a3ed0 0x7ff4fc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.279+0000 7ff5037fe640 1 -- 172.21.15.5:0/3834695773 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff5080d0ef0 con 0x7ff514106160 2024-04-22T13:28:22.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.279+0000 7ff51259c640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4fc0a3ed0 0x7ff4fc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.279+0000 7ff51b5b3740 1 Processor -- start 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.279+0000 7ff51259c640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4fc0a3ed0 0x7ff4fc0a6390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff504009b60 tx=0x7ff504005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 -- start start 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf3749cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf3752840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf3755260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bf36adf00 con 0x564bf3752e10 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564bf364b590 con 0x564bf37498e0 2024-04-22T13:28:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564bf36a9ec0 con 0x564bf374a290 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff51259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf3749cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff511d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf3752840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf3749cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50060/0 (socket says 172.21.15.5:50060) 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 -- 172.21.15.5:0/1258691263 learned_addr learned my addr 172.21.15.5:0/1258691263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.280+0000 7ff512d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf3755260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 -- 172.21.15.5:0/1258691263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 msgr2=0x564bf3752840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf3752840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 -- 172.21.15.5:0/1258691263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 msgr2=0x564bf3755260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf3755260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 -- 172.21.15.5:0/1258691263 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf34887e0 con 0x564bf37498e0 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff512d9d640 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf3755260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff511d9b640 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf3752840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.281+0000 7ff51259c640 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf3749cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff504011fa0 tx=0x7ff504014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e0920db3a9222e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.282+0000 7ff500ff9640 1 -- 172.21.15.5:0/1258691263 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50401f620 con 0x564bf37498e0 2024-04-22T13:28:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.282+0000 7ff500ff9640 1 -- 172.21.15.5:0/1258691263 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50401fc70 con 0x564bf37498e0 2024-04-22T13:28:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.282+0000 7ff500ff9640 1 -- 172.21.15.5:0/1258691263 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504027bc0 con 0x564bf37498e0 2024-04-22T13:28:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 -- 172.21.15.5:0/1258691263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 msgr2=0x564bf3749cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf3749cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff504011fa0 tx=0x7ff504014000 comp rx=0 tx=0).stop 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 -- 172.21.15.5:0/1258691263 shutdown_connections 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf3755260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf3752840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/1258691263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf3749cc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 -- 172.21.15.5:0/1258691263 >> 172.21.15.5:0/1258691263 conn(0x564bf3743fb0 msgr2=0x564bf3744390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 -- 172.21.15.5:0/1258691263 shutdown_connections 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 -- 172.21.15.5:0/1258691263 wait complete. 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 -- start start 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.283+0000 7ff51b5b3740 1 Processor -- start 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf37498e0 0x564bf374fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf374a290 0x564bf37503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf37508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bf3755d80 con 0x564bf3752e10 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564bf3667580 con 0x564bf374a290 2024-04-22T13:28:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564bf3755f00 con 0x564bf37498e0 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf37508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51b5b3740 10 client.0 _ll_register_callbacks cb 0x564bf366d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf37508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56190/0 (socket says 172.21.15.5:56190) 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 -- 172.21.15.5:0/24759974 learned_addr learned my addr 172.21.15.5:0/24759974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff511d9b640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf374a290 0x564bf37503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff51259c640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf37498e0 0x564bf374fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 -- 172.21.15.5:0/24759974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf37498e0 msgr2=0x564bf374fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf37498e0 0x564bf374fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 -- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf374a290 msgr2=0x564bf37503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf374a290 0x564bf37503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.284+0000 7ff512d9d640 1 -- 172.21.15.5:0/24759974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf34e8f70 con 0x564bf3752e10 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff512d9d640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf37508f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40192d4f6977ce2e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff4f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/24759974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c044070 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/24759974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff50c032dc0 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50401fc70 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/24759974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff50c050050 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff51b5b3740 10 client.16254 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bf3380170 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.285+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/24759974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50c003e20 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/24759974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff50c0043b0 con 0x564bf3752e10 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51b5b3740 10 client.16254 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51b5b3740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51b5b3740 20 client.16254 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf3763370 0x564bf3765760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51b5b3740 4 client.16254 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff50401fc70 con 0x564bf3763370 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51259c640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf3763370 0x564bf3765760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff4f9ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.286+0000 7ff51259c640 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf3763370 0x564bf3765760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0e05e0a7134b54 server_cookie=e3e32ebb2098eef2 in_seq=0 out_seq=0 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/24759974 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff50401fc70 con 0x564bf3763370 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff4f9ffb640 10 client.16254 handle_command_reply: tid=0 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 client.16254 shutdown 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 msgr2=0x564bf37508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf37508f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c005f60 comp rx=0 tx=0).stop 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 shutdown_connections 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf3763370 0x564bf3765760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf3752e10 0x564bf37508f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf374a290 0x564bf37503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/24759974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf37498e0 0x564bf374fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 >> 172.21.15.5:0/24759974 conn(0x564bf3743fb0 msgr2=0x564bf3745d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff4f9ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 shutdown_connections 2024-04-22T13:28:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.287+0000 7ff51b5b3740 1 -- 172.21.15.5:0/24759974 wait complete. 2024-04-22T13:28:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.288+0000 7ff51b5b3740 20 client.16254 trim_cache size 0 max 16384 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.305+0000 7ff51b5b3740 1 Processor -- start 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.305+0000 7ff51b5b3740 1 -- start start 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.305+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 0x564bf37525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.305+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf375ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf3752e10 0x564bf376b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141afd80 con 0x564bf37498e0 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141b18c0 con 0x564bf3752e10 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141b0450 con 0x564bf374a290 2024-04-22T13:28:22.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 0x564bf37525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 0x564bf37525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56202/0 (socket says 172.21.15.5:56202) 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 -- 172.21.15.5:0/807894794 learned_addr learned my addr 172.21.15.5:0/807894794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff511d9b640 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf375ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff512d9d640 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf3752e10 0x564bf376b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 -- 172.21.15.5:0/807894794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 msgr2=0x564bf375ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf375ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 -- 172.21.15.5:0/807894794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf3752e10 msgr2=0x564bf376b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf3752e10 0x564bf376b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 -- 172.21.15.5:0/807894794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf374ba30 con 0x564bf37498e0 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff512d9d640 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf3752e10 0x564bf376b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff51259c640 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 0x564bf37525a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff504011f70 tx=0x7ff504011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30497caa9009fd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff4fbfff640 1 -- 172.21.15.5:0/807894794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504027a90 con 0x564bf37498e0 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff4fbfff640 1 -- 172.21.15.5:0/807894794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5040054f0 con 0x564bf37498e0 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.306+0000 7ff4fbfff640 1 -- 172.21.15.5:0/807894794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040048e0 con 0x564bf37498e0 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 -- 172.21.15.5:0/807894794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 msgr2=0x564bf37525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 0x564bf37525a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff504011f70 tx=0x7ff504011be0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 -- 172.21.15.5:0/807894794 shutdown_connections 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf3752e10 0x564bf376b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf374a290 0x564bf375ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/807894794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf37498e0 0x564bf37525a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.307+0000 7ff51b5b3740 1 -- 172.21.15.5:0/807894794 >> 172.21.15.5:0/807894794 conn(0x564bf3743fb0 msgr2=0x564bf374e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 -- 172.21.15.5:0/807894794 shutdown_connections 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 -- 172.21.15.5:0/807894794 wait complete. 2024-04-22T13:28:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 -- start start 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 Processor -- start 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf375b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf374a290 0x564bf374c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf3752e10 0x564bf374d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141b11f0 con 0x564bf374a290 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141b0b20 con 0x564bf37498e0 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141add60 con 0x564bf3752e10 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.308+0000 7ff51b5b3740 10 client.0 _ll_register_callbacks cb 0x7ff508005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff511d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf374a290 0x564bf374c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf375b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf375b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50082/0 (socket says 172.21.15.5:50082) 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 -- 172.21.15.5:0/3310529630 learned_addr learned my addr 172.21.15.5:0/3310529630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff512d9d640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf3752e10 0x564bf374d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 -- 172.21.15.5:0/3310529630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf3752e10 msgr2=0x564bf374d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf3752e10 0x564bf374d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 -- 172.21.15.5:0/3310529630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf374a290 msgr2=0x564bf374c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf374a290 0x564bf374c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 -- 172.21.15.5:0/3310529630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf37c8880 con 0x564bf37498e0 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff512d9d640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf3752e10 0x564bf374d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff511d9b640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf374a290 0x564bf374c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51259c640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf375b650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff504038040 tx=0x7ff504005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462b1a39d6e24fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff4f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:28:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/3310529630 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504012cb0 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/3310529630 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff50401f420 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.309+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5040054f0 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/3310529630 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff50401f660 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 10 client.25757 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bf37c9960 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/3310529630 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5040054f0 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/3310529630 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff504035560 con 0x564bf37498e0 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 20 client.25757 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf37c9e70 0x564bf37cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 4 client.25757 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:28:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bf36bb920 con 0x564bf37c9e70 2024-04-22T13:28:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.310+0000 7ff511d9b640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf37c9e70 0x564bf37cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.311+0000 7ff4f9ffb640 10 client.25757 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.311+0000 7ff511d9b640 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf37c9e70 0x564bf37cc260 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd916bab765b71b server_cookie=ae0b0dc0f73e7c81 in_seq=0 out_seq=0 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff4f9ffb640 1 -- 172.21.15.5:0/3310529630 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bf36bb920 con 0x564bf37c9e70 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff4f9ffb640 10 client.25757 handle_command_reply: tid=0 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 client.25757 shutdown 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 msgr2=0x564bf375b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf375b650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff504038040 tx=0x7ff504005730 comp rx=0 tx=0).stop 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 shutdown_connections 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564bf37c9e70 0x564bf37cc260 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff4f9ffb640 0 client.25757 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bf374a290 0x564bf374c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bf3752e10 0x564bf374d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 --2- 172.21.15.5:0/3310529630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564bf37498e0 0x564bf375b650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 >> 172.21.15.5:0/3310529630 conn(0x564bf3743fb0 msgr2=0x564bf3742780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.312+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 shutdown_connections 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff51b5b3740 1 -- 172.21.15.5:0/3310529630 wait complete. 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff51b5b3740 20 client.25757 trim_cache size 0 max 16384 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:22.316 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:28:22.317 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "88cf0d4f-bc08-4b40-ae52-b0ac6297d14b", 2024-04-22T13:28:22.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:28:22.317 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:22.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4fc0a3ed0 msgr2=0x7ff4fc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4fc0a3ed0 0x7ff4fc0a6390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff504009b60 tx=0x7ff504005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 msgr2=0x7ff514198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff514198c80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9c0 tx=0x7ff50800ee80 comp rx=0 tx=0).stop 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 shutdown_connections 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff514106160 0x7ff514198c80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4fc0a3ed0 0x7ff4fc0a6390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff51419d000 0x7ff51419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 --2- 172.21.15.5:0/3834695773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff514101090 0x7ff514198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 >> 172.21.15.5:0/3834695773 conn(0x7ff514078000 msgr2=0x7ff5140695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 shutdown_connections 2024-04-22T13:28:22.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.313+0000 7ff5188e4640 1 -- 172.21.15.5:0/3834695773 wait complete. 2024-04-22T13:28:22.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:22.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 -- 172.21.15.5:0/65613975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 msgr2=0x7f16cc10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 --2- 172.21.15.5:0/65613975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc10e3b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 -- 172.21.15.5:0/65613975 shutdown_connections 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 --2- 172.21.15.5:0/65613975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc10e3b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 --2- 172.21.15.5:0/65613975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16cc10e8f0 0x7f16cc110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 --2- 172.21.15.5:0/65613975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc0692f0 0x7f16cc0696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 -- 172.21.15.5:0/65613975 >> 172.21.15.5:0/65613975 conn(0x7f16cc077fa0 msgr2=0x7f16cc0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 -- 172.21.15.5:0/65613975 shutdown_connections 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.474+0000 7f16d26ce640 1 -- 172.21.15.5:0/65613975 wait complete. 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 Processor -- start 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 -- start start 2024-04-22T13:28:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc0692f0 0x7f16cc1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16cc10e8f0 0x7f16cc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc114350 con 0x7f16cc069c10 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1141d0 con 0x7f16cc0692f0 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16d26ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1144d0 con 0x7f16cc10e8f0 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.475+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc1a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56222/0 (socket says 172.21.15.5:56222) 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 -- 172.21.15.5:0/2071820363 learned_addr learned my addr 172.21.15.5:0/2071820363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cbfff640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc0692f0 0x7f16cc1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16cc10e8f0 0x7f16cc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 -- 172.21.15.5:0/2071820363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16cc10e8f0 msgr2=0x7f16cc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16cc10e8f0 0x7f16cc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 -- 172.21.15.5:0/2071820363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc0692f0 msgr2=0x7f16cc1a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc0692f0 0x7f16cc1a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 -- 172.21.15.5:0/2071820363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc19af30 con 0x7f16cc069c10 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc1a0af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005940 tx=0x7f16c002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16c97fa640 1 -- 172.21.15.5:0/2071820363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0043020 con 0x7f16cc069c10 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc19b1c0 con 0x7f16cc069c10 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16c97fa640 1 -- 172.21.15.5:0/2071820363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c002dc90 con 0x7f16cc069c10 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16c97fa640 1 -- 172.21.15.5:0/2071820363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0003ab0 con 0x7f16cc069c10 2024-04-22T13:28:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.476+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a7530 con 0x7f16cc069c10 2024-04-22T13:28:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.479+0000 7f16c97fa640 1 -- 172.21.15.5:0/2071820363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f16c0046020 con 0x7f16cc069c10 2024-04-22T13:28:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16c97fa640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16b40a3ed0 0x7f16b40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16cbfff640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16b40a3ed0 0x7f16b40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16c97fa640 1 -- 172.21.15.5:0/2071820363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f16c00f2a30 con 0x7f16cc069c10 2024-04-22T13:28:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 Processor -- start 2024-04-22T13:28:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 -- start start 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc6b580 0x55bdabc6b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc74540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 0x55bdabc76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdabbd01e0 con 0x55bdabc74a80 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bdabb6d390 con 0x55bdabc6bea0 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d5395740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bdabbcc1a0 con 0x55bdabc6b580 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16cbfff640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16b40a3ed0 0x7f16b40a6390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005d00 tx=0x7f16bc00b410 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d0c6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 0x55bdabc76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.480+0000 7f16d0c6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 0x55bdabc76e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56232/0 (socket says 172.21.15.5:56232) 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 -- 172.21.15.5:0/3244236909 learned_addr learned my addr 172.21.15.5:0/3244236909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16cbfff640 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc6b580 0x55bdabc6b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc74540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 -- 172.21.15.5:0/3244236909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc6b580 msgr2=0x55bdabc6b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc6b580 0x55bdabc6b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 -- 172.21.15.5:0/3244236909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 msgr2=0x55bdabc74540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc74540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 -- 172.21.15.5:0/3244236909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdab9ac7d0 con 0x55bdabc74a80 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 0x55bdabc76e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009f70 tx=0x7f16c4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bb31ca9eb35f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16b27fc640 1 -- 172.21.15.5:0/3244236909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4016070 con 0x55bdabc74a80 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16b27fc640 1 -- 172.21.15.5:0/3244236909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4004d20 con 0x55bdabc74a80 2024-04-22T13:28:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.481+0000 7f16b27fc640 1 -- 172.21.15.5:0/3244236909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4012790 con 0x55bdabc74a80 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 -- 172.21.15.5:0/3244236909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 msgr2=0x55bdabc76e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 0x55bdabc76e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009f70 tx=0x7f16c4010800 comp rx=0 tx=0).stop 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 -- 172.21.15.5:0/3244236909 shutdown_connections 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc74a80 0x55bdabc76e70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc74540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 --2- 172.21.15.5:0/3244236909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc6b580 0x55bdabc6b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 -- 172.21.15.5:0/3244236909 >> 172.21.15.5:0/3244236909 conn(0x55bdabc65cb0 msgr2=0x55bdabc66090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 -- 172.21.15.5:0/3244236909 shutdown_connections 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.482+0000 7f16d5395740 1 -- 172.21.15.5:0/3244236909 wait complete. 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.483+0000 7f16d5395740 1 -- start start 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.483+0000 7f16d5395740 1 Processor -- start 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.483+0000 7f16d5395740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.483+0000 7f16d5395740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.483+0000 7f16d5395740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc72520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d5395740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdabc77870 con 0x55bdabc6b580 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc71aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56236/0 (socket says 172.21.15.5:56236) 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 -- 172.21.15.5:0/1315820005 learned_addr learned my addr 172.21.15.5:0/1315820005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bdabb892f0 con 0x55bdabc6bea0 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bdabc779f0 con 0x55bdabc74a80 2024-04-22T13:28:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc72520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d5395740 10 client.0 _ll_register_callbacks cb 0x55bdabb96580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 -- 172.21.15.5:0/1315820005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 msgr2=0x55bdabc72520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc72520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 -- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 msgr2=0x55bdabc71fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc71fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 -- 172.21.15.5:0/1315820005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdab8e23f0 con 0x55bdabc6b580 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc72520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc71fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4004d20 con 0x55bdabc6b580 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16cbfff640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc71aa0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f16bc011860 tx=0x7f16bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707564c2148021ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16ab7fe640 10 client.16272 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.484+0000 7f16ab7fe640 1 -- 172.21.15.5:0/1315820005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc01e940 con 0x55bdabc6b580 2024-04-22T13:28:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16ab7fe640 1 -- 172.21.15.5:0/1315820005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f16bc01eae0 con 0x55bdabc6b580 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16ab7fe640 1 -- 172.21.15.5:0/1315820005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16bc01ece0 con 0x55bdabc6b580 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55bdab8a20f0 con 0x55bdabc6b580 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16ab7fe640 1 -- 172.21.15.5:0/1315820005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16bc032470 con 0x55bdabc6b580 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 20 client.16272 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabc84940 0x55bdabc86dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.485+0000 7f16d5395740 4 client.16272 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.486+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdabbddc00 con 0x55bdabc84940 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.486+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabc84940 0x55bdabc86dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.486+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabc84940 0x55bdabc86dd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2fb0e9835473c9 server_cookie=2ff41123c15314e5 in_seq=0 out_seq=0 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.486+0000 7f16ab7fe640 10 client.16272 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16ab7fe640 1 -- 172.21.15.5:0/1315820005 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdabbddc00 con 0x55bdabc84940 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16ab7fe640 10 client.16272 handle_command_reply: tid=0 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 client.16272 shutdown 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 msgr2=0x55bdabc71aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc71aa0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f16bc011860 tx=0x7f16bc004730 comp rx=0 tx=0).stop 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 shutdown_connections 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabc84940 0x55bdabc86dd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc72520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc71fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 --2- 172.21.15.5:0/1315820005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc71aa0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 >> 172.21.15.5:0/1315820005 conn(0x55bdabc65cb0 msgr2=0x55bdabc67a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16ab7fe640 0 client.16272 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 shutdown_connections 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.487+0000 7f16d5395740 1 -- 172.21.15.5:0/1315820005 wait complete. 2024-04-22T13:28:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.488+0000 7f16d5395740 20 client.16272 trim_cache size 0 max 16384 2024-04-22T13:28:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 Processor -- start 2024-04-22T13:28:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 -- start start 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1afc60 con 0x55bdabc6b580 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1b17a0 con 0x55bdabc6bea0 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.512+0000 7f16d5395740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1b0330 con 0x55bdabc74a80 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56244/0 (socket says 172.21.15.5:56244) 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 -- 172.21.15.5:0/681792973 learned_addr learned my addr 172.21.15.5:0/681792973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 -- 172.21.15.5:0/681792973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 msgr2=0x55bdabc70c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc70c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 -- 172.21.15.5:0/681792973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 msgr2=0x55bdabc80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 -- 172.21.15.5:0/681792973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdabc6bcd0 con 0x55bdabc6b580 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc70c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16cbfff640 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc741d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f16bc032a20 tx=0x7f16bc021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68237df643b8fe2c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16b1ffb640 1 -- 172.21.15.5:0/681792973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc01ee80 con 0x55bdabc6b580 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16b1ffb640 1 -- 172.21.15.5:0/681792973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4004d20 con 0x55bdabc6b580 2024-04-22T13:28:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.513+0000 7f16b1ffb640 1 -- 172.21.15.5:0/681792973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc028b40 con 0x55bdabc6b580 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 -- 172.21.15.5:0/681792973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 msgr2=0x55bdabc741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc741d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f16bc032a20 tx=0x7f16bc021070 comp rx=0 tx=0).stop 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 -- 172.21.15.5:0/681792973 shutdown_connections 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc70c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6bea0 0x55bdabc80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 --2- 172.21.15.5:0/681792973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6b580 0x55bdabc741d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 -- 172.21.15.5:0/681792973 >> 172.21.15.5:0/681792973 conn(0x55bdabc65cb0 msgr2=0x55bdabc644e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 -- 172.21.15.5:0/681792973 shutdown_connections 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.514+0000 7f16d5395740 1 -- 172.21.15.5:0/681792973 wait complete. 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 -- start start 2024-04-22T13:28:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 Processor -- start 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6b580 0x55bdabc7ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6bea0 0x55bdabc6e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc6e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1b10d0 con 0x55bdabc6bea0 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1b0a00 con 0x55bdabc6b580 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1adc40 con 0x55bdabc74a80 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d5395740 10 client.0 _ll_register_callbacks cb 0x7f16bc00bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6bea0 0x55bdabc6e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d0c6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc6e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d0c6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc6e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:54080/0 (socket says 172.21.15.5:54080) 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.515+0000 7f16d0c6b640 1 -- 172.21.15.5:0/453345407 learned_addr learned my addr 172.21.15.5:0/453345407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16cbfff640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6b580 0x55bdabc7ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d0c6b640 1 -- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6b580 msgr2=0x55bdabc7ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6b580 0x55bdabc7ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d0c6b640 1 -- 172.21.15.5:0/453345407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6bea0 msgr2=0x55bdabc6e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6bea0 0x55bdabc6e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d0c6b640 1 -- 172.21.15.5:0/453345407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdabc6edf0 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16cb7fe640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6bea0 0x55bdabc6e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d0c6b640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc6e830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009f70 tx=0x7f16c4004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab0b02f613e08eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16ab7fe640 1 -- 172.21.15.5:0/453345407 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4016040 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4004d20 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.516+0000 7f16ab7fe640 1 -- 172.21.15.5:0/453345407 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f16c40128d0 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16ab7fe640 1 -- 172.21.15.5:0/453345407 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16c4032050 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 10 client.25864 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bdabceacd0 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16ab7fe640 1 -- 172.21.15.5:0/453345407 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c4004d20 con 0x55bdabc74a80 2024-04-22T13:28:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16ab7fe640 1 -- 172.21.15.5:0/453345407 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16c400f040 con 0x55bdabc74a80 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 10 client.25864 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 20 client.25864 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabceb140 0x55bdabced560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 4 client.25864 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdabbd7a70 con 0x55bdabceb140 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.517+0000 7f16cbfff640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabceb140 0x55bdabced560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16ab7fe640 10 client.25864 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16cbfff640 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabceb140 0x55bdabced560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a1694ea58deec1 server_cookie=a71640e5afa6eb6b in_seq=0 out_seq=0 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16ab7fe640 1 -- 172.21.15.5:0/453345407 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bdabbd7a70 con 0x55bdabceb140 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16ab7fe640 10 client.25864 handle_command_reply: tid=0 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 client.25864 shutdown 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 msgr2=0x55bdabc6e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc6e830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009f70 tx=0x7f16c4004a00 comp rx=0 tx=0).stop 2024-04-22T13:28:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 shutdown_connections 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bdabceb140 0x55bdabced560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdabc6bea0 0x55bdabc6e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16ab7fe640 0 client.25864 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.518+0000 7f16d5395740 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdabc74a80 0x55bdabc6e830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d5395740 1 --2- 172.21.15.5:0/453345407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdabc6b580 0x55bdabc7ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 >> 172.21.15.5:0/453345407 conn(0x55bdabc65cb0 msgr2=0x55bdabc71a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 shutdown_connections 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d5395740 1 -- 172.21.15.5:0/453345407 wait complete. 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d5395740 20 client.25864 trim_cache size 0 max 16384 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:22.520 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: "88cf0d4f-bc08-4b40-ae52-b0ac6297d14b": { 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "88cf0d4f-bc08-4b40-ae52-b0ac6297d14b", 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:22.521 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16b40a3ed0 msgr2=0x7f16b40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16b40a3ed0 0x7f16b40a6390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005d00 tx=0x7f16bc00b410 comp rx=0 tx=0).stop 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 msgr2=0x7f16cc1a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc1a0af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005940 tx=0x7f16c002d890 comp rx=0 tx=0).stop 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 shutdown_connections 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16cc069c10 0x7f16cc1a0af0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16b40a3ed0 0x7f16b40a6390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16cc10e8f0 0x7f16cc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 --2- 172.21.15.5:0/2071820363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16cc0692f0 0x7f16cc1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.519+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 >> 172.21.15.5:0/2071820363 conn(0x7f16cc077fa0 msgr2=0x7f16cc10feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.520+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 shutdown_connections 2024-04-22T13:28:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:22.520+0000 7f16d26ce640 1 -- 172.21.15.5:0/2071820363 wait complete. 2024-04-22T13:28:22.526 INFO:tasks.cephfs.filesystem:scrub status for tag:88cf0d4f-bc08-4b40-ae52-b0ac6297d14b - {'path': '/', 'tag': '88cf0d4f-bc08-4b40-ae52-b0ac6297d14b', 'options': 'recursive,force'} 2024-04-22T13:28:23.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:23 smithi049 ceph-mon[31379]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-22T13:28:23.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:23 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:28:23.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:23 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:23.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:23 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:28:23.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:23 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:23.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:23 smithi107 ceph-mon[31859]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-22T13:28:23.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:23 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:28:23.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:23 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:23.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:23 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:28:23.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:23 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:23 smithi005 ceph-mon[28006]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-22T13:28:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:23 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:28:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:23 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:23 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:28:23.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:23 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 -- 172.21.15.5:0/3564078817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f481010c0 msgr2=0x7f1f481014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 --2- 172.21.15.5:0/3564078817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f481010c0 0x7f1f481014a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009900 tx=0x7f1f3802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 -- 172.21.15.5:0/3564078817 shutdown_connections 2024-04-22T13:28:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 --2- 172.21.15.5:0/3564078817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48106190 0x7f1f4810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 --2- 172.21.15.5:0/3564078817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f48101a70 0x7f1f48105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 --2- 172.21.15.5:0/3564078817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f481010c0 0x7f1f481014a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 -- 172.21.15.5:0/3564078817 >> 172.21.15.5:0/3564078817 conn(0x7f1f48077fa0 msgr2=0x7f1f480783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 -- 172.21.15.5:0/3564078817 shutdown_connections 2024-04-22T13:28:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.992+0000 7f1f4d196640 1 -- 172.21.15.5:0/3564078817 wait complete. 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.993+0000 7f1f4d196640 1 Processor -- start 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.993+0000 7f1f4d196640 1 -- start start 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4d196640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481010c0 0x7f1f48196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4d196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 0x7f1f48196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4d196640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f4819add0 0x7f1f4819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 0x7f1f48196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 0x7f1f48196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56274/0 (socket says 172.21.15.5:56274) 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 -- 172.21.15.5:0/722772622 learned_addr learned my addr 172.21.15.5:0/722772622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48110030 con 0x7f1f48106190 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810feb0 con 0x7f1f4819add0 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481101b0 con 0x7f1f481010c0 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4759e640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f4819add0 0x7f1f4819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481010c0 0x7f1f48196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 -- 172.21.15.5:0/722772622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481010c0 msgr2=0x7f1f48196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481010c0 0x7f1f48196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 -- 172.21.15.5:0/722772622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f4819add0 msgr2=0x7f1f4819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f4819add0 0x7f1f4819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f3ffff640 1 -- 172.21.15.5:0/722772622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4819b8c0 con 0x7f1f48106190 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.994+0000 7f1f4759e640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f4819add0 0x7f1f4819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.995+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 0x7f1f48196a50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009990 tx=0x7f1f30009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.995+0000 7f1f44d99640 1 -- 172.21.15.5:0/722772622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30007c50 con 0x7f1f48106190 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.995+0000 7f1f44d99640 1 -- 172.21.15.5:0/722772622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3001c070 con 0x7f1f48106190 2024-04-22T13:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.995+0000 7f1f44d99640 1 -- 172.21.15.5:0/722772622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f300165b0 con 0x7f1f48106190 2024-04-22T13:28:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.995+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4819bbb0 con 0x7f1f48106190 2024-04-22T13:28:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.995+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481a3350 con 0x7f1f48106190 2024-04-22T13:28:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.998+0000 7f1f44d99640 1 -- 172.21.15.5:0/722772622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f1f300173f0 con 0x7f1f48106190 2024-04-22T13:28:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.998+0000 7f1f4fd85740 1 Processor -- start 2024-04-22T13:28:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 -- start start 2024-04-22T13:28:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0c9a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0c9afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0bff8d0 con 0x55d7a0c9b8d0 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0b9ce80 con 0x55d7a0ca40e0 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0bfb890 con 0x55d7a0c9ab60 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0c9a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0c9a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56278/0 (socket says 172.21.15.5:56278) 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f4759e640 1 -- 172.21.15.5:0/2868002095 learned_addr learned my addr 172.21.15.5:0/2868002095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0c9afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:23.999+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f46d9d640 1 -- 172.21.15.5:0/2868002095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0ca40e0 msgr2=0x55d7a0ca64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f46d9d640 1 -- 172.21.15.5:0/2868002095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 msgr2=0x55d7a0c9a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f44d99640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f340a3f40 0x7f1f340a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0c9a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f46d9d640 1 -- 172.21.15.5:0/2868002095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0bbe320 con 0x55d7a0c9ab60 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f4759e640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0c9a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f44d99640 1 -- 172.21.15.5:0/722772622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1f300d2900 con 0x7f1f48106190 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f4759e640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f340a3f40 0x7f1f340a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0c9afe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f3802c170 tx=0x7f1f3802c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e081f20ae46f0ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.000+0000 7f1f4759e640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f340a3f40 0x7f1f340a6400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f40014c60 tx=0x7f1f40004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f3d7fa640 1 -- 172.21.15.5:0/2868002095 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38045020 con 0x55d7a0c9ab60 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f3d7fa640 1 -- 172.21.15.5:0/2868002095 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3802c7e0 con 0x55d7a0c9ab60 2024-04-22T13:28:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f3d7fa640 1 -- 172.21.15.5:0/2868002095 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38005600 con 0x55d7a0c9ab60 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 -- 172.21.15.5:0/2868002095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 msgr2=0x55d7a0c9afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0c9afe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f3802c170 tx=0x7f1f3802c5f0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 -- 172.21.15.5:0/2868002095 shutdown_connections 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0c9afe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/2868002095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0c9a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.001+0000 7f1f4fd85740 1 -- 172.21.15.5:0/2868002095 >> 172.21.15.5:0/2868002095 conn(0x55d7a0c61c90 msgr2=0x55d7a0c95850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.002+0000 7f1f4fd85740 1 -- 172.21.15.5:0/2868002095 shutdown_connections 2024-04-22T13:28:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.002+0000 7f1f4fd85740 1 -- 172.21.15.5:0/2868002095 wait complete. 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.002+0000 7f1f4fd85740 1 -- start start 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.002+0000 7f1f4fd85740 1 Processor -- start 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ca6db0 con 0x55d7a0c9ab60 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0b87eb0 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ca6f30 con 0x55d7a0ca40e0 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56280/0 (socket says 172.21.15.5:56280) 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f46d9d640 1 -- 172.21.15.5:0/1534217001 learned_addr learned my addr 172.21.15.5:0/1534217001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4759e640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 -- 172.21.15.5:0/1534217001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0ca40e0 msgr2=0x55d7a0ca1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 -- 172.21.15.5:0/1534217001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 msgr2=0x55d7a0ca3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 -- 172.21.15.5:0/1534217001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0912210 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 10 client.0 _ll_register_callbacks cb 0x55d7a0bb4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3802c7e0 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca0e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f300cfe40 tx=0x7f1f3009a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5960483e1ace3fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.003+0000 7f1f2e7fc640 10 client.25784 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1534217001 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3009ed90 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1534217001 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1f300176e0 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1534217001 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f300178e0 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 10 client.25784 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55d7a09de620 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1534217001 <== mon.2 v2:172.21.15.49:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f3000c020 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 10 client.25784 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 20 client.25784 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cb1580 0x55d7a0cb39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 4 client.25784 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a0c13880 con 0x55d7a0cb1580 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.004+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cb1580 0x55d7a0cb39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.005+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cb1580 0x55d7a0cb39f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2f2f4020d52be0 server_cookie=aa4dc0ef04395e8 in_seq=0 out_seq=0 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.005+0000 7f1f2e7fc640 10 client.25784 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1534217001 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7a0c13880 con 0x55d7a0cb1580 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f2e7fc640 10 client.25784 handle_command_reply: tid=0 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 client.25784 shutdown 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 msgr2=0x55d7a0ca0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca0e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f300cfe40 tx=0x7f1f3009a9f0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 shutdown_connections 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cb1580 0x55d7a0cb39f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca0e00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f2e7fc640 0 client.25784 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1534217001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 >> 172.21.15.5:0/1534217001 conn(0x55d7a0c61c90 msgr2=0x55d7a0c975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 shutdown_connections 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1534217001 wait complete. 2024-04-22T13:28:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.006+0000 7f1f4fd85740 20 client.25784 trim_cache size 0 max 16384 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 Processor -- start 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 -- start start 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0cafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0cc4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ab810 con 0x55d7a0ca40e0 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ad350 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481abee0 con 0x55d7a0c9ab60 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0cc4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0cc4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56294/0 (socket says 172.21.15.5:56294) 2024-04-22T13:28:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f4759e640 1 -- 172.21.15.5:0/1769678314 learned_addr learned my addr 172.21.15.5:0/1769678314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0cafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.021+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f4759e640 1 -- 172.21.15.5:0/1769678314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 msgr2=0x55d7a0cafcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f4759e640 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0cafcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f4759e640 1 -- 172.21.15.5:0/1769678314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 msgr2=0x55d7a0ca3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f4759e640 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f4759e640 1 -- 172.21.15.5:0/1769678314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0cb4490 con 0x55d7a0ca40e0 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f4759e640 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0cc4880 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1f40014890 tx=0x7f1f40009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862238b4c8eb16ce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f3cff9640 1 -- 172.21.15.5:0/1769678314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40027070 con 0x55d7a0ca40e0 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f3cff9640 1 -- 172.21.15.5:0/1769678314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4001b040 con 0x55d7a0ca40e0 2024-04-22T13:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.022+0000 7f1f3cff9640 1 -- 172.21.15.5:0/1769678314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4001a070 con 0x55d7a0ca40e0 2024-04-22T13:28:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1769678314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 msgr2=0x55d7a0cc4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0cc4880 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1f40014890 tx=0x7f1f40009d40 comp rx=0 tx=0).stop 2024-04-22T13:28:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1769678314 shutdown_connections 2024-04-22T13:28:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0cc4880 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1769678314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9ab60 0x55d7a0cafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1769678314 >> 172.21.15.5:0/1769678314 conn(0x55d7a0c61c90 msgr2=0x55d7a0c9f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1769678314 shutdown_connections 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.023+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1769678314 wait complete. 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 -- start start 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 Processor -- start 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481acc80 con 0x55d7a0ca40e0 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ac5b0 con 0x55d7a0c9ab60 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:50202/0 (socket says 172.21.15.5:50202) 2024-04-22T13:28:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f46d9d640 1 -- 172.21.15.5:0/1881040754 learned_addr learned my addr 172.21.15.5:0/1881040754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481a9810 con 0x55d7a0c9b8d0 2024-04-22T13:28:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4759e640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.024+0000 7f1f4fd85740 10 client.0 _ll_register_callbacks cb 0x7f1f3009a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f46d9d640 1 -- 172.21.15.5:0/1881040754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9b8d0 msgr2=0x55d7a0ca4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f46d9d640 1 -- 172.21.15.5:0/1881040754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 msgr2=0x55d7a0ca4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f46d9d640 1 -- 172.21.15.5:0/1881040754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0cab270 con 0x55d7a0c9ab60 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f46d9d640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca44c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005c40 tx=0x7f1f3802c1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec227de1e8670f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1881040754 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38050070 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4001b040 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1881040754 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1f38004450 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1881040754 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f38045040 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f4fd85740 10 client.25790 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.025+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7a0cac540 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1881040754 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4001b040 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1881040754 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f38041610 con 0x55d7a0c9ab60 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f4fd85740 10 client.25790 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f4fd85740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f4fd85740 20 client.25790 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cac830 0x55d7a0d25100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f4fd85740 4 client.25790 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:28:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7a0b9f3d0 con 0x55d7a0cac830 2024-04-22T13:28:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cac830 0x55d7a0d25100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f2e7fc640 10 client.25790 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.026+0000 7f1f3ffff640 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cac830 0x55d7a0d25100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e66beb43d4a930c server_cookie=70b85e3ded86b3ca in_seq=0 out_seq=0 2024-04-22T13:28:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.030+0000 7f1f2e7fc640 1 -- 172.21.15.5:0/1881040754 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7a0b9f3d0 con 0x55d7a0cac830 2024-04-22T13:28:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.030+0000 7f1f2e7fc640 10 client.25790 handle_command_reply: tid=0 2024-04-22T13:28:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.030+0000 7f1f4fd85740 1 client.25790 shutdown 2024-04-22T13:28:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 msgr2=0x55d7a0ca44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca44c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005c40 tx=0x7f1f3802c1f0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 shutdown_connections 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d7a0cac830 0x55d7a0d25100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7a0ca40e0 0x55d7a0ca4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a0c9b8d0 0x55d7a0ca4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 --2- 172.21.15.5:0/1881040754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a0c9ab60 0x55d7a0ca44c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 >> 172.21.15.5:0/1881040754 conn(0x55d7a0c61c90 msgr2=0x55d7a0c9f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f2e7fc640 0 client.25790 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 shutdown_connections 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 1 -- 172.21.15.5:0/1881040754 wait complete. 2024-04-22T13:28:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4fd85740 20 client.25790 trim_cache size 0 max 16384 2024-04-22T13:28:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:28:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:28:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:28:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f340a3f40 msgr2=0x7f1f340a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.031+0000 7f1f4d196640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f340a3f40 0x7f1f340a6400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f40014c60 tx=0x7f1f40004ab0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 msgr2=0x7f1f48196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 0x7f1f48196a50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009990 tx=0x7f1f30009e50 comp rx=0 tx=0).stop 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 shutdown_connections 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f48106190 0x7f1f48196a50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f340a3f40 0x7f1f340a6400 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f4819add0 0x7f1f4819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 --2- 172.21.15.5:0/722772622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481010c0 0x7f1f48196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 >> 172.21.15.5:0/722772622 conn(0x7f1f48077fa0 msgr2=0x7f1f480ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 shutdown_connections 2024-04-22T13:28:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:24.032+0000 7f1f4d196640 1 -- 172.21.15.5:0/722772622 wait complete. 2024-04-22T13:28:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:25 smithi049 ceph-mon[31379]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:28:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:25 smithi107 ceph-mon[31859]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:28:25.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:25 smithi005 ceph-mon[28006]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:28:26.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:26 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:28:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:26 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:28:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:26 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:28:27.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:27 smithi049 ceph-mon[31379]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:28:27.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:27 smithi107 ceph-mon[31859]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:28:27.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:27 smithi005 ceph-mon[28006]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:28:29.336 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:29 smithi107 ceph-mon[31859]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:28:29.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:29 smithi049 ceph-mon[31379]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:28:29.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:29 smithi005 ceph-mon[28006]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:28:31.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:31 smithi049 ceph-mon[31379]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:31.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:31 smithi107 ceph-mon[31859]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:31.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:31 smithi005 ceph-mon[28006]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:33.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:33 smithi049 ceph-mon[31379]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:28:33.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:33 smithi107 ceph-mon[31859]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:28:33.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:33 smithi005 ceph-mon[28006]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:28:35.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:35 smithi049 ceph-mon[31379]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:35.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:35 smithi107 ceph-mon[31859]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:35.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:35 smithi005 ceph-mon[28006]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.169+0000 7fb06b65c640 1 -- 172.21.15.5:0/2927022677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 msgr2=0x7fb0641025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.169+0000 7fb06b65c640 1 --2- 172.21.15.5:0/2927022677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb0641025c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb060009900 tx=0x7fb06002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.169+0000 7fb06b65c640 1 -- 172.21.15.5:0/2927022677 shutdown_connections 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.169+0000 7fb06b65c640 1 --2- 172.21.15.5:0/2927022677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 0x7fb06410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.169+0000 7fb06b65c640 1 --2- 172.21.15.5:0/2927022677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb064102b90 0x7fb064106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.169+0000 7fb06b65c640 1 --2- 172.21.15.5:0/2927022677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb0641025c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.170+0000 7fb06b65c640 1 -- 172.21.15.5:0/2927022677 >> 172.21.15.5:0/2927022677 conn(0x7fb0640fda00 msgr2=0x7fb0640ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.170+0000 7fb06b65c640 1 -- 172.21.15.5:0/2927022677 shutdown_connections 2024-04-22T13:28:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.170+0000 7fb06b65c640 1 -- 172.21.15.5:0/2927022677 wait complete. 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.171+0000 7fb06b65c640 1 Processor -- start 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.171+0000 7fb06b65c640 1 -- start start 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.171+0000 7fb06b65c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb06419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.171+0000 7fb06b65c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 0x7fb06419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb06b65c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0641985c0 0x7fb064198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb06b65c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb064112240 con 0x7fb0641021e0 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb06b65c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641120c0 con 0x7fb0641985c0 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb06419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb06b65c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641123c0 con 0x7fb0641072b0 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb068bf7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 0x7fb06419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb06419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59358/0 (socket says 172.21.15.5:59358) 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb0693f8640 1 -- 172.21.15.5:0/1586327799 learned_addr learned my addr 172.21.15.5:0/1586327799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0641985c0 0x7fb064198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb068bf7640 1 -- 172.21.15.5:0/1586327799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0641985c0 msgr2=0x7fb064198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb068bf7640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0641985c0 0x7fb064198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb068bf7640 1 -- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 msgr2=0x7fb06419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb068bf7640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb06419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb068bf7640 1 -- 172.21.15.5:0/1586327799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0641990e0 con 0x7fb0641072b0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.172+0000 7fb0693f8640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb06419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.173+0000 7fb068bf7640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 0x7fb06419e950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb054009b90 tx=0x7fb054009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.173+0000 7fb05a7fc640 1 -- 172.21.15.5:0/1586327799 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05401e070 con 0x7fb0641072b0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.173+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0641993d0 con 0x7fb0641072b0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.173+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb064074520 con 0x7fb0641072b0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.173+0000 7fb05a7fc640 1 -- 172.21.15.5:0/1586327799 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05400bc30 con 0x7fb0641072b0 2024-04-22T13:28:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.173+0000 7fb05a7fc640 1 -- 172.21.15.5:0/1586327799 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054018610 con 0x7fb0641072b0 2024-04-22T13:28:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 Processor -- start 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 -- start start 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f70620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 0x55aca4f70fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f7a0e0 0x55aca4f7c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aca4ed58d0 con 0x55aca4f7a0e0 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aca4e72e80 con 0x55aca4f718d0 2024-04-22T13:28:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb06e255740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55aca4ed1890 con 0x55aca4f70b60 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb05a7fc640 1 -- 172.21.15.5:0/1586327799 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7fb054021070 con 0x7fb0641072b0 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 0x55aca4f70fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb069bf9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f70620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 0x55aca4f70fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:54816/0 (socket says 172.21.15.5:54816) 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb069bf9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f70620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56094/0 (socket says 172.21.15.5:56094) 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.177+0000 7fb0693f8640 1 -- 172.21.15.5:0/3907050499 learned_addr learned my addr 172.21.15.5:0/3907050499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb0693f8640 1 -- 172.21.15.5:0/3907050499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 msgr2=0x55aca4f70620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb0693f8640 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f70620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb0693f8640 1 -- 172.21.15.5:0/3907050499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f7a0e0 msgr2=0x55aca4f7c4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb0693f8640 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f7a0e0 0x55aca4f7c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb0693f8640 1 -- 172.21.15.5:0/3907050499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca4e94320 con 0x55aca4f70b60 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb069bf9640 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f70620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb0693f8640 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 0x55aca4f70fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb060036040 tx=0x7fb060009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7a93c6750da9be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb04f7fe640 1 -- 172.21.15.5:0/3907050499 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060045070 con 0x55aca4f70b60 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb05a7fc640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0500a3ec0 0x7fb0500a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb04f7fe640 1 -- 172.21.15.5:0/3907050499 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06002c7e0 con 0x55aca4f70b60 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.178+0000 7fb04f7fe640 1 -- 172.21.15.5:0/3907050499 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060003c20 con 0x55aca4f70b60 2024-04-22T13:28:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.179+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0500a3ec0 0x7fb0500a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.179+0000 7fb06e255740 1 -- 172.21.15.5:0/3907050499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 msgr2=0x55aca4f70fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.179+0000 7fb06e255740 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 0x55aca4f70fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb060036040 tx=0x7fb060009670 comp rx=0 tx=0).stop 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.179+0000 7fb05a7fc640 1 -- 172.21.15.5:0/1586327799 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb0540d3fb0 con 0x7fb0641072b0 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.179+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0500a3ec0 0x7fb0500a6380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005fd0 tx=0x7fb05c009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 -- 172.21.15.5:0/3907050499 shutdown_connections 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f7a0e0 0x55aca4f7c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f70b60 0x55aca4f70fe0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 --2- 172.21.15.5:0/3907050499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f70620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 -- 172.21.15.5:0/3907050499 >> 172.21.15.5:0/3907050499 conn(0x55aca4f37c90 msgr2=0x55aca4f6b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 -- 172.21.15.5:0/3907050499 shutdown_connections 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.180+0000 7fb06e255740 1 -- 172.21.15.5:0/3907050499 wait complete. 2024-04-22T13:28:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.181+0000 7fb06e255740 1 -- start start 2024-04-22T13:28:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.181+0000 7fb06e255740 1 Processor -- start 2024-04-22T13:28:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb06e255740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f70b60 0x55aca4f799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb06e255740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 0x55aca4f76d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb06e255740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f7a0e0 0x55aca4f77280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb06e255740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aca4f7cdb0 con 0x55aca4f718d0 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb068bf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 0x55aca4f76d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb068bf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 0x55aca4f76d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59368/0 (socket says 172.21.15.5:59368) 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb068bf7640 1 -- 172.21.15.5:0/1586090048 learned_addr learned my addr 172.21.15.5:0/1586090048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb0693f8640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f70b60 0x55aca4f799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aca4e5deb0 con 0x55aca4f70b60 2024-04-22T13:28:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.182+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f7a0e0 0x55aca4f77280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55aca4f7cf30 con 0x55aca4f7a0e0 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb06e255740 10 client.0 _ll_register_callbacks cb 0x55aca4e8ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb069bf9640 1 -- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f70b60 msgr2=0x55aca4f799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f70b60 0x55aca4f799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb069bf9640 1 -- 172.21.15.5:0/1586090048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 msgr2=0x55aca4f76d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 0x55aca4f76d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb069bf9640 1 -- 172.21.15.5:0/1586090048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca4be8210 con 0x55aca4f7a0e0 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb068bf7640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 0x55aca4f76d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb069bf9640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f7a0e0 0x55aca4f77280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb05c012f60 tx=0x7fb05c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c94f91efeb6d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.183+0000 7fb04cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.184+0000 7fb04cff9640 1 -- 172.21.15.5:0/1586090048 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c003d00 con 0x55aca4f7a0e0 2024-04-22T13:28:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.184+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06002c7e0 con 0x55aca4f7a0e0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.184+0000 7fb04cff9640 1 -- 172.21.15.5:0/1586090048 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb05c004050 con 0x55aca4f7a0e0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.184+0000 7fb04cff9640 1 -- 172.21.15.5:0/1586090048 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb05c004250 con 0x55aca4f7a0e0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.184+0000 7fb06e255740 10 client.25894 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.184+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aca4cb4620 con 0x55aca4f7a0e0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.185+0000 7fb04cff9640 1 -- 172.21.15.5:0/1586090048 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c032450 con 0x55aca4f7a0e0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb04cff9640 1 -- 172.21.15.5:0/1586090048 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb05c027020 con 0x55aca4f7a0e0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb06e255740 10 client.25894 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb06e255740 10 client.25894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb06e255740 20 client.25894 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb06e255740 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f8a500 0x55aca4f8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb06e255740 4 client.25894 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb06002c7e0 con 0x55aca4f8a500 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.186+0000 7fb0693f8640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f8a500 0x55aca4f8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.187+0000 7fb04cff9640 10 client.25894 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.187+0000 7fb0693f8640 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f8a500 0x55aca4f8c960 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228140369d3919f2 server_cookie=10e2c12a6a64e8c2 in_seq=0 out_seq=0 2024-04-22T13:28:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb04cff9640 1 -- 172.21.15.5:0/1586090048 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb06002c7e0 con 0x55aca4f8a500 2024-04-22T13:28:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb04cff9640 10 client.25894 handle_command_reply: tid=0 2024-04-22T13:28:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 client.25894 shutdown 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f7a0e0 msgr2=0x55aca4f77280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f7a0e0 0x55aca4f77280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb05c012f60 tx=0x7fb05c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 shutdown_connections 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f8a500 0x55aca4f8c960 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f718d0 0x55aca4f76d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb04cff9640 0 client.25894 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f7a0e0 0x55aca4f77280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.188+0000 7fb06e255740 1 --2- 172.21.15.5:0/1586090048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f70b60 0x55aca4f799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.189+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 >> 172.21.15.5:0/1586090048 conn(0x55aca4f37c90 msgr2=0x55aca4f6d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.189+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 shutdown_connections 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.189+0000 7fb06e255740 1 -- 172.21.15.5:0/1586090048 wait complete. 2024-04-22T13:28:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.189+0000 7fb06e255740 20 client.25894 trim_cache size 0 max 16384 2024-04-22T13:28:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.219+0000 7fb06e255740 1 Processor -- start 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.219+0000 7fb06e255740 1 -- start start 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.219+0000 7fb06e255740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f37c90 0x55aca4f85d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.219+0000 7fb06e255740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.219+0000 7fb06e255740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f79470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb06e255740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb048016ee0 con 0x55aca4f70b60 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb068bf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f37c90 0x55aca4f85d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb068bf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f72080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59378/0 (socket says 172.21.15.5:59378) 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb068bf7640 1 -- 172.21.15.5:0/2610789906 learned_addr learned my addr 172.21.15.5:0/2610789906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb069bf9640 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f79470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb048018a20 con 0x55aca4f718d0 2024-04-22T13:28:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480175b0 con 0x55aca4f37c90 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 -- 172.21.15.5:0/2610789906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 msgr2=0x55aca4f79470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f79470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 -- 172.21.15.5:0/2610789906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 msgr2=0x55aca4f72080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f72080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 -- 172.21.15.5:0/2610789906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca4e8ae90 con 0x55aca4f37c90 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb068bf7640 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.220+0000 7fb0693f8640 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f37c90 0x55aca4f85d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb060006010 tx=0x7fb060041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20848d703ab1d707 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.221+0000 7fb04effd640 1 -- 172.21.15.5:0/2610789906 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060052070 con 0x55aca4f37c90 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.221+0000 7fb04effd640 1 -- 172.21.15.5:0/2610789906 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06002c7e0 con 0x55aca4f37c90 2024-04-22T13:28:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.221+0000 7fb04effd640 1 -- 172.21.15.5:0/2610789906 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060045040 con 0x55aca4f37c90 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f37c90 msgr2=0x55aca4f85d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f37c90 0x55aca4f85d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb060006010 tx=0x7fb060041040 comp rx=0 tx=0).stop 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 shutdown_connections 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f718d0 0x55aca4f79470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 --2- 172.21.15.5:0/2610789906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f37c90 0x55aca4f85d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 >> 172.21.15.5:0/2610789906 conn(0x55aca4f773e0 msgr2=0x55aca4f75610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 shutdown_connections 2024-04-22T13:28:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 -- 172.21.15.5:0/2610789906 wait complete. 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 -- start start 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.222+0000 7fb06e255740 1 Processor -- start 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 0x55aca4f82690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f7aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f718d0 0x55aca4f7b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb048018350 con 0x55aca4f70b60 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 0x55aca4f82690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb0693f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 0x55aca4f82690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:56112/0 (socket says 172.21.15.5:56112) 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb068bf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f7aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb0693f8640 1 -- 172.21.15.5:0/84625139 learned_addr learned my addr 172.21.15.5:0/84625139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb048017c80 con 0x55aca4f37c90 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb048014ec0 con 0x55aca4f718d0 2024-04-22T13:28:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb069bf9640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f718d0 0x55aca4f7b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.223+0000 7fb06e255740 10 client.0 _ll_register_callbacks cb 0x7fb05c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb0693f8640 1 -- 172.21.15.5:0/84625139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f718d0 msgr2=0x55aca4f7b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb0693f8640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f718d0 0x55aca4f7b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb0693f8640 1 -- 172.21.15.5:0/84625139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 msgr2=0x55aca4f7aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb0693f8640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f7aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb0693f8640 1 -- 172.21.15.5:0/84625139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca4f7b870 con 0x55aca4f37c90 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb068bf7640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f7aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb0693f8640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 0x55aca4f82690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb06002c780 tx=0x7fb060054e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad054dcbba7b72b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb04cff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb04cff9640 1 -- 172.21.15.5:0/84625139 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060045040 con 0x55aca4f37c90 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06002c7e0 con 0x55aca4f37c90 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb04cff9640 1 -- 172.21.15.5:0/84625139 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb060054520 con 0x55aca4f37c90 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb04cff9640 1 -- 172.21.15.5:0/84625139 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0600035f0 con 0x55aca4f37c90 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb06e255740 10 client.25808 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.224+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aca4ff0b00 con 0x55aca4f37c90 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb04cff9640 1 -- 172.21.15.5:0/84625139 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06002c7e0 con 0x55aca4f37c90 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb04cff9640 1 -- 172.21.15.5:0/84625139 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb06005d070 con 0x55aca4f37c90 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb06e255740 10 client.25808 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb06e255740 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb06e255740 20 client.25808 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb06e255740 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f7a0e0 0x55aca4ff2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb06e255740 4 client.25808 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aca4ee9880 con 0x55aca4f7a0e0 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb068bf7640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f7a0e0 0x55aca4ff2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb04cff9640 10 client.25808 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.225+0000 7fb068bf7640 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f7a0e0 0x55aca4ff2db0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0f68ad7506d120 server_cookie=ec7ded49ccd6f876 in_seq=0 out_seq=0 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb04cff9640 1 -- 172.21.15.5:0/84625139 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aca4ee9880 con 0x55aca4f7a0e0 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb04cff9640 10 client.25808 handle_command_reply: tid=0 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 client.25808 shutdown 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 msgr2=0x55aca4f82690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 0x55aca4f82690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb06002c780 tx=0x7fb060054e00 comp rx=0 tx=0).stop 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 shutdown_connections 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55aca4f7a0e0 0x55aca4ff2db0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aca4f70b60 0x55aca4f7aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aca4f718d0 0x55aca4f7b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 --2- 172.21.15.5:0/84625139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aca4f37c90 0x55aca4f82690 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 >> 172.21.15.5:0/84625139 conn(0x55aca4f773e0 msgr2=0x55aca4f6a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb04cff9640 0 client.25808 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 shutdown_connections 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.230+0000 7fb06e255740 1 -- 172.21.15.5:0/84625139 wait complete. 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06e255740 20 client.25808 trim_cache size 0 max 16384 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:28:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:28:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0500a3ec0 msgr2=0x7fb0500a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0500a3ec0 0x7fb0500a6380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005fd0 tx=0x7fb05c009500 comp rx=0 tx=0).stop 2024-04-22T13:28:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 msgr2=0x7fb06419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 0x7fb06419e950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb054009b90 tx=0x7fb054009f90 comp rx=0 tx=0).stop 2024-04-22T13:28:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 shutdown_connections 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb0500a3ec0 0x7fb0500a6380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0641985c0 0x7fb064198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0641072b0 0x7fb06419e950 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 --2- 172.21.15.5:0/1586327799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0641021e0 0x7fb06419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.231+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 >> 172.21.15.5:0/1586327799 conn(0x7fb0640fda00 msgr2=0x7fb0640fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.232+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 shutdown_connections 2024-04-22T13:28:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:37.232+0000 7fb06b65c640 1 -- 172.21.15.5:0/1586327799 wait complete. 2024-04-22T13:28:37.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:37 smithi005 ceph-mon[28006]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:28:37.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:37 smithi049 ceph-mon[31379]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:28:37.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:37 smithi107 ceph-mon[31859]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:28:39.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:39 smithi049 ceph-mon[31379]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:39.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:39 smithi107 ceph-mon[31859]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:39.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:39 smithi005 ceph-mon[28006]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:39.812 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-22T13:28:39.812 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-04-22T13:28:39.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 -- 172.21.15.5:0/2683867761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101a20 msgr2=0x7f93ac105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 --2- 172.21.15.5:0/2683867761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101a20 0x7f93ac105a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f939c009900 tx=0x7f939c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 -- 172.21.15.5:0/2683867761 shutdown_connections 2024-04-22T13:28:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 --2- 172.21.15.5:0/2683867761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 --2- 172.21.15.5:0/2683867761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101a20 0x7f93ac105a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 --2- 172.21.15.5:0/2683867761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac101070 0x7f93ac101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 -- 172.21.15.5:0/2683867761 >> 172.21.15.5:0/2683867761 conn(0x7f93ac0771d0 msgr2=0x7f93ac0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 -- 172.21.15.5:0/2683867761 shutdown_connections 2024-04-22T13:28:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.975+0000 7f93b4315640 1 -- 172.21.15.5:0/2683867761 wait complete. 2024-04-22T13:28:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.976+0000 7f93b4315640 1 Processor -- start 2024-04-22T13:28:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.976+0000 7f93b4315640 1 -- start start 2024-04-22T13:28:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b4315640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101070 0x7f93ac197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b4315640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b4315640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac19bcc0 0x7f93ac19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b4315640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac1110d0 con 0x7f93ac106140 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b4315640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac110f50 con 0x7f93ac19bcc0 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b4315640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac111250 con 0x7f93ac101070 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46332/0 (socket says 172.21.15.5:46332) 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 -- 172.21.15.5:0/2650907201 learned_addr learned my addr 172.21.15.5:0/2650907201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b20b1640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101070 0x7f93ac197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b28b2640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac19bcc0 0x7f93ac19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 -- 172.21.15.5:0/2650907201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101070 msgr2=0x7f93ac197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101070 0x7f93ac197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 -- 172.21.15.5:0/2650907201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac19bcc0 msgr2=0x7f93ac19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac19bcc0 0x7f93ac19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.977+0000 7f93b18b0640 1 -- 172.21.15.5:0/2650907201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac19c7b0 con 0x7f93ac106140 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.978+0000 7f93b18b0640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac197990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f939c02c170 tx=0x7f939c02ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.978+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c044030 con 0x7f93ac106140 2024-04-22T13:28:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.978+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac19ca40 con 0x7f93ac106140 2024-04-22T13:28:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.978+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f939c02cc80 con 0x7f93ac106140 2024-04-22T13:28:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.978+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c0035f0 con 0x7f93ac106140 2024-04-22T13:28:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.978+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac069540 con 0x7f93ac106140 2024-04-22T13:28:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.980+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac068e00 con 0x7f93ac106140 2024-04-22T13:28:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.981+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f939c047030 con 0x7f93ac106140 2024-04-22T13:28:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.982+0000 7f93a37fe640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f93980a3f50 0x7f93980a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.983+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f939c0f6aa0 con 0x7f93ac106140 2024-04-22T13:28:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.983+0000 7f93b20b1640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f93980a3f50 0x7f93980a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.983+0000 7f93b20b1640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f93980a3f50 0x7f93980a6410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009c20 tx=0x7f93a8005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:39.991+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f939c0c3040 con 0x7f93ac106140 2024-04-22T13:28:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.252+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93ac101a20 con 0x7f93ac106140 2024-04-22T13:28:40.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.253+0000 7f93a37fe640 1 -- 172.21.15.5:0/2650907201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f939c0c2940 con 0x7f93ac106140 2024-04-22T13:28:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:28:40.253 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f93980a3f50 msgr2=0x7f93980a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f93980a3f50 0x7f93980a6410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009c20 tx=0x7f93a8005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 msgr2=0x7f93ac197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac197990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f939c02c170 tx=0x7f939c02ca90 comp rx=0 tx=0).stop 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 shutdown_connections 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93ac106140 0x7f93ac197990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f93980a3f50 0x7f93980a6410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac19bcc0 0x7f93ac19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 --2- 172.21.15.5:0/2650907201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ac101070 0x7f93ac197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.256+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 >> 172.21.15.5:0/2650907201 conn(0x7f93ac0771d0 msgr2=0x7f93ac0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.257+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 shutdown_connections 2024-04-22T13:28:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.257+0000 7f93b4315640 1 -- 172.21.15.5:0/2650907201 wait complete. 2024-04-22T13:28:40.269 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db / --timeout 60 --expiration 86.5 --await' 2024-04-22T13:28:40.269 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db / --timeout 60 --expiration 86.5 --await 2024-04-22T13:28:40.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 -- 172.21.15.5:0/681784003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069b70 msgr2=0x7f088810e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 --2- 172.21.15.5:0/681784003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069b70 0x7f088810e710 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0878009900 tx=0x7f087802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 -- 172.21.15.5:0/681784003 shutdown_connections 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 --2- 172.21.15.5:0/681784003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069b70 0x7f088810e710 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 --2- 172.21.15.5:0/681784003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f088810ec50 0x7f0888111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 --2- 172.21.15.5:0/681784003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0888069250 0x7f0888069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.386+0000 7f088fc43640 1 -- 172.21.15.5:0/681784003 >> 172.21.15.5:0/681784003 conn(0x7f0888077c40 msgr2=0x7f0888078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.387+0000 7f088fc43640 1 -- 172.21.15.5:0/681784003 shutdown_connections 2024-04-22T13:28:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.387+0000 7f088fc43640 1 -- 172.21.15.5:0/681784003 wait complete. 2024-04-22T13:28:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.387+0000 7f088fc43640 1 Processor -- start 2024-04-22T13:28:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.387+0000 7f088fc43640 1 -- start start 2024-04-22T13:28:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.387+0000 7f088fc43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 0x7f08881a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088fc43640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0888069b70 0x7f08881a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088fc43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f088810ec50 0x7f088819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088fc43640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08881143d0 con 0x7f0888069250 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088fc43640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0888114250 con 0x7f0888069b70 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088fc43640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0888114550 con 0x7f088810ec50 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 0x7f08881a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 0x7f08881a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46356/0 (socket says 172.21.15.5:46356) 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 -- 172.21.15.5:0/1451183541 learned_addr learned my addr 172.21.15.5:0/1451183541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d1de640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0888069b70 0x7f08881a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088e1e0640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f088810ec50 0x7f088819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 -- 172.21.15.5:0/1451183541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f088810ec50 msgr2=0x7f088819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f088810ec50 0x7f088819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 -- 172.21.15.5:0/1451183541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0888069b70 msgr2=0x7f08881a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0888069b70 0x7f08881a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 -- 172.21.15.5:0/1451183541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088819afa0 con 0x7f0888069250 2024-04-22T13:28:40.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.388+0000 7f088d9df640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 0x7f08881a0660 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9f0 tx=0x7f088400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:40.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.389+0000 7f087effd640 1 -- 172.21.15.5:0/1451183541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088400cd80 con 0x7f0888069250 2024-04-22T13:28:40.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.389+0000 7f087effd640 1 -- 172.21.15.5:0/1451183541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08840108e0 con 0x7f0888069250 2024-04-22T13:28:40.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.389+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088819b290 con 0x7f0888069250 2024-04-22T13:28:40.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.389+0000 7f087effd640 1 -- 172.21.15.5:0/1451183541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884018980 con 0x7f0888069250 2024-04-22T13:28:40.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.389+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a74b0 con 0x7f0888069250 2024-04-22T13:28:40.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.391+0000 7f087effd640 1 -- 172.21.15.5:0/1451183541 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f0884010430 con 0x7f0888069250 2024-04-22T13:28:40.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.392+0000 7f087effd640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08740a3fd0 0x7f08740a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.392+0000 7f088d1de640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08740a3fd0 0x7f08740a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.392+0000 7f087effd640 1 -- 172.21.15.5:0/1451183541 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f08840d0d80 con 0x7f0888069250 2024-04-22T13:28:40.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 Processor -- start 2024-04-22T13:28:40.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 -- start start 2024-04-22T13:28:40.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f088d1de640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08740a3fd0 0x7f08740a6490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0878006010 tx=0x7f0878005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5d9c0 0x560525a5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5e2e0 0x560525a66980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5605259be0b0 con 0x560525a66ec0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56052596f680 con 0x560525a5d9c0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f0892834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56052597bc60 con 0x560525a5e2e0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f088e1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f088d1de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5e2e0 0x560525a66980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f088e1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46360/0 (socket says 172.21.15.5:46360) 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f088e1e0640 1 -- 172.21.15.5:0/3403439022 learned_addr learned my addr 172.21.15.5:0/3403439022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.393+0000 7f088d9df640 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5d9c0 0x560525a5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d1de640 1 -- 172.21.15.5:0/3403439022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5d9c0 msgr2=0x560525a5dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d1de640 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5d9c0 0x560525a5dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d1de640 1 -- 172.21.15.5:0/3403439022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 msgr2=0x560525a692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d1de640 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d1de640 1 -- 172.21.15.5:0/3403439022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605257a0c90 con 0x560525a5e2e0 2024-04-22T13:28:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d9df640 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5d9c0 0x560525a5dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088e1e0640 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f088d1de640 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5e2e0 0x560525a66980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f087803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112ddc8c6f1df963 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f0873fff640 1 -- 172.21.15.5:0/3403439022 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878050070 con 0x560525a5e2e0 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f0873fff640 1 -- 172.21.15.5:0/3403439022 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087802c880 con 0x560525a5e2e0 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.394+0000 7f0873fff640 1 -- 172.21.15.5:0/3403439022 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878047040 con 0x560525a5e2e0 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 -- 172.21.15.5:0/3403439022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5e2e0 msgr2=0x560525a66980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5e2e0 0x560525a66980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f087803e000 comp rx=0 tx=0).stop 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 -- 172.21.15.5:0/3403439022 shutdown_connections 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5e2e0 0x560525a66980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 --2- 172.21.15.5:0/3403439022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5d9c0 0x560525a5dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 -- 172.21.15.5:0/3403439022 >> 172.21.15.5:0/3403439022 conn(0x560525a580f0 msgr2=0x560525a584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 -- 172.21.15.5:0/3403439022 shutdown_connections 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.395+0000 7f0892834740 1 -- 172.21.15.5:0/3403439022 wait complete. 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.396+0000 7f0892834740 1 -- start start 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.396+0000 7f0892834740 1 Processor -- start 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.396+0000 7f0892834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.396+0000 7f0892834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f0892834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a64a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088e1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a64a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a64000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:55908/0 (socket says 172.21.15.5:55908) 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088e1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a64a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46370/0 (socket says 172.21.15.5:46370) 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d1de640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 -- 172.21.15.5:0/3698385430 learned_addr learned my addr 172.21.15.5:0/3698385430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5605259c1400 con 0x560525a66ec0 2024-04-22T13:28:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56052597d840 con 0x560525a5e2e0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5605259c6720 con 0x560525a5d9c0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 -- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 msgr2=0x560525a64540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a64540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 -- 172.21.15.5:0/3698385430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 msgr2=0x560525a64a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a64a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d9df640 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560525801420 con 0x560525a5d9c0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088d1de640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.397+0000 7f088e1e0640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a64a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f0892834740 10 client.0 _ll_register_callbacks cb 0x5605259ceea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087802c880 con 0x560525a5d9c0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f088d9df640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a64000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f088409cb50 tx=0x7f0884010870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd810c9d50c61432 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f08717fa640 10 client.25918 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f08717fa640 1 -- 172.21.15.5:0/3698385430 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088409c010 con 0x560525a5d9c0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f08717fa640 1 -- 172.21.15.5:0/3698385430 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f088409c1b0 con 0x560525a5d9c0 2024-04-22T13:28:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f08717fa640 1 -- 172.21.15.5:0/3698385430 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0884018b20 con 0x560525a5d9c0 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f0892834740 10 client.25918 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.398+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=27}) -- 0x560525694580 con 0x560525a5d9c0 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f08717fa640 1 -- 172.21.15.5:0/3698385430 <== mon.1 v2:172.21.15.107:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0884018cc0 con 0x560525a5d9c0 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f0892834740 10 client.25918 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f0892834740 10 client.25918 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f0892834740 20 client.25918 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f0892834740 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525a769d0 0x560525a78e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f0892834740 4 client.25918 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605259cfb10 con 0x560525a769d0 2024-04-22T13:28:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f088d1de640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525a769d0 0x560525a78e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.399+0000 7f088d1de640 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525a769d0 0x560525a78e20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0c5677ebd0c51c server_cookie=e7100dc301b28284 in_seq=0 out_seq=0 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.400+0000 7f08717fa640 10 client.25918 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.400+0000 7f08717fa640 1 -- 172.21.15.5:0/3698385430 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605259cfb10 con 0x560525a769d0 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.400+0000 7f08717fa640 10 client.25918 handle_command_reply: tid=0 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.400+0000 7f0892834740 1 client.25918 shutdown 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 msgr2=0x560525a64000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a64000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f088409cb50 tx=0x7f0884010870 comp rx=0 tx=0).stop 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 shutdown_connections 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525a769d0 0x560525a78e20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f08717fa640 0 client.25918 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a66ec0 0x560525a64a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 --2- 172.21.15.5:0/3698385430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a64000 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 >> 172.21.15.5:0/3698385430 conn(0x560525a580f0 msgr2=0x560525a59e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 shutdown_connections 2024-04-22T13:28:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 1 -- 172.21.15.5:0/3698385430 wait complete. 2024-04-22T13:28:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.401+0000 7f0892834740 20 client.25918 trim_cache size 0 max 16384 2024-04-22T13:28:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 Processor -- start 2024-04-22T13:28:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 -- start start 2024-04-22T13:28:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a66730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 0x560525a7e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a66ec0 0x560525a7ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b18f0 con 0x560525a5e2e0 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b0480 con 0x560525a66ec0 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f0892834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b0b50 con 0x560525a5d9c0 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f088d1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 0x560525a7e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f088d1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 0x560525a7e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46384/0 (socket says 172.21.15.5:46384) 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f088d1de640 1 -- 172.21.15.5:0/2892405682 learned_addr learned my addr 172.21.15.5:0/2892405682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.435+0000 7f088d9df640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a66730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088e1e0640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a66ec0 0x560525a7ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d9df640 1 -- 172.21.15.5:0/2892405682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a66ec0 msgr2=0x560525a7ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d9df640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a66ec0 0x560525a7ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d9df640 1 -- 172.21.15.5:0/2892405682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 msgr2=0x560525a7e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d9df640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 0x560525a7e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d9df640 1 -- 172.21.15.5:0/2892405682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560525a69620 con 0x560525a5d9c0 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d1de640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 0x560525a7e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088e1e0640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a66ec0 0x560525a7ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f088d9df640 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a66730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f088400c6a0 tx=0x7f088401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b6395e1407cc9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f08737fe640 1 -- 172.21.15.5:0/2892405682 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088409c010 con 0x560525a5d9c0 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.436+0000 7f08737fe640 1 -- 172.21.15.5:0/2892405682 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087802c880 con 0x560525a5d9c0 2024-04-22T13:28:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f08737fe640 1 -- 172.21.15.5:0/2892405682 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840ce830 con 0x560525a5d9c0 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f0892834740 1 -- 172.21.15.5:0/2892405682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 msgr2=0x560525a66730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f0892834740 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a66730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f088400c6a0 tx=0x7f088401a000 comp rx=0 tx=0).stop 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f0892834740 1 -- 172.21.15.5:0/2892405682 shutdown_connections 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f0892834740 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5e2e0 0x560525a7e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f0892834740 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a66ec0 0x560525a7ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.437+0000 7f0892834740 1 --2- 172.21.15.5:0/2892405682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a5d9c0 0x560525a66730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.438+0000 7f0892834740 1 -- 172.21.15.5:0/2892405682 >> 172.21.15.5:0/2892405682 conn(0x560525a580f0 msgr2=0x560525a56940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.438+0000 7f0892834740 1 -- 172.21.15.5:0/2892405682 shutdown_connections 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.438+0000 7f0892834740 1 -- 172.21.15.5:0/2892405682 wait complete. 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.438+0000 7f0892834740 1 -- start start 2024-04-22T13:28:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.438+0000 7f0892834740 1 Processor -- start 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 0x560525a6f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a60a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a66ec0 0x560525a60fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08881add90 con 0x560525a5d9c0 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b1220 con 0x560525a5e2e0 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ae400 con 0x560525a66ec0 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 0x560525a6f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 0x560525a6f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46398/0 (socket says 172.21.15.5:46398) 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 -- 172.21.15.5:0/2098343994 learned_addr learned my addr 172.21.15.5:0/2098343994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088e1e0640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a66ec0 0x560525a60fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 -- 172.21.15.5:0/2098343994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a66ec0 msgr2=0x560525a60fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d1de640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a60a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a66ec0 0x560525a60fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 -- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 msgr2=0x560525a60a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a60a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 -- 172.21.15.5:0/2098343994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560525a61800 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f0892834740 10 client.0 _ll_register_callbacks cb 0x7f088400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088e1e0640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a66ec0 0x560525a60fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d1de640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a60a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f088d9df640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 0x560525a6f190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f08840a5250 tx=0x7f088409bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4439a34b39be92 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.439+0000 7f08717fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f08717fa640 1 -- 172.21.15.5:0/2098343994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088408c060 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087802c880 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f08717fa640 1 -- 172.21.15.5:0/2098343994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f08840b1030 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f08717fa640 1 -- 172.21.15.5:0/2098343994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f088409c210 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f0892834740 10 client.16335 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=27}) -- 0x560525ae5620 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f08717fa640 1 -- 172.21.15.5:0/2098343994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0884097240 con 0x560525a5d9c0 2024-04-22T13:28:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f0892834740 10 client.16335 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f0892834740 10 client.16335 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.440+0000 7f0892834740 20 client.16335 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.441+0000 7f0892834740 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525ae5a20 0x560525ae7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.441+0000 7f0892834740 4 client.16335 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.5, "await": true}] 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.441+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.5, "await": true}) -- 0x5605259c9980 con 0x560525ae5a20 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.441+0000 7f088d1de640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525ae5a20 0x560525ae7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.441+0000 7f088d1de640 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525ae5a20 0x560525ae7e50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63d5e8bf522b6c6 server_cookie=5ec9804282e2828c in_seq=0 out_seq=0 2024-04-22T13:28:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.441+0000 7f08717fa640 10 client.16335 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f08717fa640 1 -- 172.21.15.5:0/2098343994 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5605259c9980 con 0x560525ae5a20 2024-04-22T13:28:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f08717fa640 10 client.16335 handle_command_reply: tid=0 2024-04-22T13:28:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 client.16335 shutdown 2024-04-22T13:28:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 msgr2=0x560525a6f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 0x560525a6f190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f08840a5250 tx=0x7f088409bd70 comp rx=0 tx=0).stop 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 shutdown_connections 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560525ae5a20 0x560525ae7e50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560525a66ec0 0x560525a60fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560525a5e2e0 0x560525a60a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 --2- 172.21.15.5:0/2098343994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525a5d9c0 0x560525a6f190 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 >> 172.21.15.5:0/2098343994 conn(0x560525a580f0 msgr2=0x560525a571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f08717fa640 0 client.16335 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 shutdown_connections 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.453+0000 7f0892834740 1 -- 172.21.15.5:0/2098343994 wait complete. 2024-04-22T13:28:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f0892834740 20 client.16335 trim_cache size 0 max 16384 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 10, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "ce13b8c6": { 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 10, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 86.5, 2024-04-22T13:28:40.452 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:40.453 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08740a3fd0 msgr2=0x7f08740a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08740a3fd0 0x7f08740a6490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0878006010 tx=0x7f0878005e40 comp rx=0 tx=0).stop 2024-04-22T13:28:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 msgr2=0x7f08881a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 0x7f08881a0660 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9f0 tx=0x7f088400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 shutdown_connections 2024-04-22T13:28:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08740a3fd0 0x7f08740a6490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f088810ec50 0x7f088819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0888069b70 0x7f08881a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 --2- 172.21.15.5:0/1451183541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888069250 0x7f08881a0660 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.454+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 >> 172.21.15.5:0/1451183541 conn(0x7f0888077c40 msgr2=0x7f08881101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.455+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 shutdown_connections 2024-04-22T13:28:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:40.455+0000 7f088fc43640 1 -- 172.21.15.5:0/1451183541 wait complete. 2024-04-22T13:28:40.466 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: ce13b8c6, quiesce took 0.7 seconds. Will release after: 26.5 2024-04-22T13:28:41.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:41 smithi049 ceph-mon[31379]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:41.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:41 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2650907201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:41.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:41 smithi107 ceph-mon[31859]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:41.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:41 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2650907201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:41.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:41 smithi005 ceph-mon[28006]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:41.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:41 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2650907201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:43.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:43 smithi049 ceph-mon[31379]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-22T13:28:43.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:43 smithi049 ceph-mon[31379]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T13:28:43.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:43 smithi107 ceph-mon[31859]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-22T13:28:43.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:43 smithi107 ceph-mon[31859]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T13:28:43.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:43 smithi005 ceph-mon[28006]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-22T13:28:43.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:43 smithi005 ceph-mon[28006]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T13:28:44.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:44 smithi049 ceph-mon[31379]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:44.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:44 smithi107 ceph-mon[31859]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:44.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:44 smithi005 ceph-mon[28006]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:46.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:46 smithi107 ceph-mon[31859]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-22T13:28:46.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:46 smithi005 ceph-mon[28006]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-22T13:28:47.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:46 smithi049 ceph-mon[31379]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-22T13:28:48.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:48 smithi107 ceph-mon[31859]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:48.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:48 smithi005 ceph-mon[28006]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:48.903 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 93% (11506/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T13:28:49.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:48 smithi049 ceph-mon[31379]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to 'd69f13d827952bb07bb9b7973bca14f97d764d6f'. 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-04-22T13:28:49.766 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:28:49.767 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at d69f13d8279 mds/cache: request_drop_foreign_locks shouldn't strip local xlocks 2024-04-22T13:28:49.772 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T13:28:49.793 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T13:28:49.795 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T13:28:49.795 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T13:28:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 -- 172.21.15.5:0/4016655314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990102b70 msgr2=0x7f4990106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 --2- 172.21.15.5:0/4016655314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990102b70 0x7f4990106bb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4980009900 tx=0x7f498002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 -- 172.21.15.5:0/4016655314 shutdown_connections 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 --2- 172.21.15.5:0/4016655314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990102b70 0x7f4990106bb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 --2- 172.21.15.5:0/4016655314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4990107290 0x7f499010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 --2- 172.21.15.5:0/4016655314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49901021c0 0x7f49901025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 -- 172.21.15.5:0/4016655314 >> 172.21.15.5:0/4016655314 conn(0x7f49900fda40 msgr2=0x7f49900ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 -- 172.21.15.5:0/4016655314 shutdown_connections 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.327+0000 7f499563f640 1 -- 172.21.15.5:0/4016655314 wait complete. 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 Processor -- start 2024-04-22T13:28:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 -- start start 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901021c0 0x7f499019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990107290 0x7f499019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 0x7f4990198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49901121d0 con 0x7f4990198590 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4990112050 con 0x7f49901021c0 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f499563f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4990112350 con 0x7f4990107290 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 0x7f4990198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 0x7f4990198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48152/0 (socket says 172.21.15.5:48152) 2024-04-22T13:28:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 -- 172.21.15.5:0/3375570306 learned_addr learned my addr 172.21.15.5:0/3375570306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498e7fc640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990107290 0x7f499019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498effd640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901021c0 0x7f499019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 -- 172.21.15.5:0/3375570306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990107290 msgr2=0x7f499019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990107290 0x7f499019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 -- 172.21.15.5:0/3375570306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901021c0 msgr2=0x7f499019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901021c0 0x7f499019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498f7fe640 1 -- 172.21.15.5:0/3375570306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49901990b0 con 0x7f4990198590 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.328+0000 7f498effd640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901021c0 0x7f499019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.329+0000 7f498f7fe640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 0x7f4990198a40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f498800eaa0 tx=0x7f498800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.329+0000 7f4977fff640 1 -- 172.21.15.5:0/3375570306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498800cdb0 con 0x7f4990198590 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.329+0000 7f4977fff640 1 -- 172.21.15.5:0/3375570306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4988005020 con 0x7f4990198590 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.329+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49901993a0 con 0x7f4990198590 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.329+0000 7f4977fff640 1 -- 172.21.15.5:0/3375570306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49880116d0 con 0x7f4990198590 2024-04-22T13:28:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.329+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49901a55a0 con 0x7f4990198590 2024-04-22T13:28:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f4977fff640 1 -- 172.21.15.5:0/3375570306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f49880051d0 con 0x7f4990198590 2024-04-22T13:28:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 Processor -- start 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 -- start start 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec828a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec828afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dec81ef8d0 con 0x55dec828ab60 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dec818ce80 con 0x55dec82984f0 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f499822b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dec81eb890 con 0x55dec828b8d0 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f4977fff640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f49780a3d10 0x7f49780a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f498effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec828afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f498effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec828afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48166/0 (socket says 172.21.15.5:48166) 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f498effd640 1 -- 172.21.15.5:0/83245079 learned_addr learned my addr 172.21.15.5:0/83245079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f498e7fc640 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f498f7fe640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f49780a3d10 0x7f49780a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.332+0000 7f4977fff640 1 -- 172.21.15.5:0/3375570306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f49880d0970 con 0x7f4990198590 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498e7fc640 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec828a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498effd640 1 -- 172.21.15.5:0/83245079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 msgr2=0x55dec828a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498effd640 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec828a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498effd640 1 -- 172.21.15.5:0/83245079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 msgr2=0x55dec829a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498effd640 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498effd640 1 -- 172.21.15.5:0/83245079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec81ae320 con 0x55dec828ab60 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498e7fc640 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec828a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498effd640 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec828afe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f498400bd40 tx=0x7f498400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e8973b1a9fa2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f498f7fe640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f49780a3d10 0x7f49780a61d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49880ce150 tx=0x7f4988067490 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f49757fa640 1 -- 172.21.15.5:0/83245079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984015070 con 0x55dec828ab60 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f49757fa640 1 -- 172.21.15.5:0/83245079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498400be30 con 0x55dec828ab60 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.333+0000 7f49757fa640 1 -- 172.21.15.5:0/83245079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984011940 con 0x55dec828ab60 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- 172.21.15.5:0/83245079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 msgr2=0x55dec828afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec828afe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f498400bd40 tx=0x7f498400f730 comp rx=0 tx=0).stop 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- 172.21.15.5:0/83245079 shutdown_connections 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec828afe0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- 172.21.15.5:0/83245079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec828a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- 172.21.15.5:0/83245079 >> 172.21.15.5:0/83245079 conn(0x55dec8251c90 msgr2=0x55dec8285850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- 172.21.15.5:0/83245079 shutdown_connections 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- 172.21.15.5:0/83245079 wait complete. 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- start start 2024-04-22T13:28:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 Processor -- start 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828ab60 0x55dec8297ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec8295270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 0x55dec8295880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dec829b220 con 0x55dec82984f0 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dec8177eb0 con 0x55dec828ab60 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dec829b3a0 con 0x55dec828b8d0 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.334+0000 7f499822b740 10 client.0 _ll_register_callbacks cb 0x55dec81a4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 0x55dec8295880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 0x55dec8295880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48174/0 (socket says 172.21.15.5:48174) 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 -- 172.21.15.5:0/1967259963 learned_addr learned my addr 172.21.15.5:0/1967259963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828ab60 0x55dec8297ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498effd640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec8295270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 -- 172.21.15.5:0/1967259963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 msgr2=0x55dec8295270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec8295270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 -- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828ab60 msgr2=0x55dec8297ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828ab60 0x55dec8297ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 -- 172.21.15.5:0/1967259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec7f02210 con 0x55dec82984f0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f498e7fc640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 0x55dec8295880 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f499006b610 tx=0x7f498002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a42584dea9b8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f4972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f4972ff5640 1 -- 172.21.15.5:0/1967259963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49800052d0 con 0x55dec82984f0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f4972ff5640 1 -- 172.21.15.5:0/1967259963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f4980005470 con 0x55dec82984f0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498400be30 con 0x55dec82984f0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f4972ff5640 1 -- 172.21.15.5:0/1967259963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4980054050 con 0x55dec82984f0 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f499822b740 10 client.16347 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dec7fce620 con 0x55dec82984f0 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.335+0000 7f4972ff5640 1 -- 172.21.15.5:0/1967259963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498400be30 con 0x55dec82984f0 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f4972ff5640 1 -- 172.21.15.5:0/1967259963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4980003710 con 0x55dec82984f0 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f499822b740 10 client.16347 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f499822b740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f499822b740 20 client.16347 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f499822b740 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec82a84d0 0x55dec82aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f499822b740 4 client.16347 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec8203880 con 0x55dec82a84d0 2024-04-22T13:28:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f498effd640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec82a84d0 0x55dec82aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f498effd640 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec82a84d0 0x55dec82aa8c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc644e97c9394e5f server_cookie=e392645231aab145 in_seq=0 out_seq=0 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.336+0000 7f4972ff5640 10 client.16347 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f4972ff5640 1 -- 172.21.15.5:0/1967259963 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dec8203880 con 0x55dec82a84d0 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f4972ff5640 10 client.16347 handle_command_reply: tid=0 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 client.16347 shutdown 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 msgr2=0x55dec8295880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 0x55dec8295880 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f499006b610 tx=0x7f498002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 shutdown_connections 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec82a84d0 0x55dec82aa8c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec82984f0 0x55dec8295880 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828b8d0 0x55dec8295270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f4972ff5640 0 client.16347 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 --2- 172.21.15.5:0/1967259963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828ab60 0x55dec8297ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 >> 172.21.15.5:0/1967259963 conn(0x55dec8251c90 msgr2=0x55dec82875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 shutdown_connections 2024-04-22T13:28:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 1 -- 172.21.15.5:0/1967259963 wait complete. 2024-04-22T13:28:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.337+0000 7f499822b740 20 client.16347 trim_cache size 0 max 16384 2024-04-22T13:28:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 Processor -- start 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 -- start start 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec82a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828b8d0 0x55dec82979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec82984f0 0x55dec8297ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ab990 con 0x55dec828ab60 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ad4d0 con 0x55dec828b8d0 2024-04-22T13:28:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f499822b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ac060 con 0x55dec82984f0 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f498e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec82a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f498e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec82a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48182/0 (socket says 172.21.15.5:48182) 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.352+0000 7f498e7fc640 1 -- 172.21.15.5:0/4117130277 learned_addr learned my addr 172.21.15.5:0/4117130277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498effd640 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828b8d0 0x55dec82979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498e7fc640 1 -- 172.21.15.5:0/4117130277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec82984f0 msgr2=0x55dec8297ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498e7fc640 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec82984f0 0x55dec8297ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498e7fc640 1 -- 172.21.15.5:0/4117130277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828b8d0 msgr2=0x55dec82979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498e7fc640 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828b8d0 0x55dec82979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498e7fc640 1 -- 172.21.15.5:0/4117130277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec829ad50 con 0x55dec828ab60 2024-04-22T13:28:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498effd640 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828b8d0 0x55dec82979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f498e7fc640 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec82a4120 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4980003c80 tx=0x7f4980005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e57e1623dd02a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f4974ff9640 1 -- 172.21.15.5:0/4117130277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498005d070 con 0x55dec828ab60 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f4974ff9640 1 -- 172.21.15.5:0/4117130277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4980046040 con 0x55dec828ab60 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.353+0000 7f4974ff9640 1 -- 172.21.15.5:0/4117130277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980051070 con 0x55dec828ab60 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- 172.21.15.5:0/4117130277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 msgr2=0x55dec82a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec82a4120 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4980003c80 tx=0x7f4980005b70 comp rx=0 tx=0).stop 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- 172.21.15.5:0/4117130277 shutdown_connections 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec82984f0 0x55dec8297ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec828b8d0 0x55dec82979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- 172.21.15.5:0/4117130277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828ab60 0x55dec82a4120 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- 172.21.15.5:0/4117130277 >> 172.21.15.5:0/4117130277 conn(0x55dec8251c90 msgr2=0x55dec8285380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- 172.21.15.5:0/4117130277 shutdown_connections 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- 172.21.15.5:0/4117130277 wait complete. 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- start start 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 Processor -- start 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828ab60 0x55dec8299840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 0x55dec8299d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ace00 con 0x55dec828b8d0 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49901ac730 con 0x55dec82984f0 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49901a9970 con 0x55dec828ab60 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.354+0000 7f499822b740 10 client.0 _ll_register_callbacks cb 0x7f4980009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 0x55dec8299d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 0x55dec8299d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48196/0 (socket says 172.21.15.5:48196) 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 -- 172.21.15.5:0/295132049 learned_addr learned my addr 172.21.15.5:0/295132049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498e7fc640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828ab60 0x55dec8299840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498e7fc640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 -- 172.21.15.5:0/295132049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828ab60 msgr2=0x55dec8299840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828ab60 0x55dec8299840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 -- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 msgr2=0x55dec829ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 -- 172.21.15.5:0/295132049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec82aac30 con 0x55dec828b8d0 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498e7fc640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828ab60 0x55dec8299840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f498effd640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 0x55dec8299d80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f49840114d0 tx=0x7f498400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb9f9c04e91283e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f4972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f4972ff5640 1 -- 172.21.15.5:0/295132049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984015040 con 0x55dec828b8d0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f4972ff5640 1 -- 172.21.15.5:0/295132049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f49840054d0 con 0x55dec828b8d0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4980046040 con 0x55dec828b8d0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f4972ff5640 1 -- 172.21.15.5:0/295132049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4984036050 con 0x55dec828b8d0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f499822b740 10 client.16353 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.355+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dec830ef60 con 0x55dec828b8d0 2024-04-22T13:28:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f4972ff5640 1 -- 172.21.15.5:0/295132049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4980046040 con 0x55dec828b8d0 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f4972ff5640 1 -- 172.21.15.5:0/295132049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4984004840 con 0x55dec828b8d0 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f499822b740 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f499822b740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f499822b740 20 client.16353 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f499822b740 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec830f450 0x55dec8311870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f499822b740 4 client.16353 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dec818f3d0 con 0x55dec830f450 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.356+0000 7f498e7fc640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec830f450 0x55dec8311870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.357+0000 7f4972ff5640 10 client.16353 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.357+0000 7f498e7fc640 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec830f450 0x55dec8311870 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38241350b08c43c server_cookie=b902b202dbe1c8d5 in_seq=0 out_seq=0 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.362+0000 7f4972ff5640 1 -- 172.21.15.5:0/295132049 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dec818f3d0 con 0x55dec830f450 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.362+0000 7f4972ff5640 10 client.16353 handle_command_reply: tid=0 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.362+0000 7f499822b740 1 client.16353 shutdown 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 msgr2=0x55dec8299d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 0x55dec8299d80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f49840114d0 tx=0x7f498400bf10 comp rx=0 tx=0).stop 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 shutdown_connections 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dec830f450 0x55dec8311870 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec828b8d0 0x55dec8299d80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dec82984f0 0x55dec829ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f4972ff5640 0 client.16353 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 --2- 172.21.15.5:0/295132049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec828ab60 0x55dec8299840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 >> 172.21.15.5:0/295132049 conn(0x55dec8251c90 msgr2=0x55dec8284040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 shutdown_connections 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 1 -- 172.21.15.5:0/295132049 wait complete. 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499822b740 20 client.16353 trim_cache size 0 max 16384 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:28:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:28:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f49780a3d10 msgr2=0x7f49780a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f49780a3d10 0x7f49780a61d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49880ce150 tx=0x7f4988067490 comp rx=0 tx=0).stop 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 msgr2=0x7f4990198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 0x7f4990198a40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f498800eaa0 tx=0x7f498800ef60 comp rx=0 tx=0).stop 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 shutdown_connections 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4990198590 0x7f4990198a40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f49780a3d10 0x7f49780a61d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990107290 0x7f499019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 --2- 172.21.15.5:0/3375570306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49901021c0 0x7f499019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.363+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 >> 172.21.15.5:0/3375570306 conn(0x7f49900fda40 msgr2=0x7f49900fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.364+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 shutdown_connections 2024-04-22T13:28:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:28:50.364+0000 7f499563f640 1 -- 172.21.15.5:0/3375570306 wait complete. 2024-04-22T13:28:50.370 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T13:28:50.430 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T13:28:50.471 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T13:28:50.498 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T13:28:50.499 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T13:28:50.513 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T13:28:50.515 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:28:50.516 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T13:28:50.572 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-04-22T13:28:50.573 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-04-22T13:28:50.573 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d69f13d827952bb07bb9b7973bca14f97d764d6f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-22T13:28:50.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:50 smithi005 ceph-mon[28006]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:51.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:50 smithi049 ceph-mon[31379]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:51.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:50 smithi107 ceph-mon[31859]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-22T13:28:52.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:52.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 -- 172.21.15.5:0/384382800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 msgr2=0x7fbf7010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 --2- 172.21.15.5:0/384382800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf7010bfc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009900 tx=0x7fbf6002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 -- 172.21.15.5:0/384382800 shutdown_connections 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 --2- 172.21.15.5:0/384382800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf7010bfc0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 --2- 172.21.15.5:0/384382800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf7010c590 0x7fbf7010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 --2- 172.21.15.5:0/384382800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf701031c0 0x7fbf701035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 -- 172.21.15.5:0/384382800 >> 172.21.15.5:0/384382800 conn(0x7fbf70077f40 msgr2=0x7fbf70078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 -- 172.21.15.5:0/384382800 shutdown_connections 2024-04-22T13:28:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 -- 172.21.15.5:0/384382800 wait complete. 2024-04-22T13:28:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 Processor -- start 2024-04-22T13:28:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.650+0000 7fbf76552640 1 -- start start 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf76552640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf701031c0 0x7fbf701a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf76552640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf701a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf76552640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf7010c590 0x7fbf7019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf76552640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701121d0 con 0x7fbf70103ae0 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf76552640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70112050 con 0x7fbf701031c0 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf76552640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70112350 con 0x7fbf7010c590 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf701a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf701a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48212/0 (socket says 172.21.15.5:48212) 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3473253799 learned_addr learned my addr 172.21.15.5:0/3473253799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf74aef640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf7010c590 0x7fbf7019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf701031c0 0x7fbf701a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3473253799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf7010c590 msgr2=0x7fbf7019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf7010c590 0x7fbf7019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3473253799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf701031c0 msgr2=0x7fbf701a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf701031c0 0x7fbf701a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3473253799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7019aff0 con 0x7fbf70103ae0 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf74aef640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf7010c590 0x7fbf7019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf701031c0 0x7fbf701a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.651+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf701a0bb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005940 tx=0x7fbf6002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.652+0000 7fbf6d7fa640 1 -- 172.21.15.5:0/3473253799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60043020 con 0x7fbf70103ae0 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.652+0000 7fbf6d7fa640 1 -- 172.21.15.5:0/3473253799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6002dc90 con 0x7fbf70103ae0 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.652+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7019b280 con 0x7fbf70103ae0 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.652+0000 7fbf6d7fa640 1 -- 172.21.15.5:0/3473253799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003ab0 con 0x7fbf70103ae0 2024-04-22T13:28:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.652+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf701a75f0 con 0x7fbf70103ae0 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf6d7fa640 1 -- 172.21.15.5:0/3473253799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbf60046020 con 0x7fbf70103ae0 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 Processor -- start 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 -- start start 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597580 0x5622b8597960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 0x5622b85a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5622b84fc1e0 con 0x5622b85a0a80 2024-04-22T13:28:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf6d7fa640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf5c0a3da0 0x7fbf5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5622b8499390 con 0x5622b8597ea0 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.654+0000 7fbf79226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622b84f81a0 con 0x5622b8597580 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 0x5622b85a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf74aef640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf5c0a3da0 0x7fbf5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 0x5622b85a2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48224/0 (socket says 172.21.15.5:48224) 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3379037699 learned_addr learned my addr 172.21.15.5:0/3379037699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6d7fa640 1 -- 172.21.15.5:0/3473253799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbf600f2860 con 0x7fbf70103ae0 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf74aef640 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597580 0x5622b8597960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3379037699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597580 msgr2=0x5622b8597960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597580 0x5622b8597960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3379037699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 msgr2=0x5622b85a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/3379037699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b82d87d0 con 0x5622b85a0a80 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf74aef640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf5c0a3da0 0x7fbf5c0a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf68005c40 tx=0x7fbf68009d50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 0x5622b85a2e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf600915e0 tx=0x7fbf60090560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abf3e3e7824536e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf667fc640 1 -- 172.21.15.5:0/3379037699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6003b2b0 con 0x5622b85a0a80 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf667fc640 1 -- 172.21.15.5:0/3379037699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6002de40 con 0x5622b85a0a80 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.655+0000 7fbf667fc640 1 -- 172.21.15.5:0/3379037699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600c7670 con 0x5622b85a0a80 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 -- 172.21.15.5:0/3379037699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 msgr2=0x5622b85a2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 0x5622b85a2e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf600915e0 tx=0x7fbf60090560 comp rx=0 tx=0).stop 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 -- 172.21.15.5:0/3379037699 shutdown_connections 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b85a0a80 0x5622b85a2e70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 --2- 172.21.15.5:0/3379037699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597580 0x5622b8597960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 -- 172.21.15.5:0/3379037699 >> 172.21.15.5:0/3379037699 conn(0x5622b8591cb0 msgr2=0x5622b8592090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 -- 172.21.15.5:0/3379037699 shutdown_connections 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 -- 172.21.15.5:0/3379037699 wait complete. 2024-04-22T13:28:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.656+0000 7fbf79226740 1 -- start start 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 Processor -- start 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b859db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b859e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b85a0a80 0x5622b859e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5622b85a3630 con 0x5622b8597580 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5622b84b52f0 con 0x5622b8597ea0 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622b85a37b0 con 0x5622b85a0a80 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf79226740 10 client.0 _ll_register_callbacks cb 0x5622b84c2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b859db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b859db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48238/0 (socket says 172.21.15.5:48238) 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 -- 172.21.15.5:0/1038366764 learned_addr learned my addr 172.21.15.5:0/1038366764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf74aef640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b85a0a80 0x5622b859e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b859e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 -- 172.21.15.5:0/1038366764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b85a0a80 msgr2=0x5622b859e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b85a0a80 0x5622b859e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 -- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 msgr2=0x5622b859e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b859e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6ffff640 1 -- 172.21.15.5:0/1038366764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b820e3f0 con 0x5622b8597580 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.657+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b859e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b859db50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009fc0 tx=0x7fbf580117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160ff14f8917c7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/1038366764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58012860 con 0x5622b8597580 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/1038366764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbf58012a00 con 0x5622b8597580 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5622b8509c00 con 0x5622b8597580 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/1038366764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf58012c00 con 0x5622b8597580 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf79226740 10 client.16365 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622b81ce0f0 con 0x5622b8597580 2024-04-22T13:28:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/1038366764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5622b8509c00 con 0x5622b8597580 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/1038366764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf5800f040 con 0x5622b8597580 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf79226740 10 client.16365 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.658+0000 7fbf79226740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf79226740 20 client.16365 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf79226740 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85b08b0 0x5622b85b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf79226740 4 client.16365 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622b8503a70 con 0x5622b85b08b0 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85b08b0 0x5622b85b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85b08b0 0x5622b85b2d50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f4a565555dd7af server_cookie=91b21d2042666de3 in_seq=0 out_seq=0 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.659+0000 7fbf4f7fe640 10 client.16365 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/1038366764 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622b8503a70 con 0x5622b85b08b0 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf4f7fe640 10 client.16365 handle_command_reply: tid=0 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 client.16365 shutdown 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 msgr2=0x5622b859db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b859db50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009fc0 tx=0x7fbf580117b0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 shutdown_connections 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85b08b0 0x5622b85b2d50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b85a0a80 0x5622b859e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b859e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf4f7fe640 0 client.16365 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 --2- 172.21.15.5:0/1038366764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b859db50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 >> 172.21.15.5:0/1038366764 conn(0x5622b8591cb0 msgr2=0x5622b8593a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 shutdown_connections 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 1 -- 172.21.15.5:0/1038366764 wait complete. 2024-04-22T13:28:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.660+0000 7fbf79226740 20 client.16365 trim_cache size 0 max 16384 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.678+0000 7fbf79226740 1 Processor -- start 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.678+0000 7fbf79226740 1 -- start start 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf79226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 0x5622b85a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf79226740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597580 0x5622b85ac380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf79226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597ea0 0x5622b8592090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf79226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701afce0 con 0x5622b8591cb0 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf79226740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701b1820 con 0x5622b8597580 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf79226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701b03b0 con 0x5622b8597ea0 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 0x5622b85a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 0x5622b85a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48240/0 (socket says 172.21.15.5:48240) 2024-04-22T13:28:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 -- 172.21.15.5:0/2055518711 learned_addr learned my addr 172.21.15.5:0/2055518711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597580 0x5622b85ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf74aef640 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597ea0 0x5622b8592090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 -- 172.21.15.5:0/2055518711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597ea0 msgr2=0x5622b8592090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597ea0 0x5622b8592090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 -- 172.21.15.5:0/2055518711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597580 msgr2=0x5622b85ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597580 0x5622b85ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 -- 172.21.15.5:0/2055518711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b8597cd0 con 0x5622b8591cb0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 0x5622b85a0280 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbf58001eb0 tx=0x7fbf58005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca751baa732192a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf65ffb640 1 -- 172.21.15.5:0/2055518711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58011cb0 con 0x5622b8591cb0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf65ffb640 1 -- 172.21.15.5:0/2055518711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf58012da0 con 0x5622b8591cb0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.679+0000 7fbf65ffb640 1 -- 172.21.15.5:0/2055518711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004880 con 0x5622b8591cb0 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 -- 172.21.15.5:0/2055518711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 msgr2=0x5622b85a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 0x5622b85a0280 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbf58001eb0 tx=0x7fbf58005fb0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 -- 172.21.15.5:0/2055518711 shutdown_connections 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8597ea0 0x5622b8592090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597580 0x5622b85ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 --2- 172.21.15.5:0/2055518711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8591cb0 0x5622b85a0280 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 -- 172.21.15.5:0/2055518711 >> 172.21.15.5:0/2055518711 conn(0x5622b8599520 msgr2=0x5622b85904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 -- 172.21.15.5:0/2055518711 shutdown_connections 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 -- 172.21.15.5:0/2055518711 wait complete. 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.680+0000 7fbf79226740 1 -- start start 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 Processor -- start 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8591cb0 0x5622b85a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b85a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701b1150 con 0x5622b8597580 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701b0a80 con 0x5622b8597ea0 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701adcc0 con 0x5622b8591cb0 2024-04-22T13:28:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf79226740 10 client.0 _ll_register_callbacks cb 0x7fbf580097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b85a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b85a3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48250/0 (socket says 172.21.15.5:48250) 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/745222298 learned_addr learned my addr 172.21.15.5:0/745222298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8591cb0 0x5622b85a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/745222298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8591cb0 msgr2=0x5622b85a8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8591cb0 0x5622b85a8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf74aef640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 msgr2=0x5622b85a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 -- 172.21.15.5:0/745222298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b85a4d20 con 0x5622b8597580 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf6f7fe640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b85a3fe0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbf600bb3d0 tx=0x7fbf6002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6181615766a94d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.681+0000 7fbf4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/745222298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6008d070 con 0x5622b8597580 2024-04-22T13:28:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/745222298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbf60003c50 con 0x5622b8597580 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf58012da0 con 0x5622b8597580 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/745222298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf600bf100 con 0x5622b8597580 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622b8616f70 con 0x5622b8597580 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/745222298 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf58012da0 con 0x5622b8597580 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/745222298 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf60090030 con 0x5622b8597580 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 20 client.16371 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85a0a80 0x5622b86195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 4 client.16371 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622b847f590 con 0x5622b85a0a80 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.682+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85a0a80 0x5622b86195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf4f7fe640 10 client.16371 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf6ffff640 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85a0a80 0x5622b86195d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c403d4891ee0e427 server_cookie=ee618abc523456a5 in_seq=0 out_seq=0 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf4f7fe640 1 -- 172.21.15.5:0/745222298 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622b847f590 con 0x5622b85a0a80 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf4f7fe640 10 client.16371 handle_command_reply: tid=0 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 client.16371 shutdown 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 msgr2=0x5622b85a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b85a3fe0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbf600bb3d0 tx=0x7fbf6002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 shutdown_connections 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5622b85a0a80 0x5622b86195d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5622b8597580 0x5622b85a3fe0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5622b8597ea0 0x5622b85a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf4f7fe640 0 client.16371 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 --2- 172.21.15.5:0/745222298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622b8591cb0 0x5622b85a8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 >> 172.21.15.5:0/745222298 conn(0x5622b8599520 msgr2=0x5622b8590d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 shutdown_connections 2024-04-22T13:28:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.683+0000 7fbf79226740 1 -- 172.21.15.5:0/745222298 wait complete. 2024-04-22T13:28:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf79226740 20 client.16371 trim_cache size 0 max 16384 2024-04-22T13:28:52.686 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:52.686 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:28:52.686 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:28:52.686 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf5c0a3da0 msgr2=0x7fbf5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf5c0a3da0 0x7fbf5c0a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf68005c40 tx=0x7fbf68009d50 comp rx=0 tx=0).stop 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 msgr2=0x7fbf701a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf701a0bb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005940 tx=0x7fbf6002d890 comp rx=0 tx=0).stop 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 shutdown_connections 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70103ae0 0x7fbf701a0bb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf5c0a3da0 0x7fbf5c0a6260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf7010c590 0x7fbf7019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 --2- 172.21.15.5:0/3473253799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf701031c0 0x7fbf701a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 >> 172.21.15.5:0/3473253799 conn(0x7fbf70077f40 msgr2=0x7fbf701021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 shutdown_connections 2024-04-22T13:28:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.684+0000 7fbf76552640 1 -- 172.21.15.5:0/3473253799 wait complete. 2024-04-22T13:28:52.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:28:52.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- 172.21.15.5:0/2966502947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc1072c0 msgr2=0x7f8ffc10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- 172.21.15.5:0/2966502947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc1072c0 0x7f8ffc10ee70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009ca0 tx=0x7f8ff402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- 172.21.15.5:0/2966502947 shutdown_connections 2024-04-22T13:28:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- 172.21.15.5:0/2966502947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc1072c0 0x7f8ffc10ee70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- 172.21.15.5:0/2966502947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ffc069f10 0x7f8ffc106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- 172.21.15.5:0/2966502947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 0x7f8ffc069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- 172.21.15.5:0/2966502947 >> 172.21.15.5:0/2966502947 conn(0x7f8ffc0771d0 msgr2=0x7f8ffc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- 172.21.15.5:0/2966502947 shutdown_connections 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- 172.21.15.5:0/2966502947 wait complete. 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 Processor -- start 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- start start 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 0x7f8ffc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ffc1072c0 0x7f8ffc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 0x7f8ffc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1121c0 con 0x7f8ffc19cf60 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc112040 con 0x7f8ffc069560 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.798+0000 7f90040e4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc112340 con 0x7f8ffc1072c0 2024-04-22T13:28:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 0x7f8ffc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 0x7f8ffc19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48266/0 (socket says 172.21.15.5:48266) 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 -- 172.21.15.5:0/1131334438 learned_addr learned my addr 172.21.15.5:0/1131334438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f900167f640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ffc1072c0 0x7f8ffc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9001e80640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 0x7f8ffc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 -- 172.21.15.5:0/1131334438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ffc1072c0 msgr2=0x7f8ffc198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ffc1072c0 0x7f8ffc198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 -- 172.21.15.5:0/1131334438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 msgr2=0x7f8ffc1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 0x7f8ffc1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 -- 172.21.15.5:0/1131334438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc19da50 con 0x7f8ffc19cf60 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9001e80640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 0x7f8ffc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f9002681640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 0x7f8ffc19d410 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005e30 tx=0x7f8ff4032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4041020 con 0x7f8ffc19cf60 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff4032970 con 0x7f8ffc19cf60 2024-04-22T13:28:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc19dce0 con 0x7f8ffc19cf60 2024-04-22T13:28:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4039040 con 0x7f8ffc19cf60 2024-04-22T13:28:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.799+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a5500 con 0x7f8ffc19cf60 2024-04-22T13:28:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.800+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc068e00 con 0x7f8ffc19cf60 2024-04-22T13:28:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.801+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8ff4044020 con 0x7f8ffc19cf60 2024-04-22T13:28:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.802+0000 7f8ff2ffd640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8fe80a3da0 0x7f8fe80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.802+0000 7f9001e80640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8fe80a3da0 0x7f8fe80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.802+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8ff40f2800 con 0x7f8ffc19cf60 2024-04-22T13:28:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.803+0000 7f9001e80640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8fe80a3da0 0x7f8fe80a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8005d00 tx=0x7f8ff8005c70 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:52.806+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ff404b020 con 0x7f8ffc19cf60 2024-04-22T13:28:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:52 smithi005 ceph-mon[28006]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-22T13:28:53.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:52 smithi049 ceph-mon[31379]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-22T13:28:53.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.030+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ffc1999f0 con 0x7f8ffc19cf60 2024-04-22T13:28:53.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.031+0000 7f8ff2ffd640 1 -- 172.21.15.5:0/1131334438 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f8ffc1999f0 con 0x7f8ffc19cf60 2024-04-22T13:28:53.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:28:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:28:53.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:28:53.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8fe80a3da0 msgr2=0x7f8fe80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8fe80a3da0 0x7f8fe80a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8005d00 tx=0x7f8ff8005c70 comp rx=0 tx=0).stop 2024-04-22T13:28:53.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 msgr2=0x7f8ffc19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 0x7f8ffc19d410 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005e30 tx=0x7f8ff4032420 comp rx=0 tx=0).stop 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 shutdown_connections 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc19cf60 0x7f8ffc19d410 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8fe80a3da0 0x7f8fe80a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ffc1072c0 0x7f8ffc198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 --2- 172.21.15.5:0/1131334438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ffc069560 0x7f8ffc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 >> 172.21.15.5:0/1131334438 conn(0x7f8ffc0771d0 msgr2=0x7f8ffc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 shutdown_connections 2024-04-22T13:28:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.033+0000 7f90040e4640 1 -- 172.21.15.5:0/1131334438 wait complete. 2024-04-22T13:28:53.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:52 smithi107 ceph-mon[31859]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-22T13:28:53.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 -- 172.21.15.5:0/2862028014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f856810c500 msgr2=0x7f856810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 --2- 172.21.15.5:0/2862028014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f856810c500 0x7f856810e9a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ca0 tx=0x7f856002c180 comp rx=0 tx=0).stop 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 -- 172.21.15.5:0/2862028014 shutdown_connections 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 --2- 172.21.15.5:0/2862028014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f856810c500 0x7f856810e9a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 --2- 172.21.15.5:0/2862028014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8568103ae0 0x7f856810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 --2- 172.21.15.5:0/2862028014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85681031c0 0x7f85681035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 -- 172.21.15.5:0/2862028014 >> 172.21.15.5:0/2862028014 conn(0x7f8568078000 msgr2=0x7f8568078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 -- 172.21.15.5:0/2862028014 shutdown_connections 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.148+0000 7f856d8f2640 1 -- 172.21.15.5:0/2862028014 wait complete. 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 Processor -- start 2024-04-22T13:28:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 -- start start 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85681031c0 0x7f8568198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 0x7f8568198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f856810c500 0x7f856819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8568112190 con 0x7f8568103ae0 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8568112010 con 0x7f856810c500 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f856d8f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8568112310 con 0x7f85681031c0 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f855ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 0x7f8568198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f855ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 0x7f8568198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48280/0 (socket says 172.21.15.5:48280) 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f855ffff640 1 -- 172.21.15.5:0/1753883112 learned_addr learned my addr 172.21.15.5:0/1753883112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f8566ffd640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85681031c0 0x7f8568198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f855ffff640 1 -- 172.21.15.5:0/1753883112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85681031c0 msgr2=0x7f8568198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.149+0000 7f855ffff640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85681031c0 0x7f8568198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f855ffff640 1 -- 172.21.15.5:0/1753883112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f856810c500 msgr2=0x7f856819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f855ffff640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f856810c500 0x7f856819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f855ffff640 1 -- 172.21.15.5:0/1753883112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856819d660 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f855ffff640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 0x7f8568198bb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f855000e9f0 tx=0x7f855000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f856c8f0640 1 -- 172.21.15.5:0/1753883112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000cd80 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856819d950 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f856c8f0640 1 -- 172.21.15.5:0/1753883112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85500108e0 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f856c8f0640 1 -- 172.21.15.5:0/1753883112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550018980 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.150+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a51e0 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f856c8f0640 1 -- 172.21.15.5:0/1753883112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8550010430 con 0x7f8568103ae0 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 Processor -- start 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 -- start start 2024-04-22T13:28:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 0x5571119048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x55711190d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55711190da00 0x55711190fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571118691f0 con 0x5571119044d0 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571118064b0 con 0x557111904e80 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85705c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571118651b0 con 0x55711190da00 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 0x5571119048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 0x5571119048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48284/0 (socket says 172.21.15.5:48284) 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f85677fe640 1 -- 172.21.15.5:0/82905930 learned_addr learned my addr 172.21.15.5:0/82905930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.153+0000 7f8566ffd640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x55711190d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f855ffff640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55711190da00 0x55711190fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 -- 172.21.15.5:0/82905930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55711190da00 msgr2=0x55711190fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f856c8f0640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85540a3d20 0x7f85540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55711190da00 0x55711190fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 -- 172.21.15.5:0/82905930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 msgr2=0x55711190d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x55711190d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 -- 172.21.15.5:0/82905930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571117f81b0 con 0x5571119044d0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f855ffff640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55711190da00 0x55711190fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f8566ffd640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x55711190d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f856c8f0640 1 -- 172.21.15.5:0/1753883112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f85500d0ae0 con 0x7f8568103ae0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85540a3d20 0x7f85540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f85677fe640 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 0x5571119048b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8560009dc0 tx=0x7f85600324a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0b87ca15d24082 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f855dffb640 1 -- 172.21.15.5:0/82905930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560043030 con 0x5571119044d0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f855dffb640 1 -- 172.21.15.5:0/82905930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85600046b0 con 0x5571119044d0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.154+0000 7f855dffb640 1 -- 172.21.15.5:0/82905930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560039040 con 0x5571119044d0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 -- 172.21.15.5:0/82905930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 msgr2=0x5571119048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85677fe640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85540a3d20 0x7f85540a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8560009a60 tx=0x7f856004a000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 0x5571119048b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8560009dc0 tx=0x7f85600324a0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 -- 172.21.15.5:0/82905930 shutdown_connections 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55711190da00 0x55711190fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x55711190d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 --2- 172.21.15.5:0/82905930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571119044d0 0x5571119048b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 -- 172.21.15.5:0/82905930 >> 172.21.15.5:0/82905930 conn(0x5571118feba0 msgr2=0x5571118fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 -- 172.21.15.5:0/82905930 shutdown_connections 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 -- 172.21.15.5:0/82905930 wait complete. 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.155+0000 7f85705c4740 1 -- start start 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 Processor -- start 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x55711190d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x55711190a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x55711190ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557111910970 con 0x55711190da00 2024-04-22T13:28:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557111822500 con 0x5571119044d0 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557111910af0 con 0x557111904e80 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85705c4740 10 client.0 _ll_register_callbacks cb 0x5571117f0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x55711190ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x55711190ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48294/0 (socket says 172.21.15.5:48294) 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 -- 172.21.15.5:0/3040615532 learned_addr learned my addr 172.21.15.5:0/3040615532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f8566ffd640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x55711190d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f855ffff640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x55711190a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 -- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 msgr2=0x55711190a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x55711190a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 -- 172.21.15.5:0/3040615532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 msgr2=0x55711190d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x55711190d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 -- 172.21.15.5:0/3040615532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711157b3f0 con 0x55711190da00 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f85677fe640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x55711190ac10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8560032870 tx=0x7f8560004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6336098754bc442 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.156+0000 7f8546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f8546ffd640 1 -- 172.21.15.5:0/3040615532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560056060 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f8546ffd640 1 -- 172.21.15.5:0/3040615532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8560032a50 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85600046b0 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f8546ffd640 1 -- 172.21.15.5:0/3040615532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f856004d070 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 10 client.16389 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571116457d0 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f8546ffd640 1 -- 172.21.15.5:0/3040615532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85600046b0 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f8546ffd640 1 -- 172.21.15.5:0/3040615532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f856003c030 con 0x55711190da00 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 10 client.16389 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 20 client.16389 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55711191da20 0x55711191fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 4 client.16389 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557111808930 con 0x55711191da20 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.157+0000 7f8566ffd640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55711191da20 0x55711191fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.158+0000 7f8546ffd640 10 client.16389 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.158+0000 7f8566ffd640 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55711191da20 0x55711191fe10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d1b9ebe2c37ce7d server_cookie=e40dd2edfef27edc in_seq=0 out_seq=0 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f8546ffd640 1 -- 172.21.15.5:0/3040615532 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557111808930 con 0x55711191da20 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f8546ffd640 10 client.16389 handle_command_reply: tid=0 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 client.16389 shutdown 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 msgr2=0x55711190ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x55711190ac10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8560032870 tx=0x7f8560004620 comp rx=0 tx=0).stop 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 shutdown_connections 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55711191da20 0x55711191fe10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x55711190ac10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x55711190a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f8546ffd640 0 client.16389 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 --2- 172.21.15.5:0/3040615532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x55711190d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 >> 172.21.15.5:0/3040615532 conn(0x5571118feba0 msgr2=0x5571119008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 shutdown_connections 2024-04-22T13:28:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 1 -- 172.21.15.5:0/3040615532 wait complete. 2024-04-22T13:28:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.159+0000 7f85705c4740 20 client.16389 trim_cache size 0 max 16384 2024-04-22T13:28:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.172+0000 7f85705c4740 1 Processor -- start 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.172+0000 7f85705c4740 1 -- start start 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85705c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571119044d0 0x557111919860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85705c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x5571118fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85705c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111916150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85705c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85681ada90 con 0x55711190da00 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85705c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85681af5d0 con 0x557111904e80 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85705c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85681ae160 con 0x5571119044d0 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111916150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111916150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48296/0 (socket says 172.21.15.5:48296) 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 -- 172.21.15.5:0/3566580575 learned_addr learned my addr 172.21.15.5:0/3566580575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f855ffff640 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x5571118fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f8566ffd640 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571119044d0 0x557111919860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 -- 172.21.15.5:0/3566580575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571119044d0 msgr2=0x557111919860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571119044d0 0x557111919860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 -- 172.21.15.5:0/3566580575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 msgr2=0x5571118fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x5571118fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 -- 172.21.15.5:0/3566580575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571118fe220 con 0x55711190da00 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f8566ffd640 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571119044d0 0x557111919860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f85677fe640 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111916150 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8560009bf0 tx=0x7f85600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd87acca76d362f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f855d7fa640 1 -- 172.21.15.5:0/3566580575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560056040 con 0x55711190da00 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f855d7fa640 1 -- 172.21.15.5:0/3566580575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8560032560 con 0x55711190da00 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.173+0000 7f855d7fa640 1 -- 172.21.15.5:0/3566580575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560004780 con 0x55711190da00 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 -- 172.21.15.5:0/3566580575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 msgr2=0x557111916150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111916150 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8560009bf0 tx=0x7f85600095c0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 -- 172.21.15.5:0/3566580575 shutdown_connections 2024-04-22T13:28:53.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111916150 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557111904e80 0x5571118fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 --2- 172.21.15.5:0/3566580575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571119044d0 0x557111919860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 -- 172.21.15.5:0/3566580575 >> 172.21.15.5:0/3566580575 conn(0x5571118feba0 msgr2=0x5571119096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 -- 172.21.15.5:0/3566580575 shutdown_connections 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 -- 172.21.15.5:0/3566580575 wait complete. 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.174+0000 7f85705c4740 1 -- start start 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 Processor -- start 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x557111906900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x557111906e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111907380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85681aef00 con 0x55711190da00 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85681ae830 con 0x5571119044d0 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85681aba70 con 0x557111904e80 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85705c4740 10 client.0 _ll_register_callbacks cb 0x7f8558005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111907380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111907380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48308/0 (socket says 172.21.15.5:48308) 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 -- 172.21.15.5:0/2567882201 learned_addr learned my addr 172.21.15.5:0/2567882201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f8566ffd640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x557111906900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f855ffff640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x557111906e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 -- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 msgr2=0x557111906e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x557111906e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 -- 172.21.15.5:0/2567882201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 msgr2=0x557111906900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x557111906900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 -- 172.21.15.5:0/2567882201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557111907a70 con 0x55711190da00 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f855ffff640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x557111906e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f85677fe640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111907380 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ca0 tx=0x7f85600048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502d72c66ee019b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.175+0000 7f8546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f8546ffd640 1 -- 172.21.15.5:0/2567882201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560056070 con 0x55711190da00 2024-04-22T13:28:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f8546ffd640 1 -- 172.21.15.5:0/2567882201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8560032920 con 0x55711190da00 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8560032560 con 0x55711190da00 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f8546ffd640 1 -- 172.21.15.5:0/2567882201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8560072050 con 0x55711190da00 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557111988060 con 0x55711190da00 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f8546ffd640 1 -- 172.21.15.5:0/2567882201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8560032560 con 0x55711190da00 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f8546ffd640 1 -- 172.21.15.5:0/2567882201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f856003c070 con 0x55711190da00 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 10 client.16395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 20 client.16395 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5571119886a0 0x55711198aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 4 client.16395 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557111876c10 con 0x5571119886a0 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.176+0000 7f8566ffd640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5571119886a0 0x55711198aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f8546ffd640 10 client.16395 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f8566ffd640 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5571119886a0 0x55711198aac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a91d2051c9f7c6b7 server_cookie=21d588570c7e9427 in_seq=0 out_seq=0 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f8546ffd640 1 -- 172.21.15.5:0/2567882201 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557111876c10 con 0x5571119886a0 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f8546ffd640 10 client.16395 handle_command_reply: tid=0 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 client.16395 shutdown 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 msgr2=0x557111907380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111907380 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ca0 tx=0x7f85600048a0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 shutdown_connections 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5571119886a0 0x55711198aac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55711190da00 0x557111907380 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f8546ffd640 0 client.16395 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557111904e80 0x557111906e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 --2- 172.21.15.5:0/2567882201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5571119044d0 0x557111906900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 >> 172.21.15.5:0/2567882201 conn(0x5571118feba0 msgr2=0x557111908e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 shutdown_connections 2024-04-22T13:28:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.177+0000 7f85705c4740 1 -- 172.21.15.5:0/2567882201 wait complete. 2024-04-22T13:28:53.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f85705c4740 20 client.16395 trim_cache size 0 max 16384 2024-04-22T13:28:53.180 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85540a3d20 msgr2=0x7f85540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85540a3d20 0x7f85540a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8560009a60 tx=0x7f856004a000 comp rx=0 tx=0).stop 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 msgr2=0x7f8568198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 0x7f8568198bb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f855000e9f0 tx=0x7f855000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 shutdown_connections 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8568103ae0 0x7f8568198bb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f85540a3d20 0x7f85540a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f856810c500 0x7f856819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 --2- 172.21.15.5:0/1753883112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85681031c0 0x7f8568198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 >> 172.21.15.5:0/1753883112 conn(0x7f8568078000 msgr2=0x7f85680fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 shutdown_connections 2024-04-22T13:28:53.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.178+0000 7f856d8f2640 1 -- 172.21.15.5:0/1753883112 wait complete. 2024-04-22T13:28:53.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 -- 172.21.15.5:0/3083665656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c100d40 msgr2=0x7f378c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 --2- 172.21.15.5:0/3083665656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c100d40 0x7f378c101120 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3788009900 tx=0x7f378802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 -- 172.21.15.5:0/3083665656 shutdown_connections 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 --2- 172.21.15.5:0/3083665656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c10e8f0 0x7f378c110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 --2- 172.21.15.5:0/3083665656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 0x7f378c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 --2- 172.21.15.5:0/3083665656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c100d40 0x7f378c101120 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.292+0000 7f37935a4640 1 -- 172.21.15.5:0/3083665656 >> 172.21.15.5:0/3083665656 conn(0x7f378c078000 msgr2=0x7f378c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 -- 172.21.15.5:0/3083665656 shutdown_connections 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 -- 172.21.15.5:0/3083665656 wait complete. 2024-04-22T13:28:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 Processor -- start 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 -- start start 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c100d40 0x7f378c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 0x7f378c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 0x7f378c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f378c114410 con 0x7f378c10e8f0 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f378c114290 con 0x7f378c100d40 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.293+0000 7f37935a4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f378c114590 con 0x7f378c101660 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 0x7f378c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791340640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c100d40 0x7f378c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 0x7f378c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48314/0 (socket says 172.21.15.5:48314) 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791340640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c100d40 0x7f378c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:53662/0 (socket says 172.21.15.5:53662) 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 -- 172.21.15.5:0/973456744 learned_addr learned my addr 172.21.15.5:0/973456744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3790b3f640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 0x7f378c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 -- 172.21.15.5:0/973456744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 msgr2=0x7f378c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 0x7f378c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 -- 172.21.15.5:0/973456744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c100d40 msgr2=0x7f378c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c100d40 0x7f378c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 -- 172.21.15.5:0/973456744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c19b000 con 0x7f378c10e8f0 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3790b3f640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 0x7f378c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f3791b41640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 0x7f378c19a830 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f378400e920 tx=0x7f378400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f37827fc640 1 -- 172.21.15.5:0/973456744 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004d20 con 0x7f378c10e8f0 2024-04-22T13:28:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f37827fc640 1 -- 172.21.15.5:0/973456744 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3784005020 con 0x7f378c10e8f0 2024-04-22T13:28:53.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c19b2f0 con 0x7f378c10e8f0 2024-04-22T13:28:53.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f37827fc640 1 -- 172.21.15.5:0/973456744 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840116b0 con 0x7f378c10e8f0 2024-04-22T13:28:53.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.294+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c1a7510 con 0x7f378c10e8f0 2024-04-22T13:28:53.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.296+0000 7f37827fc640 1 -- 172.21.15.5:0/973456744 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f37840051d0 con 0x7f378c10e8f0 2024-04-22T13:28:53.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 Processor -- start 2024-04-22T13:28:53.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 -- start start 2024-04-22T13:28:53.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 0x558bba0ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bba0461f0 con 0x558bba0eaa00 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558bb9fe34b0 con 0x558bba0e14d0 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f379619a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558bba0421b0 con 0x558bba0e1e80 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f37827fc640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37780a3da0 0x7f37780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f3790b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 0x558bba0ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f3790b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 0x558bba0ece50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48328/0 (socket says 172.21.15.5:48328) 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f3790b3f640 1 -- 172.21.15.5:0/557382796 learned_addr learned my addr 172.21.15.5:0/557382796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f3790b3f640 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f3791340640 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.297+0000 7f3791b41640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37780a3da0 0x7f37780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3790b3f640 1 -- 172.21.15.5:0/557382796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 msgr2=0x558bba0ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f37827fc640 1 -- 172.21.15.5:0/973456744 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f37840d0870 con 0x7f378c10e8f0 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3790b3f640 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3790b3f640 1 -- 172.21.15.5:0/557382796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 msgr2=0x558bba0e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3790b3f640 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3790b3f640 1 -- 172.21.15.5:0/557382796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb9fd51b0 con 0x558bba0eaa00 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3790b3f640 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 0x558bba0ece50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f377c014a20 tx=0x7f377c00c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c446d8a8fd5c75ac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f3791b41640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37780a3da0 0x7f37780a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f378409c7d0 tx=0x7f3784081050 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f37777fe640 1 -- 172.21.15.5:0/557382796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c0142d0 con 0x558bba0eaa00 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f37777fe640 1 -- 172.21.15.5:0/557382796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377c0054d0 con 0x558bba0eaa00 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.298+0000 7f37777fe640 1 -- 172.21.15.5:0/557382796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c00f760 con 0x558bba0eaa00 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 -- 172.21.15.5:0/557382796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 msgr2=0x558bba0ece50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 0x558bba0ece50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f377c014a20 tx=0x7f377c00c540 comp rx=0 tx=0).stop 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 -- 172.21.15.5:0/557382796 shutdown_connections 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0eaa00 0x558bba0ece50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 --2- 172.21.15.5:0/557382796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 -- 172.21.15.5:0/557382796 >> 172.21.15.5:0/557382796 conn(0x558bba0dbba0 msgr2=0x558bba0dbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 -- 172.21.15.5:0/557382796 shutdown_connections 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 -- 172.21.15.5:0/557382796 wait complete. 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 -- start start 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.299+0000 7f379619a740 1 Processor -- start 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e1e80 0x558bba0e7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba0e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bba0ed910 con 0x558bba0e14d0 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558bb9fff500 con 0x558bba0e1e80 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558bba0eda90 con 0x558bba0eaa00 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 10 client.0 _ll_register_callbacks cb 0x558bb9fcdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48332/0 (socket says 172.21.15.5:48332) 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 -- 172.21.15.5:0/3487334239 learned_addr learned my addr 172.21.15.5:0/3487334239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3791340640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e1e80 0x558bba0e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba0e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 -- 172.21.15.5:0/3487334239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 msgr2=0x558bba0e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba0e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 -- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e1e80 msgr2=0x558bba0e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e1e80 0x558bba0e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 -- 172.21.15.5:0/3487334239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb9d583f0 con 0x558bba0e14d0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3790b3f640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0ea2d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f377c001eb0 tx=0x7f377c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2eb761c3584850d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3774ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3774ff9640 1 -- 172.21.15.5:0/3487334239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c004fe0 con 0x558bba0e14d0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f3774ff9640 1 -- 172.21.15.5:0/3487334239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f377c005180 con 0x558bba0e14d0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.300+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c0054d0 con 0x558bba0e14d0 2024-04-22T13:28:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f3774ff9640 1 -- 172.21.15.5:0/3487334239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f377c00f730 con 0x558bba0e14d0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 10 client.16407 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bb9e227d0 con 0x558bba0e14d0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f3774ff9640 1 -- 172.21.15.5:0/3487334239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377c0054d0 con 0x558bba0e14d0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f3774ff9640 1 -- 172.21.15.5:0/3487334239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f377c02e5c0 con 0x558bba0e14d0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 10 client.16407 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 10 client.16407 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 20 client.16407 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba0faab0 0x558bba0fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 4 client.16407 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bb9fe5930 con 0x558bba0faab0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.301+0000 7f3791340640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba0faab0 0x558bba0fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.302+0000 7f3774ff9640 10 client.16407 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.302+0000 7f3791340640 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba0faab0 0x558bba0fcea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d20fd186d0e6ac4 server_cookie=cdeec7ebb756b04c in_seq=0 out_seq=0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.302+0000 7f3774ff9640 1 -- 172.21.15.5:0/3487334239 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bb9fe5930 con 0x558bba0faab0 2024-04-22T13:28:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.302+0000 7f3774ff9640 10 client.16407 handle_command_reply: tid=0 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.302+0000 7f379619a740 1 client.16407 shutdown 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 msgr2=0x558bba0ea2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0ea2d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f377c001eb0 tx=0x7f377c004bc0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 shutdown_connections 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba0faab0 0x558bba0fcea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba0e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e1e80 0x558bba0e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f3774ff9640 0 client.16407 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 --2- 172.21.15.5:0/3487334239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0ea2d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 >> 172.21.15.5:0/3487334239 conn(0x558bba0dbba0 msgr2=0x558bba0dd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 shutdown_connections 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 1 -- 172.21.15.5:0/3487334239 wait complete. 2024-04-22T13:28:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.303+0000 7f379619a740 20 client.16407 trim_cache size 0 max 16384 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 Processor -- start 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 -- start start 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0dac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0eaa00 0x558bba1024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f378c1afd80 con 0x558bba0e14d0 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f378c1b18c0 con 0x558bba0eaa00 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f379619a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f378c1b0450 con 0x558bba0e1e80 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f3790b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f3790b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48334/0 (socket says 172.21.15.5:48334) 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.316+0000 7f3790b3f640 1 -- 172.21.15.5:0/368681827 learned_addr learned my addr 172.21.15.5:0/368681827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3791340640 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3790b3f640 1 -- 172.21.15.5:0/368681827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 msgr2=0x558bba0dac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3790b3f640 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0dac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3790b3f640 1 -- 172.21.15.5:0/368681827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0eaa00 msgr2=0x558bba1024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3790b3f640 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0eaa00 0x558bba1024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3790b3f640 1 -- 172.21.15.5:0/368681827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bba0e0e60 con 0x558bba0e14d0 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3790b3f640 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0f67d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f377c00cb60 tx=0x7f377c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c89c2a27e6d3276 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3776ffd640 1 -- 172.21.15.5:0/368681827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c03a070 con 0x558bba0e14d0 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3776ffd640 1 -- 172.21.15.5:0/368681827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377c029070 con 0x558bba0e14d0 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.317+0000 7f3776ffd640 1 -- 172.21.15.5:0/368681827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c02e450 con 0x558bba0e14d0 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- 172.21.15.5:0/368681827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 msgr2=0x558bba0f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0f67d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f377c00cb60 tx=0x7f377c0055a0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- 172.21.15.5:0/368681827 shutdown_connections 2024-04-22T13:28:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0eaa00 0x558bba1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0e1e80 0x558bba0dac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- 172.21.15.5:0/368681827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e14d0 0x558bba0f67d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- 172.21.15.5:0/368681827 >> 172.21.15.5:0/368681827 conn(0x558bba0dbba0 msgr2=0x558bba0dcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- 172.21.15.5:0/368681827 shutdown_connections 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- 172.21.15.5:0/368681827 wait complete. 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- start start 2024-04-22T13:28:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 Processor -- start 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 0x558bba0ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f378c1b11f0 con 0x558bba0e1e80 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f378c1b0b20 con 0x558bba0e14d0 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.318+0000 7f379619a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f378c1add60 con 0x558bba0eaa00 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f379619a740 10 client.0 _ll_register_callbacks cb 0x7f377c0148f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 0x558bba0ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 0x558bba0ee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48342/0 (socket says 172.21.15.5:48342) 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 -- 172.21.15.5:0/4127856950 learned_addr learned my addr 172.21.15.5:0/4127856950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3790b3f640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3790b3f640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 -- 172.21.15.5:0/4127856950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 msgr2=0x558bba15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 -- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 msgr2=0x558bba0ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0ede90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 -- 172.21.15.5:0/4127856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bba15f2c0 con 0x558bba0e1e80 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3790b3f640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3791340640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 0x558bba0ee3d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3788009230 tx=0x7f378802c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d5ca8622707953 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3774ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3774ff9640 1 -- 172.21.15.5:0/4127856950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378804bbd0 con 0x558bba0e1e80 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c029070 con 0x558bba0e1e80 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3774ff9640 1 -- 172.21.15.5:0/4127856950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f378804bd70 con 0x558bba0e1e80 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f3774ff9640 1 -- 172.21.15.5:0/4127856950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3788059050 con 0x558bba0e1e80 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f379619a740 10 client.16413 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.319+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bba160630 con 0x558bba0e1e80 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f3774ff9640 1 -- 172.21.15.5:0/4127856950 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377c029070 con 0x558bba0e1e80 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f3774ff9640 1 -- 172.21.15.5:0/4127856950 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f378804b3f0 con 0x558bba0e1e80 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f379619a740 10 client.16413 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f379619a740 10 client.16413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f379619a740 20 client.16413 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f379619a740 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba160e30 0x558bba163250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f379619a740 4 client.16413 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bba053c10 con 0x558bba160e30 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.320+0000 7f3790b3f640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba160e30 0x558bba163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f3790b3f640 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba160e30 0x558bba163250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=528360cc1483ba34 server_cookie=d8edca77e35b5db1 in_seq=0 out_seq=0 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f3774ff9640 10 client.16413 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f3774ff9640 1 -- 172.21.15.5:0/4127856950 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bba053c10 con 0x558bba160e30 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f3774ff9640 10 client.16413 handle_command_reply: tid=0 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 client.16413 shutdown 2024-04-22T13:28:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 msgr2=0x558bba0ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 0x558bba0ee3d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3788009230 tx=0x7f378802c3d0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 shutdown_connections 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558bba160e30 0x558bba163250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bba0e1e80 0x558bba0ee3d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bba0eaa00 0x558bba15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f3774ff9640 0 client.16413 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 --2- 172.21.15.5:0/4127856950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bba0e14d0 0x558bba0ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 >> 172.21.15.5:0/4127856950 conn(0x558bba0dbba0 msgr2=0x558bba0daa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 shutdown_connections 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 1 -- 172.21.15.5:0/4127856950 wait complete. 2024-04-22T13:28:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.321+0000 7f379619a740 20 client.16413 trim_cache size 0 max 16384 2024-04-22T13:28:53.324 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37780a3da0 msgr2=0x7f37780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37780a3da0 0x7f37780a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f378409c7d0 tx=0x7f3784081050 comp rx=0 tx=0).stop 2024-04-22T13:28:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 msgr2=0x7f378c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 0x7f378c19a830 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f378400e920 tx=0x7f378400ede0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 shutdown_connections 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f378c10e8f0 0x7f378c19a830 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37780a3da0 0x7f37780a6260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f378c101660 0x7f378c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 --2- 172.21.15.5:0/973456744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f378c100d40 0x7f378c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 >> 172.21.15.5:0/973456744 conn(0x7f378c078000 msgr2=0x7f378c110040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 shutdown_connections 2024-04-22T13:28:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.322+0000 7f37935a4640 1 -- 172.21.15.5:0/973456744 wait complete. 2024-04-22T13:28:53.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:28:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.436+0000 7fb64c28c640 1 -- 172.21.15.5:0/1533419574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 msgr2=0x7fb6441011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.436+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1533419574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441011b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb640009900 tx=0x7fb64002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- 172.21.15.5:0/1533419574 shutdown_connections 2024-04-22T13:28:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1533419574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64410e8f0 0x7fb644110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1533419574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101780 0x7fb64410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1533419574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441011b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- 172.21.15.5:0/1533419574 >> 172.21.15.5:0/1533419574 conn(0x7fb644077f70 msgr2=0x7fb644078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- 172.21.15.5:0/1533419574 shutdown_connections 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- 172.21.15.5:0/1533419574 wait complete. 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 Processor -- start 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- start start 2024-04-22T13:28:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb644101780 0x7fb6441a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64410e8f0 0x7fb64419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb644114500 con 0x7fb644100dd0 2024-04-22T13:28:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb644114380 con 0x7fb644101780 2024-04-22T13:28:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64c28c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb644114680 con 0x7fb64410e8f0 2024-04-22T13:28:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.437+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48354/0 (socket says 172.21.15.5:48354) 2024-04-22T13:28:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 -- 172.21.15.5:0/1538430181 learned_addr learned my addr 172.21.15.5:0/1538430181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb649827640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb644101780 0x7fb6441a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a829640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64410e8f0 0x7fb64419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 -- 172.21.15.5:0/1538430181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64410e8f0 msgr2=0x7fb64419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64410e8f0 0x7fb64419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 -- 172.21.15.5:0/1538430181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb644101780 msgr2=0x7fb6441a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb644101780 0x7fb6441a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 -- 172.21.15.5:0/1538430181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64419afe0 con 0x7fb644100dd0 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb649827640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb644101780 0x7fb6441a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64a028640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441a06b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb640000c00 tx=0x7fb64002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb63b7fe640 1 -- 172.21.15.5:0/1538430181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640043020 con 0x7fb644100dd0 2024-04-22T13:28:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb63b7fe640 1 -- 172.21.15.5:0/1538430181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400052b0 con 0x7fb644100dd0 2024-04-22T13:28:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64419b270 con 0x7fb644100dd0 2024-04-22T13:28:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb63b7fe640 1 -- 172.21.15.5:0/1538430181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640003c60 con 0x7fb644100dd0 2024-04-22T13:28:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.438+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6441a7570 con 0x7fb644100dd0 2024-04-22T13:28:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb63b7fe640 1 -- 172.21.15.5:0/1538430181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb640046020 con 0x7fb644100dd0 2024-04-22T13:28:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 Processor -- start 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 -- start start 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd827108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 0x55fd82719430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd8271be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd826751f0 con 0x55fd82710e80 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd826124b0 con 0x55fd82719a00 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.441+0000 7fb64eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fd826711b0 con 0x55fd827104d0 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 0x55fd82719430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 0x55fd82719430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48364/0 (socket says 172.21.15.5:48364) 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 -- 172.21.15.5:0/70095856 learned_addr learned my addr 172.21.15.5:0/70095856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb649827640 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd8271be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb63b7fe640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6300a3d20 0x7fb6300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a829640 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd827108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a829640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6300a3d20 0x7fb6300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 -- 172.21.15.5:0/70095856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 msgr2=0x55fd827108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd827108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 -- 172.21.15.5:0/70095856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 msgr2=0x55fd8271be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd8271be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb63b7fe640 1 -- 172.21.15.5:0/1538430181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb6400f2990 con 0x7fb644100dd0 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 -- 172.21.15.5:0/70095856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd826041b0 con 0x55fd82710e80 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a028640 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 0x55fd82719430 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb6400bea00 tx=0x7fb64004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164d06c4dcbb1c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb64a829640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6300a3d20 0x7fb6300a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009fd0 tx=0x7fb63c005bd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb638bfe640 1 -- 172.21.15.5:0/70095856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400be5b0 con 0x55fd82710e80 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb638bfe640 1 -- 172.21.15.5:0/70095856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400bf170 con 0x55fd82710e80 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.442+0000 7fb638bfe640 1 -- 172.21.15.5:0/70095856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400df0f0 con 0x55fd82710e80 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 -- 172.21.15.5:0/70095856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 msgr2=0x55fd82719430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 0x55fd82719430 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb6400bea00 tx=0x7fb64004f000 comp rx=0 tx=0).stop 2024-04-22T13:28:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 -- 172.21.15.5:0/70095856 shutdown_connections 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82710e80 0x55fd82719430 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd8271be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 --2- 172.21.15.5:0/70095856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd827108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 -- 172.21.15.5:0/70095856 >> 172.21.15.5:0/70095856 conn(0x55fd8270aba0 msgr2=0x55fd8270af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 -- 172.21.15.5:0/70095856 shutdown_connections 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.443+0000 7fb64eb22740 1 -- 172.21.15.5:0/70095856 wait complete. 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 -- start start 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 Processor -- start 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd827192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82716630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82716b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd8271c730 con 0x55fd827104d0 2024-04-22T13:28:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fd8262e500 con 0x55fd82719a00 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fd8271c8b0 con 0x55fd82710e80 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64eb22740 10 client.0 _ll_register_callbacks cb 0x55fd825fcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd827192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd827192a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48380/0 (socket says 172.21.15.5:48380) 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 -- 172.21.15.5:0/3198949093 learned_addr learned my addr 172.21.15.5:0/3198949093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb649827640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82716630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 -- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 msgr2=0x55fd82716630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82716630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 -- 172.21.15.5:0/3198949093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 msgr2=0x55fd82716b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82716b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb64a028640 1 -- 172.21.15.5:0/3198949093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd823873f0 con 0x55fd827104d0 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.444+0000 7fb649827640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82716630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64a028640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd827192a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb6400f02d0 tx=0x7fb640003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c78d18aee280838 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb62dffb640 1 -- 172.21.15.5:0/3198949093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64002cca0 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb62dffb640 1 -- 172.21.15.5:0/3198949093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb6400bb0e0 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6400bf170 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb62dffb640 1 -- 172.21.15.5:0/3198949093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6400be3d0 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 10 client.16425 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd824517d0 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb62dffb640 1 -- 172.21.15.5:0/3198949093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400bf170 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb62dffb640 1 -- 172.21.15.5:0/3198949093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb6400c7250 con 0x55fd827104d0 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 10 client.16425 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 10 client.16425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 20 client.16425 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd82729730 0x55fd8272bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 4 client.16425 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.445+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd82614930 con 0x55fd82729730 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.446+0000 7fb649827640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd82729730 0x55fd8272bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.446+0000 7fb649827640 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd82729730 0x55fd8272bb20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b59724d6d597314 server_cookie=fac3bac5252ae0f3 in_seq=0 out_seq=0 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.446+0000 7fb62dffb640 10 client.16425 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb62dffb640 1 -- 172.21.15.5:0/3198949093 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd82614930 con 0x55fd82729730 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb62dffb640 10 client.16425 handle_command_reply: tid=0 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 client.16425 shutdown 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 msgr2=0x55fd827192a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd827192a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb6400f02d0 tx=0x7fb640003760 comp rx=0 tx=0).stop 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 shutdown_connections 2024-04-22T13:28:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd82729730 0x55fd8272bb20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82716b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb62dffb640 0 client.16425 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82716630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 --2- 172.21.15.5:0/3198949093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd827192a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 >> 172.21.15.5:0/3198949093 conn(0x55fd8270aba0 msgr2=0x55fd8270c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 shutdown_connections 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 1 -- 172.21.15.5:0/3198949093 wait complete. 2024-04-22T13:28:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.447+0000 7fb64eb22740 20 client.16425 trim_cache size 0 max 16384 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.460+0000 7fb64eb22740 1 Processor -- start 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.460+0000 7fb64eb22740 1 -- start start 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd82725560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82711d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82718d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64eb22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441afc90 con 0x55fd827104d0 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64eb22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b17d0 con 0x55fd82719a00 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b0360 con 0x55fd82710e80 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd82725560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd82725560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48396/0 (socket says 172.21.15.5:48396) 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 -- 172.21.15.5:0/2537649262 learned_addr learned my addr 172.21.15.5:0/2537649262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a829640 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82718d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 -- 172.21.15.5:0/2537649262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 msgr2=0x55fd82711d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82711d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 -- 172.21.15.5:0/2537649262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 msgr2=0x55fd82718d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82718d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 -- 172.21.15.5:0/2537649262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd82710c20 con 0x55fd827104d0 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a829640 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82718d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb64a028640 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd82725560 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb6400bea00 tx=0x7fb640005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e24e57cb84c849 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb62ffff640 1 -- 172.21.15.5:0/2537649262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640081030 con 0x55fd827104d0 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb62ffff640 1 -- 172.21.15.5:0/2537649262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400bef30 con 0x55fd827104d0 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.461+0000 7fb62ffff640 1 -- 172.21.15.5:0/2537649262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400bad10 con 0x55fd827104d0 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- 172.21.15.5:0/2537649262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 msgr2=0x55fd82725560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd82725560 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb6400bea00 tx=0x7fb640005e20 comp rx=0 tx=0).stop 2024-04-22T13:28:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- 172.21.15.5:0/2537649262 shutdown_connections 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82719a00 0x55fd82718d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd82710e80 0x55fd82711d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- 172.21.15.5:0/2537649262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd827104d0 0x55fd82725560 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- 172.21.15.5:0/2537649262 >> 172.21.15.5:0/2537649262 conn(0x55fd8270aba0 msgr2=0x55fd82709c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- 172.21.15.5:0/2537649262 shutdown_connections 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- 172.21.15.5:0/2537649262 wait complete. 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- start start 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 Processor -- start 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd82721c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82710e80 0x55fd8271cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 0x55fd8271d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b1100 con 0x55fd82719a00 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441b0a30 con 0x55fd82710e80 2024-04-22T13:28:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.462+0000 7fb64eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441adc70 con 0x55fd827104d0 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64eb22740 10 client.0 _ll_register_callbacks cb 0x7fb6400bea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 0x55fd8271d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 0x55fd8271d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48404/0 (socket says 172.21.15.5:48404) 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 -- 172.21.15.5:0/4219123171 learned_addr learned my addr 172.21.15.5:0/4219123171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a028640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd82721c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb649827640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82710e80 0x55fd8271cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 -- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 msgr2=0x55fd82721c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd82721c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 -- 172.21.15.5:0/4219123171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82710e80 msgr2=0x55fd8271cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82710e80 0x55fd8271cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 -- 172.21.15.5:0/4219123171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8271dad0 con 0x55fd82719a00 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64a829640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 0x55fd8271d510 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb64406a4f0 tx=0x7fb63c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba19a369ea0b46b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb62dffb640 1 -- 172.21.15.5:0/4219123171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00faa0 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb62dffb640 1 -- 172.21.15.5:0/4219123171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb63c00fc40 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6400bef30 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.463+0000 7fb62dffb640 1 -- 172.21.15.5:0/4219123171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb63c00fe40 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 10 client.16431 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd8278f990 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb62dffb640 1 -- 172.21.15.5:0/4219123171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400bef30 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb62dffb640 1 -- 172.21.15.5:0/4219123171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb63c031520 con 0x55fd82719a00 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 10 client.16431 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 20 client.16431 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd827900c0 0x55fd827924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 4 client.16431 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd82682c10 con 0x55fd827900c0 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.464+0000 7fb64a028640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd827900c0 0x55fd827924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb62dffb640 10 client.16431 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64a028640 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd827900c0 0x55fd827924e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=373f24f2c829c336 server_cookie=b6015c9c4ecba29f in_seq=0 out_seq=0 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb62dffb640 1 -- 172.21.15.5:0/4219123171 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd82682c10 con 0x55fd827900c0 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb62dffb640 10 client.16431 handle_command_reply: tid=0 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 client.16431 shutdown 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 msgr2=0x55fd8271d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 0x55fd8271d510 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb64406a4f0 tx=0x7fb63c009940 comp rx=0 tx=0).stop 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 shutdown_connections 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55fd827900c0 0x55fd827924e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd82719a00 0x55fd8271d510 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fd82710e80 0x55fd8271cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb62dffb640 0 client.16431 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 --2- 172.21.15.5:0/4219123171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd827104d0 0x55fd82721c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 >> 172.21.15.5:0/4219123171 conn(0x55fd8270aba0 msgr2=0x55fd82715ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 shutdown_connections 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 1 -- 172.21.15.5:0/4219123171 wait complete. 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64eb22740 20 client.16431 trim_cache size 0 max 16384 2024-04-22T13:28:53.471 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6300a3d20 msgr2=0x7fb6300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6300a3d20 0x7fb6300a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009fd0 tx=0x7fb63c005bd0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 msgr2=0x7fb6441a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.465+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441a06b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb640000c00 tx=0x7fb64002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 shutdown_connections 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb6300a3d20 0x7fb6300a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64410e8f0 0x7fb64419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb644101780 0x7fb6441a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 --2- 172.21.15.5:0/1538430181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb644100dd0 0x7fb6441a06b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 >> 172.21.15.5:0/1538430181 conn(0x7fb644077f70 msgr2=0x7fb64410fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 shutdown_connections 2024-04-22T13:28:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.466+0000 7fb64c28c640 1 -- 172.21.15.5:0/1538430181 wait complete. 2024-04-22T13:28:53.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:28:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.583+0000 7f457673d640 1 -- 172.21.15.5:0/2237233591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4570103b70 msgr2=0x7f457010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.583+0000 7f457673d640 1 --2- 172.21.15.5:0/2237233591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4570103b70 0x7f457010bfc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4560009900 tx=0x7f456002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.583+0000 7f457673d640 1 -- 172.21.15.5:0/2237233591 shutdown_connections 2024-04-22T13:28:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 --2- 172.21.15.5:0/2237233591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4570103b70 0x7f457010bfc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 --2- 172.21.15.5:0/2237233591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f457010c6d0 0x7f457010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 --2- 172.21.15.5:0/2237233591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45701031c0 0x7f45701035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- 172.21.15.5:0/2237233591 >> 172.21.15.5:0/2237233591 conn(0x7f4570077f70 msgr2=0x7f4570078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- 172.21.15.5:0/2237233591 shutdown_connections 2024-04-22T13:28:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- 172.21.15.5:0/2237233591 wait complete. 2024-04-22T13:28:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 Processor -- start 2024-04-22T13:28:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- start start 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701031c0 0x7f457019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f457010c6d0 0x7f457019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 0x7f4570198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4570112350 con 0x7f45701987e0 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45701121d0 con 0x7f45701031c0 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.584+0000 7f457673d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45701124d0 con 0x7f457010c6d0 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 0x7f4570198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 0x7f4570198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48426/0 (socket says 172.21.15.5:48426) 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f456ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701031c0 0x7f457019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 -- 172.21.15.5:0/137157505 learned_addr learned my addr 172.21.15.5:0/137157505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f456f7fe640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f457010c6d0 0x7f457019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 -- 172.21.15.5:0/137157505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f457010c6d0 msgr2=0x7f457019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f457010c6d0 0x7f457019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 -- 172.21.15.5:0/137157505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701031c0 msgr2=0x7f457019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701031c0 0x7f457019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 -- 172.21.15.5:0/137157505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4570199370 con 0x7f45701987e0 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f456f7fe640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f457010c6d0 0x7f457019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f4574cda640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 0x7f4570198c90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f456800cad0 tx=0x7f456800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f456d7fa640 1 -- 172.21.15.5:0/137157505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456800ae20 con 0x7f45701987e0 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f456d7fa640 1 -- 172.21.15.5:0/137157505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4568005020 con 0x7f45701987e0 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4570199660 con 0x7f45701987e0 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f456d7fa640 1 -- 172.21.15.5:0/137157505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456800f6b0 con 0x7f45701987e0 2024-04-22T13:28:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.585+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a57c0 con 0x7f45701987e0 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456d7fa640 1 -- 172.21.15.5:0/137157505 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f45680051d0 con 0x7f45701987e0 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 Processor -- start 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 -- start start 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x5583673f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 0x5583673f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55836734c1f0 con 0x5583673f0a00 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5583672e94b0 con 0x5583673e7e80 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f457940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5583673481b0 con 0x5583673e74d0 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 0x5583673f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 0x5583673f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48440/0 (socket says 172.21.15.5:48440) 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456f7fe640 1 -- 172.21.15.5:0/4097472155 learned_addr learned my addr 172.21.15.5:0/4097472155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456d7fa640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f45580a3d20 0x7f45580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456f7fe640 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456ffff640 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x5583673f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f4574cda640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f45580a3d20 0x7f45580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.588+0000 7f456d7fa640 1 -- 172.21.15.5:0/137157505 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f45680d0830 con 0x7f45701987e0 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f456f7fe640 1 -- 172.21.15.5:0/4097472155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 msgr2=0x5583673e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f456f7fe640 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f456f7fe640 1 -- 172.21.15.5:0/4097472155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 msgr2=0x5583673f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f456f7fe640 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x5583673f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f456f7fe640 1 -- 172.21.15.5:0/4097472155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583672db1b0 con 0x5583673f0a00 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f456f7fe640 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 0x5583673f2e50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4560009900 tx=0x7f4560005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bae968c5782cec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f4574cda640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f45580a3d20 0x7f45580a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45680ce060 tx=0x7f4568067350 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f45567fc640 1 -- 172.21.15.5:0/4097472155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560049070 con 0x5583673f0a00 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f45567fc640 1 -- 172.21.15.5:0/4097472155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560005500 con 0x5583673f0a00 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f45567fc640 1 -- 172.21.15.5:0/4097472155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560004950 con 0x5583673f0a00 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.589+0000 7f457940a740 1 -- 172.21.15.5:0/4097472155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 msgr2=0x5583673f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 0x5583673f2e50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4560009900 tx=0x7f4560005fd0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- 172.21.15.5:0/4097472155 shutdown_connections 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673f0a00 0x5583673f2e50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x5583673f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- 172.21.15.5:0/4097472155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- 172.21.15.5:0/4097472155 >> 172.21.15.5:0/4097472155 conn(0x5583673e1ba0 msgr2=0x5583673e1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- 172.21.15.5:0/4097472155 shutdown_connections 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- 172.21.15.5:0/4097472155 wait complete. 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- start start 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 Processor -- start 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x5583673f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e7e80 0x5583673ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5583673f3910 con 0x5583673e74d0 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558367305500 con 0x5583673f0a00 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5583673f3a90 con 0x5583673e7e80 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.590+0000 7f457940a740 10 client.0 _ll_register_callbacks cb 0x5583672d3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x5583673f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x5583673f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48442/0 (socket says 172.21.15.5:48442) 2024-04-22T13:28:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 -- 172.21.15.5:0/3906508022 learned_addr learned my addr 172.21.15.5:0/3906508022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456ffff640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e7e80 0x5583673ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 -- 172.21.15.5:0/3906508022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e7e80 msgr2=0x5583673ed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e7e80 0x5583673ed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 -- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 msgr2=0x5583673edd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673edd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 -- 172.21.15.5:0/3906508022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836705e3f0 con 0x5583673e74d0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f456f7fe640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x5583673f0360 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4560001eb0 tx=0x7f45600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54594d989486d4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f454f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f454f7fe640 1 -- 172.21.15.5:0/3906508022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560056060 con 0x5583673e74d0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f454f7fe640 1 -- 172.21.15.5:0/3906508022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f45600038c0 con 0x5583673e74d0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4560005500 con 0x5583673e74d0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f454f7fe640 1 -- 172.21.15.5:0/3906508022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4560049030 con 0x5583673e74d0 2024-04-22T13:28:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f457940a740 10 client.16443 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.591+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583671287d0 con 0x5583673e74d0 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f454f7fe640 1 -- 172.21.15.5:0/3906508022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560005500 con 0x5583673e74d0 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f454f7fe640 1 -- 172.21.15.5:0/3906508022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4560040040 con 0x5583673e74d0 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f457940a740 10 client.16443 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f457940a740 10 client.16443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f457940a740 20 client.16443 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f457940a740 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367400a80 0x558367402e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f457940a740 4 client.16443 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583672eb930 con 0x558367400a80 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.592+0000 7f456ffff640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367400a80 0x558367402e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.593+0000 7f454f7fe640 10 client.16443 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.593+0000 7f456ffff640 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367400a80 0x558367402e70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d931fc7f5cb39dc server_cookie=23de8ab93f516c91 in_seq=0 out_seq=0 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.593+0000 7f454f7fe640 1 -- 172.21.15.5:0/3906508022 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583672eb930 con 0x558367400a80 2024-04-22T13:28:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.593+0000 7f454f7fe640 10 client.16443 handle_command_reply: tid=0 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.593+0000 7f457940a740 1 client.16443 shutdown 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 msgr2=0x5583673f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x5583673f0360 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4560001eb0 tx=0x7f45600057f0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 shutdown_connections 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367400a80 0x558367402e70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e7e80 0x5583673ed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f454f7fe640 0 client.16443 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 --2- 172.21.15.5:0/3906508022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x5583673f0360 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 >> 172.21.15.5:0/3906508022 conn(0x5583673e1ba0 msgr2=0x5583673e38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 shutdown_connections 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 1 -- 172.21.15.5:0/3906508022 wait complete. 2024-04-22T13:28:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.594+0000 7f457940a740 20 client.16443 trim_cache size 0 max 16384 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 Processor -- start 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 -- start start 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 0x5583673f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45701adb10 con 0x5583673e7e80 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45701af650 con 0x5583673f0a00 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f457940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ae1e0 con 0x5583673e74d0 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f456ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 0x5583673f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f456ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 0x5583673f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48458/0 (socket says 172.21.15.5:48458) 2024-04-22T13:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f456ffff640 1 -- 172.21.15.5:0/663858941 learned_addr learned my addr 172.21.15.5:0/663858941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.607+0000 7f456f7fe640 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456f7fe640 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456ffff640 1 -- 172.21.15.5:0/663858941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 msgr2=0x5583673efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456ffff640 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456ffff640 1 -- 172.21.15.5:0/663858941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 msgr2=0x5583673ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456ffff640 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456ffff640 1 -- 172.21.15.5:0/663858941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583673e7c20 con 0x5583673e7e80 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456f7fe640 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f456ffff640 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 0x5583673f0360 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x5583673e8280 tx=0x7f456400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b29ec22ae1e44f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f4555ffb640 1 -- 172.21.15.5:0/663858941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564005020 con 0x5583673e7e80 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f4555ffb640 1 -- 172.21.15.5:0/663858941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640047a0 con 0x5583673e7e80 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.608+0000 7f4555ffb640 1 -- 172.21.15.5:0/663858941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564020410 con 0x5583673e7e80 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- 172.21.15.5:0/663858941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 msgr2=0x5583673f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 0x5583673f0360 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x5583673e8280 tx=0x7f456400b390 comp rx=0 tx=0).stop 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- 172.21.15.5:0/663858941 shutdown_connections 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e7e80 0x5583673f0360 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673f0a00 0x5583673ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- 172.21.15.5:0/663858941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673e74d0 0x5583673efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- 172.21.15.5:0/663858941 >> 172.21.15.5:0/663858941 conn(0x5583673e1ba0 msgr2=0x5583673e0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- 172.21.15.5:0/663858941 shutdown_connections 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- 172.21.15.5:0/663858941 wait complete. 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- start start 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 Processor -- start 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x558367403340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x558367403880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673f0a00 0x5583674058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45701aef80 con 0x5583673e74d0 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ae8b0 con 0x5583673e7e80 2024-04-22T13:28:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45701abaf0 con 0x5583673f0a00 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f457940a740 10 client.0 _ll_register_callbacks cb 0x7f4560004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.609+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x558367403340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x558367403340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48462/0 (socket says 172.21.15.5:48462) 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 -- 172.21.15.5:0/1015172698 learned_addr learned my addr 172.21.15.5:0/1015172698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456ffff640 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x558367403880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 -- 172.21.15.5:0/1015172698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673f0a00 msgr2=0x5583674058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673f0a00 0x5583674058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 -- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 msgr2=0x558367403880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x558367403880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 -- 172.21.15.5:0/1015172698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558367405f10 con 0x5583673e74d0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f456f7fe640 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x558367403340 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4560005860 tx=0x7f4560043ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fccdc4485f16a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f454f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f454f7fe640 1 -- 172.21.15.5:0/1015172698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560056040 con 0x5583673e74d0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f454f7fe640 1 -- 172.21.15.5:0/1015172698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f4560003ad0 con 0x5583673e74d0 2024-04-22T13:28:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45640047a0 con 0x5583673e74d0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f454f7fe640 1 -- 172.21.15.5:0/1015172698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f456006a050 con 0x5583673e74d0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f457940a740 10 client.16449 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558367466730 con 0x5583673e74d0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.610+0000 7f454f7fe640 1 -- 172.21.15.5:0/1015172698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640047a0 con 0x5583673e74d0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f454f7fe640 1 -- 172.21.15.5:0/1015172698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4560063060 con 0x5583673e74d0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 10 client.16449 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 20 client.16449 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367466c60 0x558367469080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 4 client.16449 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558367359c10 con 0x558367466c60 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f456ffff640 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367466c60 0x558367469080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f456ffff640 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367466c60 0x558367469080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6bb13ac6d27ad94 server_cookie=32e8f04e6a4e59b6 in_seq=0 out_seq=0 2024-04-22T13:28:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f454f7fe640 10 client.16449 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f454f7fe640 1 -- 172.21.15.5:0/1015172698 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558367359c10 con 0x558367466c60 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f454f7fe640 10 client.16449 handle_command_reply: tid=0 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.611+0000 7f457940a740 1 client.16449 shutdown 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 msgr2=0x558367403340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x558367403340 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4560005860 tx=0x7f4560043ab0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 shutdown_connections 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x558367466c60 0x558367469080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583673f0a00 0x5583674058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583673e7e80 0x558367403880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f454f7fe640 0 client.16449 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 --2- 172.21.15.5:0/1015172698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583673e74d0 0x558367403340 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 >> 172.21.15.5:0/1015172698 conn(0x5583673e1ba0 msgr2=0x5583673fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 shutdown_connections 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 1 -- 172.21.15.5:0/1015172698 wait complete. 2024-04-22T13:28:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457940a740 20 client.16449 trim_cache size 0 max 16384 2024-04-22T13:28:53.615 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f45580a3d20 msgr2=0x7f45580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f45580a3d20 0x7f45580a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45680ce060 tx=0x7f4568067350 comp rx=0 tx=0).stop 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 msgr2=0x7f4570198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 0x7f4570198c90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f456800cad0 tx=0x7f456800cf90 comp rx=0 tx=0).stop 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 shutdown_connections 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45701987e0 0x7f4570198c90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f45580a3d20 0x7f45580a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f457010c6d0 0x7f457019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 --2- 172.21.15.5:0/137157505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701031c0 0x7f457019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.612+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 >> 172.21.15.5:0/137157505 conn(0x7f4570077f70 msgr2=0x7f45700ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.613+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 shutdown_connections 2024-04-22T13:28:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.613+0000 7f457673d640 1 -- 172.21.15.5:0/137157505 wait complete. 2024-04-22T13:28:53.618 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:53 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1131334438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:53.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.727+0000 7f48209a4640 1 -- 172.21.15.5:0/2659604564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 msgr2=0x7f481c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.727+0000 7f48209a4640 1 --2- 172.21.15.5:0/2659604564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c101470 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f480c009900 tx=0x7f480c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 -- 172.21.15.5:0/2659604564 shutdown_connections 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 --2- 172.21.15.5:0/2659604564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 0x7f481c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 --2- 172.21.15.5:0/2659604564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c101a40 0x7f481c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 --2- 172.21.15.5:0/2659604564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c101470 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 -- 172.21.15.5:0/2659604564 >> 172.21.15.5:0/2659604564 conn(0x7f481c078000 msgr2=0x7f481c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 -- 172.21.15.5:0/2659604564 shutdown_connections 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 -- 172.21.15.5:0/2659604564 wait complete. 2024-04-22T13:28:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 Processor -- start 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.728+0000 7f48209a4640 1 -- start start 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 0x7f481c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c19a0b0 0x7f481c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1110c0 con 0x7f481c101090 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f481c110f40 con 0x7f481c106160 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f481c111240 con 0x7f481c19a0b0 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48480/0 (socket says 172.21.15.5:48480) 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 -- 172.21.15.5:0/2374134092 learned_addr learned my addr 172.21.15.5:0/2374134092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f4813fff640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 0x7f481c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481ad9d640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c19a0b0 0x7f481c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 -- 172.21.15.5:0/2374134092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c19a0b0 msgr2=0x7f481c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c19a0b0 0x7f481c1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 -- 172.21.15.5:0/2374134092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 msgr2=0x7f481c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 0x7f481c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 -- 172.21.15.5:0/2374134092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c1a4e30 con 0x7f481c101090 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481ad9d640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c19a0b0 0x7f481c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f4813fff640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 0x7f481c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f481a59c640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c1995e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f480c005940 tx=0x7f480c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48137fe640 1 -- 172.21.15.5:0/2374134092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c043020 con 0x7f481c101090 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48137fe640 1 -- 172.21.15.5:0/2374134092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f480c02caf0 con 0x7f481c101090 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c1a50c0 con 0x7f481c101090 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48137fe640 1 -- 172.21.15.5:0/2374134092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c003af0 con 0x7f481c101090 2024-04-22T13:28:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.729+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a5610 con 0x7f481c101090 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f48137fe640 1 -- 172.21.15.5:0/2374134092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f480c046020 con 0x7f481c101090 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 Processor -- start 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 -- start start 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175db6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561175d1b1f0 con 0x561175db6c00 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561175cb84b0 con 0x561175dc3e10 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f4823599740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561175d171b0 con 0x561175db6250 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.732+0000 7f48137fe640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f48080a3d20 0x7f48080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481ad9d640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f48080a3d20 0x7f48080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48496/0 (socket says 172.21.15.5:48496) 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 -- 172.21.15.5:0/1544554320 learned_addr learned my addr 172.21.15.5:0/1544554320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f4813fff640 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f48137fe640 1 -- 172.21.15.5:0/2374134092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f480c0f2900 con 0x7f481c101090 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481ad9d640 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175db6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 -- 172.21.15.5:0/1544554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 msgr2=0x561175db6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175db6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 -- 172.21.15.5:0/1544554320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 msgr2=0x561175dc6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 -- 172.21.15.5:0/1544554320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561175caa1b0 con 0x561175db6c00 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481ad9d640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f48080a3d20 0x7f48080a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4814005fd0 tx=0x7f4814009e10 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f481a59c640 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc3840 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f480c0be970 tx=0x7f480c0914d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf88bbe4f93068 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f4810ff9640 1 -- 172.21.15.5:0/1544554320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c0bf0e0 con 0x561175db6c00 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f4810ff9640 1 -- 172.21.15.5:0/1544554320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f480c0894f0 con 0x561175db6c00 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.733+0000 7f4810ff9640 1 -- 172.21.15.5:0/1544554320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c0bb050 con 0x561175db6c00 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 -- 172.21.15.5:0/1544554320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 msgr2=0x561175dc3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc3840 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f480c0be970 tx=0x7f480c0914d0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 -- 172.21.15.5:0/1544554320 shutdown_connections 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc3840 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 --2- 172.21.15.5:0/1544554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175db6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 -- 172.21.15.5:0/1544554320 >> 172.21.15.5:0/1544554320 conn(0x561175db0ba0 msgr2=0x561175db0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 -- 172.21.15.5:0/1544554320 shutdown_connections 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 -- 172.21.15.5:0/1544554320 wait complete. 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.734+0000 7f4823599740 1 -- start start 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 Processor -- start 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dc3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561175dc6b00 con 0x561175db6c00 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561175cd4500 con 0x561175dc3e10 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561175dc6c80 con 0x561175db6250 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4823599740 10 client.0 _ll_register_callbacks cb 0x561175ca2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48506/0 (socket says 172.21.15.5:48506) 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 -- 172.21.15.5:0/215005655 learned_addr learned my addr 172.21.15.5:0/215005655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f481a59c640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dc3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f481ad9d640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 -- 172.21.15.5:0/215005655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 msgr2=0x561175dc3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dc3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 -- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 msgr2=0x561175dc1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 -- 172.21.15.5:0/215005655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561175a2d3f0 con 0x561175db6c00 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f481ad9d640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f481a59c640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dc3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4813fff640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc0aa0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f48040136e0 tx=0x7f4804013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28c6abd77399e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.735+0000 7f4801ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4801ffb640 1 -- 172.21.15.5:0/215005655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804011dd0 con 0x561175db6c00 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4801ffb640 1 -- 172.21.15.5:0/215005655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f4804004030 con 0x561175db6c00 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480c0894f0 con 0x561175db6c00 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4801ffb640 1 -- 172.21.15.5:0/215005655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4804004230 con 0x561175db6c00 2024-04-22T13:28:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 10 client.16461 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561175af77d0 con 0x561175db6c00 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4801ffb640 1 -- 172.21.15.5:0/215005655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f480c0894f0 con 0x561175db6c00 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4801ffb640 1 -- 172.21.15.5:0/215005655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f480400f040 con 0x561175db6c00 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 10 client.16461 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 20 client.16461 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175dd3cb0 0x561175dd60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 4 client.16461 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561175cba930 con 0x561175dd3cb0 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.736+0000 7f481a59c640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175dd3cb0 0x561175dd60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.737+0000 7f4801ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.737+0000 7f481a59c640 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175dd3cb0 0x561175dd60f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4d631d5c1a338b server_cookie=3eb62fdedb29b716 in_seq=0 out_seq=0 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.737+0000 7f4801ffb640 1 -- 172.21.15.5:0/215005655 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561175cba930 con 0x561175dd3cb0 2024-04-22T13:28:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4801ffb640 10 client.16461 handle_command_reply: tid=0 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 client.16461 shutdown 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 msgr2=0x561175dc0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc0aa0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f48040136e0 tx=0x7f4804013ba0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 shutdown_connections 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175dd3cb0 0x561175dd60f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175db6c00 0x561175dc0aa0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175dc3e10 0x561175dc1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 --2- 172.21.15.5:0/215005655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dc3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4801ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 >> 172.21.15.5:0/215005655 conn(0x561175db0ba0 msgr2=0x561175db28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 shutdown_connections 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 1 -- 172.21.15.5:0/215005655 wait complete. 2024-04-22T13:28:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.738+0000 7f4823599740 20 client.16461 trim_cache size 0 max 16384 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 Processor -- start 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 -- start start 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dcf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6c00 0x561175dc31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1ab990 con 0x561175dc3e10 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1ad4d0 con 0x561175db6c00 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f4823599740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1ac060 con 0x561175db6250 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.751+0000 7f481ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48510/0 (socket says 172.21.15.5:48510) 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 -- 172.21.15.5:0/1522176175 learned_addr learned my addr 172.21.15.5:0/1522176175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481a59c640 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dcf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f4813fff640 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6c00 0x561175dc31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 -- 172.21.15.5:0/1522176175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 msgr2=0x561175dcf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dcf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 -- 172.21.15.5:0/1522176175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6c00 msgr2=0x561175dc31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6c00 0x561175dc31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 -- 172.21.15.5:0/1522176175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561175db5560 con 0x561175dc3e10 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f4813fff640 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6c00 0x561175dc31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f481ad9d640 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc3710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4814009c50 tx=0x7f4814004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62712e625faf71c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f4803fff640 1 -- 172.21.15.5:0/1522176175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814025070 con 0x561175dc3e10 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f4803fff640 1 -- 172.21.15.5:0/1522176175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4814012da0 con 0x561175dc3e10 2024-04-22T13:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.752+0000 7f4803fff640 1 -- 172.21.15.5:0/1522176175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814021ab0 con 0x561175dc3e10 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- 172.21.15.5:0/1522176175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 msgr2=0x561175dc3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc3710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4814009c50 tx=0x7f4814004ab0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- 172.21.15.5:0/1522176175 shutdown_connections 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc3710 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6c00 0x561175dc31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- 172.21.15.5:0/1522176175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6250 0x561175dcf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- 172.21.15.5:0/1522176175 >> 172.21.15.5:0/1522176175 conn(0x561175db0ba0 msgr2=0x561175dbc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- 172.21.15.5:0/1522176175 shutdown_connections 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- 172.21.15.5:0/1522176175 wait complete. 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- start start 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 Processor -- start 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6250 0x561175dc76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6c00 0x561175dc7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1ace00 con 0x561175dc3e10 2024-04-22T13:28:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1ac730 con 0x561175db6250 2024-04-22T13:28:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1a9970 con 0x561175db6c00 2024-04-22T13:28:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f4823599740 10 client.0 _ll_register_callbacks cb 0x7f4804009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f481ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f481ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48526/0 (socket says 172.21.15.5:48526) 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.753+0000 7f481ad9d640 1 -- 172.21.15.5:0/3755805088 learned_addr learned my addr 172.21.15.5:0/3755805088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4813fff640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6c00 0x561175dc7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481a59c640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6250 0x561175dc76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481ad9d640 1 -- 172.21.15.5:0/3755805088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6c00 msgr2=0x561175dc7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481ad9d640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6c00 0x561175dc7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481ad9d640 1 -- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6250 msgr2=0x561175dc76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481ad9d640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6250 0x561175dc76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481ad9d640 1 -- 172.21.15.5:0/3755805088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561175e39230 con 0x561175dc3e10 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4813fff640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6c00 0x561175dc7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481a59c640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6250 0x561175dc76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f481ad9d640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc8140 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4814021690 tx=0x7f48140214a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3342fef81a27da server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4801ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4801ffb640 1 -- 172.21.15.5:0/3755805088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814037060 con 0x561175dc3e10 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4801ffb640 1 -- 172.21.15.5:0/3755805088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f48140219f0 con 0x561175dc3e10 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4814012da0 con 0x561175dc3e10 2024-04-22T13:28:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4801ffb640 1 -- 172.21.15.5:0/3755805088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4814025070 con 0x561175dc3e10 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4823599740 10 client.16467 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561175e3a3e0 con 0x561175dc3e10 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.754+0000 7f4801ffb640 1 -- 172.21.15.5:0/3755805088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4814012da0 con 0x561175dc3e10 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4801ffb640 1 -- 172.21.15.5:0/3755805088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4814031030 con 0x561175dc3e10 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4823599740 10 client.16467 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4823599740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4823599740 20 client.16467 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4823599740 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175e3aab0 0x561175e3ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4823599740 4 client.16467 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561175d28c10 con 0x561175e3aab0 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f481a59c640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175e3aab0 0x561175e3ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f481a59c640 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175e3aab0 0x561175e3ced0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf33f5fdfbe97a73 server_cookie=b91bb5ea301c605a in_seq=0 out_seq=0 2024-04-22T13:28:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.755+0000 7f4801ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4801ffb640 1 -- 172.21.15.5:0/3755805088 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561175d28c10 con 0x561175e3aab0 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4801ffb640 10 client.16467 handle_command_reply: tid=0 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 client.16467 shutdown 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 msgr2=0x561175dc8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc8140 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4814021690 tx=0x7f48140214a0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 shutdown_connections 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561175e3aab0 0x561175e3ced0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561175dc3e10 0x561175dc8140 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561175db6c00 0x561175dc7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4801ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 --2- 172.21.15.5:0/3755805088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561175db6250 0x561175dc76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 >> 172.21.15.5:0/3755805088 conn(0x561175db0ba0 msgr2=0x561175db1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 shutdown_connections 2024-04-22T13:28:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 1 -- 172.21.15.5:0/3755805088 wait complete. 2024-04-22T13:28:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f4823599740 20 client.16467 trim_cache size 0 max 16384 2024-04-22T13:28:53.762 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f48080a3d20 msgr2=0x7f48080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f48080a3d20 0x7f48080a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4814005fd0 tx=0x7f4814009e10 comp rx=0 tx=0).stop 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 msgr2=0x7f481c1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c1995e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f480c005940 tx=0x7f480c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 shutdown_connections 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f48080a3d20 0x7f48080a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f481c19a0b0 0x7f481c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f481c106160 0x7f481c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 --2- 172.21.15.5:0/2374134092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f481c101090 0x7f481c1995e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 >> 172.21.15.5:0/2374134092 conn(0x7f481c078000 msgr2=0x7f481c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 shutdown_connections 2024-04-22T13:28:53.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:53.756+0000 7f48209a4640 1 -- 172.21.15.5:0/2374134092 wait complete. 2024-04-22T13:28:54.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:53 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1131334438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:54.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:53 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1131334438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:54.766 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:28:54.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:28:54.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:28:54.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:54 smithi005 ceph-mon[28006]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 -- 172.21.15.5:0/1571435946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 msgr2=0x7fa704111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/1571435946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa704111170 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009ca0 tx=0x7fa6fc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 -- 172.21.15.5:0/1571435946 shutdown_connections 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/1571435946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa704111170 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/1571435946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7041014a0 0x7fa704109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/1571435946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 0x7fa704100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 -- 172.21.15.5:0/1571435946 >> 172.21.15.5:0/1571435946 conn(0x7fa7040760d0 msgr2=0x7fa7040764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 -- 172.21.15.5:0/1571435946 shutdown_connections 2024-04-22T13:28:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 -- 172.21.15.5:0/1571435946 wait complete. 2024-04-22T13:28:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 Processor -- start 2024-04-22T13:28:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.891+0000 7fa70a4c8640 1 -- start start 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa70a4c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 0x7fa7041a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa70a4c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa7041a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa70a4c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa70419a880 0x7fa70419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa70a4c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa704114520 con 0x7fa70410a6b0 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa70a4c8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041143a0 con 0x7fa704100af0 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa70a4c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041146a0 con 0x7fa70419a880 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa7041a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa7041a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48538/0 (socket says 172.21.15.5:48538) 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 -- 172.21.15.5:0/3190600080 learned_addr learned my addr 172.21.15.5:0/3190600080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa708a65640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa70419a880 0x7fa70419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa703fff640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 0x7fa7041a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 -- 172.21.15.5:0/3190600080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa70419a880 msgr2=0x7fa70419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa70419a880 0x7fa70419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 -- 172.21.15.5:0/3190600080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 msgr2=0x7fa7041a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 0x7fa7041a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 -- 172.21.15.5:0/3190600080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70419b3a0 con 0x7fa70410a6b0 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa703fff640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 0x7fa7041a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.892+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa7041a0c10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e9c0 tx=0x7fa6f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.893+0000 7fa7017fa640 1 -- 172.21.15.5:0/3190600080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400ccc0 con 0x7fa70410a6b0 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.893+0000 7fa7017fa640 1 -- 172.21.15.5:0/3190600080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f40054d0 con 0x7fa70410a6b0 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.893+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70419b690 con 0x7fa70410a6b0 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.893+0000 7fa7017fa640 1 -- 172.21.15.5:0/3190600080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4010430 con 0x7fa70410a6b0 2024-04-22T13:28:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.893+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7041a7890 con 0x7fa70410a6b0 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa7017fa640 1 -- 172.21.15.5:0/3190600080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa6f4005020 con 0x7fa70410a6b0 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 Processor -- start 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 -- start start 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a60cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a6c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595999c4f00 con 0x559599a69e10 2024-04-22T13:28:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559599962590 con 0x559599a608e0 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa70d19a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595999c0ec0 con 0x559599a61290 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a6c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a6c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48554/0 (socket says 172.21.15.5:48554) 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa7037fe640 1 -- 172.21.15.5:0/1601292592 learned_addr learned my addr 172.21.15.5:0/1601292592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa703fff640 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa7017fa640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6f00a3d20 0x7fa6f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.896+0000 7fa708a65640 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a60cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa703fff640 1 -- 172.21.15.5:0/1601292592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 msgr2=0x559599a60cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa703fff640 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a60cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa703fff640 1 -- 172.21.15.5:0/1601292592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 msgr2=0x559599a6c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa708a65640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6f00a3d20 0x7fa6f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa703fff640 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a6c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa703fff640 1 -- 172.21.15.5:0/1601292592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959979f7e0 con 0x559599a61290 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa7017fa640 1 -- 172.21.15.5:0/3190600080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa6f40d0c10 con 0x7fa70410a6b0 2024-04-22T13:28:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa7037fe640 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a6c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa703fff640 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a69840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005d00 tx=0x7fa6ec00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86101d567fae1390 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa708a65640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6f00a3d20 0x7fa6f00a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc004af0 tx=0x7fa6fc02c0f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa6fa7fc640 1 -- 172.21.15.5:0/1601292592 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec016070 con 0x559599a61290 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa6fa7fc640 1 -- 172.21.15.5:0/1601292592 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ec0035f0 con 0x559599a61290 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.897+0000 7fa6fa7fc640 1 -- 172.21.15.5:0/1601292592 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0116d0 con 0x559599a61290 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 -- 172.21.15.5:0/1601292592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 msgr2=0x559599a69840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a69840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005d00 tx=0x7fa6ec00e6c0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 -- 172.21.15.5:0/1601292592 shutdown_connections 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a6c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a69840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 --2- 172.21.15.5:0/1601292592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a60cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 -- 172.21.15.5:0/1601292592 >> 172.21.15.5:0/1601292592 conn(0x559599a5afb0 msgr2=0x559599a5b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 -- 172.21.15.5:0/1601292592 shutdown_connections 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 -- 172.21.15.5:0/1601292592 wait complete. 2024-04-22T13:28:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.898+0000 7fa70d19a740 1 -- start start 2024-04-22T13:28:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 Processor -- start 2024-04-22T13:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 0x559599a66de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a69e10 0x559599a67860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559599a6cdd0 con 0x559599a608e0 2024-04-22T13:28:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55959997e580 con 0x559599a69e10 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559599a6cf50 con 0x559599a61290 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa70d19a740 10 client.0 _ll_register_callbacks cb 0x559599984650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 0x559599a66de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 0x559599a66de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48560/0 (socket says 172.21.15.5:48560) 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 -- 172.21.15.5:0/3473431614 learned_addr learned my addr 172.21.15.5:0/3473431614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa708a65640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a69e10 0x559599a67860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 -- 172.21.15.5:0/3473431614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 msgr2=0x559599a67320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a67320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 -- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a69e10 msgr2=0x559599a67860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a69e10 0x559599a67860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 -- 172.21.15.5:0/3473431614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595997fff70 con 0x559599a608e0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa703fff640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 0x559599a66de0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000c00 tx=0x7fa6ec0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0c469b19b16ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.899+0000 7fa6e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3473431614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec028060 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3473431614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa6ec004000 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec0035f0 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3473431614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6ec016040 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 10 client.16479 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559599697170 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3473431614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ec0035f0 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3473431614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6ec00f040 con 0x559599a608e0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 10 client.16479 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 20 client.16479 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599a7a2d0 0x559599a7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 4 client.16479 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.900+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595999d2920 con 0x559599a7a2d0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.901+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599a7a2d0 0x559599a7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.901+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599a7a2d0 0x559599a7c6c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a166c8eb408907ad server_cookie=90155d2f14de5bec in_seq=0 out_seq=0 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.901+0000 7fa6e37fe640 10 client.16479 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.902+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3473431614 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595999d2920 con 0x559599a7a2d0 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.902+0000 7fa6e37fe640 10 client.16479 handle_command_reply: tid=0 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.902+0000 7fa70d19a740 1 client.16479 shutdown 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.902+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 msgr2=0x559599a66de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.902+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 0x559599a66de0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000c00 tx=0x7fa6ec0039d0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 shutdown_connections 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599a7a2d0 0x559599a7c6c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a69e10 0x559599a67860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a61290 0x559599a67320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3473431614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a608e0 0x559599a66de0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 >> 172.21.15.5:0/3473431614 conn(0x559599a5afb0 msgr2=0x559599a5cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa6e37fe640 0 client.16479 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 shutdown_connections 2024-04-22T13:28:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 1 -- 172.21.15.5:0/3473431614 wait complete. 2024-04-22T13:28:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.903+0000 7fa70d19a740 20 client.16479 trim_cache size 0 max 16384 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.922+0000 7fa70d19a740 1 Processor -- start 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.922+0000 7fa70d19a740 1 -- start start 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a608e0 0x559599a69510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a61290 0x559599a75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa70d19a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041afdf0 con 0x559599a69e10 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa70d19a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b1930 con 0x559599a61290 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa70d19a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b04c0 con 0x559599a608e0 2024-04-22T13:28:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a81c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48572/0 (socket says 172.21.15.5:48572) 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 -- 172.21.15.5:0/3188593284 learned_addr learned my addr 172.21.15.5:0/3188593284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa703fff640 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a608e0 0x559599a69510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a61290 0x559599a75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 -- 172.21.15.5:0/3188593284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a608e0 msgr2=0x559599a69510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a608e0 0x559599a69510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 -- 172.21.15.5:0/3188593284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a61290 msgr2=0x559599a75c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a61290 0x559599a75c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 -- 172.21.15.5:0/3188593284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559599a604a0 con 0x559599a69e10 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a61290 0x559599a75c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:54.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa708a65640 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a81c60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc037040 tx=0x7fa6fc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e500f6bbb3c221 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/3188593284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc039040 con 0x559599a69e10 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/3188593284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6fc003e20 con 0x559599a69e10 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.923+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/3188593284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc042450 con 0x559599a69e10 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 -- 172.21.15.5:0/3188593284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 msgr2=0x559599a81c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a81c60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc037040 tx=0x7fa6fc041000 comp rx=0 tx=0).stop 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 -- 172.21.15.5:0/3188593284 shutdown_connections 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a69e10 0x559599a81c60 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a61290 0x559599a75c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3188593284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a608e0 0x559599a69510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 -- 172.21.15.5:0/3188593284 >> 172.21.15.5:0/3188593284 conn(0x559599a5afb0 msgr2=0x559599a5a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 -- 172.21.15.5:0/3188593284 shutdown_connections 2024-04-22T13:28:54.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 -- 172.21.15.5:0/3188593284 wait complete. 2024-04-22T13:28:54.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.924+0000 7fa70d19a740 1 -- start start 2024-04-22T13:28:54.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 Processor -- start 2024-04-22T13:28:54.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a6dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 0x559599a6e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a69e10 0x559599a6e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b1260 con 0x559599a61290 2024-04-22T13:28:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b0b90 con 0x559599a608e0 2024-04-22T13:28:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041addf0 con 0x559599a69e10 2024-04-22T13:28:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa70d19a740 10 client.0 _ll_register_callbacks cb 0x7fa6f40d4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 0x559599a6e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 0x559599a6e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48576/0 (socket says 172.21.15.5:48576) 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 -- 172.21.15.5:0/3614897459 learned_addr learned my addr 172.21.15.5:0/3614897459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa708a65640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a69e10 0x559599a6e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa703fff640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a6dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 -- 172.21.15.5:0/3614897459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a69e10 msgr2=0x559599a6e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a69e10 0x559599a6e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 -- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 msgr2=0x559599a6dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a6dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa7037fe640 1 -- 172.21.15.5:0/3614897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559599adf770 con 0x559599a61290 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.925+0000 7fa703fff640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a6dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa7037fe640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 0x559599a6e3e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4004650 tx=0x7fa6f40d4760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3668a19ec4f3d012 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa6e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3614897459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b6020 con 0x559599a61290 2024-04-22T13:28:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3614897459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa6f4016dd0 con 0x559599a61290 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6fc003e20 con 0x559599a61290 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3614897459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6f4005310 con 0x559599a61290 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 10 client.16485 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559599ae0b10 con 0x559599a61290 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3614897459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6fc003e20 con 0x559599a61290 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3614897459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6f40ce610 con 0x559599a61290 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 20 client.16485 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599ae1190 0x559599ae35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 4 client.16485 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.926+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5595999cc790 con 0x559599ae1190 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.927+0000 7fa703fff640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599ae1190 0x559599ae35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.927+0000 7fa703fff640 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599ae1190 0x559599ae35b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7544cc321d5617 server_cookie=b8b234cc148a719c in_seq=0 out_seq=0 2024-04-22T13:28:54.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.927+0000 7fa6e37fe640 10 client.16485 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa6e37fe640 1 -- 172.21.15.5:0/3614897459 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5595999cc790 con 0x559599ae1190 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa6e37fe640 10 client.16485 handle_command_reply: tid=0 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 client.16485 shutdown 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 msgr2=0x559599a6e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 0x559599a6e3e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4004650 tx=0x7fa6f40d4760 comp rx=0 tx=0).stop 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 shutdown_connections 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559599ae1190 0x559599ae35b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559599a61290 0x559599a6e3e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559599a69e10 0x559599a6e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 --2- 172.21.15.5:0/3614897459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559599a608e0 0x559599a6dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa6e37fe640 0 client.16485 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.928+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 >> 172.21.15.5:0/3614897459 conn(0x559599a5afb0 msgr2=0x559599a66d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 shutdown_connections 2024-04-22T13:28:54.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70d19a740 1 -- 172.21.15.5:0/3614897459 wait complete. 2024-04-22T13:28:54.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70d19a740 20 client.16485 trim_cache size 0 max 16384 2024-04-22T13:28:54.933 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:28:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9d9f0772-7e2a-43cf-aaef-7c50e7349b31", 2024-04-22T13:28:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:28:54.933 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6f00a3d20 msgr2=0x7fa6f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6f00a3d20 0x7fa6f00a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc004af0 tx=0x7fa6fc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 msgr2=0x7fa7041a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa7041a0c10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e9c0 tx=0x7fa6f400ee80 comp rx=0 tx=0).stop 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 shutdown_connections 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa70410a6b0 0x7fa7041a0c10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa6f00a3d20 0x7fa6f00a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa70419a880 0x7fa70419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 --2- 172.21.15.5:0/3190600080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa704100af0 0x7fa7041a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 >> 172.21.15.5:0/3190600080 conn(0x7fa7040760d0 msgr2=0x7fa7041082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 shutdown_connections 2024-04-22T13:28:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:54.929+0000 7fa70a4c8640 1 -- 172.21.15.5:0/3190600080 wait complete. 2024-04-22T13:28:54.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:54.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:54 smithi049 ceph-mon[31379]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:55.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:54 smithi107 ceph-mon[31859]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 -- 172.21.15.5:0/2291960701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f641010a0 msgr2=0x7f6f64101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 --2- 172.21.15.5:0/2291960701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f641010a0 0x7f6f64101480 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009900 tx=0x7f6f5402bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 -- 172.21.15.5:0/2291960701 shutdown_connections 2024-04-22T13:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 --2- 172.21.15.5:0/2291960701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f64106170 0x7f6f6410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 --2- 172.21.15.5:0/2291960701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f64101a50 0x7f6f64105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 --2- 172.21.15.5:0/2291960701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f641010a0 0x7f6f64101480 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 -- 172.21.15.5:0/2291960701 >> 172.21.15.5:0/2291960701 conn(0x7f6f64077f40 msgr2=0x7f6f64078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 -- 172.21.15.5:0/2291960701 shutdown_connections 2024-04-22T13:28:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 -- 172.21.15.5:0/2291960701 wait complete. 2024-04-22T13:28:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 Processor -- start 2024-04-22T13:28:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.087+0000 7f6f6a398640 1 -- start start 2024-04-22T13:28:55.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f6a398640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f641010a0 0x7f6f6419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f6a398640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 0x7f6f6419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f6a398640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f64198610 0x7f6f64198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f6a398640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641110e0 con 0x7f6f64106170 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f6a398640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64110f60 con 0x7f6f641010a0 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f6a398640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64111260 con 0x7f6f64198610 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 0x7f6f6419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 0x7f6f6419e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48598/0 (socket says 172.21.15.5:48598) 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 -- 172.21.15.5:0/770913786 learned_addr learned my addr 172.21.15.5:0/770913786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f68935640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f64198610 0x7f6f64198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f63fff640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f641010a0 0x7f6f6419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 -- 172.21.15.5:0/770913786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f64198610 msgr2=0x7f6f64198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f64198610 0x7f6f64198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 -- 172.21.15.5:0/770913786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f641010a0 msgr2=0x7f6f6419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f641010a0 0x7f6f6419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 -- 172.21.15.5:0/770913786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f64199130 con 0x7f6f64106170 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.088+0000 7f6f637fe640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 0x7f6f6419e9a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9c0 tx=0x7f6f5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.089+0000 7f6f617fa640 1 -- 172.21.15.5:0/770913786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800cd80 con 0x7f6f64106170 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.089+0000 7f6f617fa640 1 -- 172.21.15.5:0/770913786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f580054d0 con 0x7f6f64106170 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.089+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f64199420 con 0x7f6f64106170 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.089+0000 7f6f617fa640 1 -- 172.21.15.5:0/770913786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58010430 con 0x7f6f64106170 2024-04-22T13:28:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.089+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f641a5670 con 0x7f6f64106170 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f617fa640 1 -- 172.21.15.5:0/770913786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6f58004fd0 con 0x7f6f64106170 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 Processor -- start 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 -- start start 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f617fa640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb8c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb99950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dccaf11e0 con 0x558dccb8c910 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.091+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558dcca8e390 con 0x558dccb99e90 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558dccaed1a0 con 0x558dccb8bf60 2024-04-22T13:28:55.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f68935640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb99950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f617fa640 1 -- 172.21.15.5:0/770913786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6f580d0c80 con 0x7f6f64106170 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb99950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48608/0 (socket says 172.21.15.5:48608) 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 -- 172.21.15.5:0/1826846343 learned_addr learned my addr 172.21.15.5:0/1826846343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f68935640 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f63fff640 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb8c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 -- 172.21.15.5:0/1826846343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 msgr2=0x558dccb8c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb8c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 -- 172.21.15.5:0/1826846343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 msgr2=0x558dccb9c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb9c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 -- 172.21.15.5:0/1826846343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dcc8cd7d0 con 0x558dccb8c910 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f68935640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009fd0 tx=0x7f6f5c013b60 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f637fe640 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb99950 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6f580cea00 tx=0x7f6f58099020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c8af4bf1b8e6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f4a7fc640 1 -- 172.21.15.5:0/1826846343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5807a270 con 0x558dccb8c910 2024-04-22T13:28:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f4a7fc640 1 -- 172.21.15.5:0/1826846343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f580046c0 con 0x558dccb8c910 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.092+0000 7f6f4a7fc640 1 -- 172.21.15.5:0/1826846343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5807be20 con 0x558dccb8c910 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 -- 172.21.15.5:0/1826846343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 msgr2=0x558dccb99950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb99950 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6f580cea00 tx=0x7f6f58099020 comp rx=0 tx=0).stop 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 -- 172.21.15.5:0/1826846343 shutdown_connections 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb99950 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb9c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/1826846343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb8c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 -- 172.21.15.5:0/1826846343 >> 172.21.15.5:0/1826846343 conn(0x558dccb86cb0 msgr2=0x558dccb87090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 -- 172.21.15.5:0/1826846343 shutdown_connections 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 -- 172.21.15.5:0/1826846343 wait complete. 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.093+0000 7f6f6cf81740 1 -- start start 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 Processor -- start 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb97920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dccb9cc50 con 0x558dccb8c910 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558dccaaa2f0 con 0x558dccb99e90 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558dccb9cdd0 con 0x558dccb8bf60 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f6cf81740 10 client.0 _ll_register_callbacks cb 0x558dccab7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48618/0 (socket says 172.21.15.5:48618) 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 -- 172.21.15.5:0/19245225 learned_addr learned my addr 172.21.15.5:0/19245225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f63fff640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f68935640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb97920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 -- 172.21.15.5:0/19245225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 msgr2=0x558dccb96ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb96ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 -- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 msgr2=0x558dccb97920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb97920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 -- 172.21.15.5:0/19245225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dcc8033f0 con 0x558dccb8c910 2024-04-22T13:28:55.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f637fe640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb973e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6f580a5390 tx=0x7f6f580a58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aad4eece5a4f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.094+0000 7f6f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f437fe640 1 -- 172.21.15.5:0/19245225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5807a090 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f437fe640 1 -- 172.21.15.5:0/19245225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6f5807ba70 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558dccafec00 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f437fe640 1 -- 172.21.15.5:0/19245225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f5809d350 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 10 client.16497 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558dcc7c30f0 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f437fe640 1 -- 172.21.15.5:0/19245225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558dccafec00 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f437fe640 1 -- 172.21.15.5:0/19245225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6f5809ceb0 con 0x558dccb8c910 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 10 client.16497 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 20 client.16497 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccba9d00 0x558dccbac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 4 client.16497 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dccaf8a70 con 0x558dccba9d00 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.095+0000 7f6f63fff640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccba9d00 0x558dccbac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.096+0000 7f6f437fe640 10 client.16497 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.096+0000 7f6f63fff640 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccba9d00 0x558dccbac0f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996f7907e52a7d3e server_cookie=e0684f063e9aa20e in_seq=0 out_seq=0 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.096+0000 7f6f437fe640 1 -- 172.21.15.5:0/19245225 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dccaf8a70 con 0x558dccba9d00 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.096+0000 7f6f437fe640 10 client.16497 handle_command_reply: tid=0 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.096+0000 7f6f6cf81740 1 client.16497 shutdown 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 msgr2=0x558dccb973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb973e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6f580a5390 tx=0x7f6f580a58c0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 shutdown_connections 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccba9d00 0x558dccbac0f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8c910 0x558dccb973e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb99e90 0x558dccb97920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f437fe640 0 client.16497 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:55.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/19245225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8bf60 0x558dccb96ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 >> 172.21.15.5:0/19245225 conn(0x558dccb86cb0 msgr2=0x558dccb88a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:55.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 shutdown_connections 2024-04-22T13:28:55.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 1 -- 172.21.15.5:0/19245225 wait complete. 2024-04-22T13:28:55.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.097+0000 7f6f6cf81740 20 client.16497 trim_cache size 0 max 16384 2024-04-22T13:28:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 Processor -- start 2024-04-22T13:28:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 -- start start 2024-04-22T13:28:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 0x558dccb995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8c910 0x558dccb87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb99e90 0x558dccb96000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641ab9a0 con 0x558dccb8bf60 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641ad4e0 con 0x558dccb8c910 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641ac070 con 0x558dccb99e90 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 0x558dccb995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 0x558dccb995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48634/0 (socket says 172.21.15.5:48634) 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 -- 172.21.15.5:0/924817128 learned_addr learned my addr 172.21.15.5:0/924817128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 -- 172.21.15.5:0/924817128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb99e90 msgr2=0x558dccb96000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb99e90 0x558dccb96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 -- 172.21.15.5:0/924817128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8c910 msgr2=0x558dccb87090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8c910 0x558dccb87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.115+0000 7f6f63fff640 1 -- 172.21.15.5:0/924817128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dccb8b670 con 0x558dccb8bf60 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f63fff640 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 0x558dccb995d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009230 tx=0x7f6f5402c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8607f5edd127f42d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f49ffb640 1 -- 172.21.15.5:0/924817128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54004380 con 0x558dccb8bf60 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f49ffb640 1 -- 172.21.15.5:0/924817128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5402cc90 con 0x558dccb8bf60 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f49ffb640 1 -- 172.21.15.5:0/924817128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54004920 con 0x558dccb8bf60 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 -- 172.21.15.5:0/924817128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 msgr2=0x558dccb995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 0x558dccb995d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009230 tx=0x7f6f5402c1f0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 -- 172.21.15.5:0/924817128 shutdown_connections 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb99e90 0x558dccb96000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8c910 0x558dccb87090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/924817128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb8bf60 0x558dccb995d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.116+0000 7f6f6cf81740 1 -- 172.21.15.5:0/924817128 >> 172.21.15.5:0/924817128 conn(0x558dccb86cb0 msgr2=0x558dccb88100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 -- 172.21.15.5:0/924817128 shutdown_connections 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 -- 172.21.15.5:0/924817128 wait complete. 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 -- start start 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 Processor -- start 2024-04-22T13:28:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8bf60 0x558dccbaf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8c910 0x558dccbaca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 0x558dccbad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641ace10 con 0x558dccb99e90 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641ac740 con 0x558dccb8bf60 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641a9980 con 0x558dccb8c910 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f6cf81740 10 client.0 _ll_register_callbacks cb 0x7f6f58098ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f68935640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 0x558dccbad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f68935640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 0x558dccbad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48642/0 (socket says 172.21.15.5:48642) 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.117+0000 7f6f68935640 1 -- 172.21.15.5:0/849224081 learned_addr learned my addr 172.21.15.5:0/849224081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f63fff640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8bf60 0x558dccbaf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f637fe640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8c910 0x558dccbaca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f68935640 1 -- 172.21.15.5:0/849224081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8c910 msgr2=0x558dccbaca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f68935640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8c910 0x558dccbaca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f68935640 1 -- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8bf60 msgr2=0x558dccbaf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f68935640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8bf60 0x558dccbaf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f68935640 1 -- 172.21.15.5:0/849224081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dccc0eee0 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f68935640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 0x558dccbad180 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00e660 tx=0x7f6f5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c2bd30af1cd958 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f437fe640 1 -- 172.21.15.5:0/849224081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c011e60 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f437fe640 1 -- 172.21.15.5:0/849224081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6f5c004d20 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5402cc90 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f437fe640 1 -- 172.21.15.5:0/849224081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f5c020610 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f6cf81740 10 client.16503 fetch_fsmap learned FSMap version 27 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558dccc10090 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.118+0000 7f6f437fe640 1 -- 172.21.15.5:0/849224081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5402cc90 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f437fe640 1 -- 172.21.15.5:0/849224081 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6f5c01f070 con 0x558dccb99e90 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f6cf81740 10 client.16503 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:28:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f6cf81740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f6cf81740 20 client.16503 populate_metadata read hostname 'smithi005' 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccc107b0 0x558dccc12bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f6cf81740 4 client.16503 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dcca74590 con 0x558dccc107b0 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f63fff640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccc107b0 0x558dccc12bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f63fff640 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccc107b0 0x558dccc12bd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e7e2d5801c1d71 server_cookie=484f7a309281744a in_seq=0 out_seq=0 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.119+0000 7f6f437fe640 10 client.16503 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f437fe640 1 -- 172.21.15.5:0/849224081 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558dcca74590 con 0x558dccc107b0 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f437fe640 10 client.16503 handle_command_reply: tid=0 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 client.16503 shutdown 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 msgr2=0x558dccbad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 0x558dccbad180 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00e660 tx=0x7f6f5c004a40 comp rx=0 tx=0).stop 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 shutdown_connections 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558dccc107b0 0x558dccc12bd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dccb99e90 0x558dccbad180 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dccb8c910 0x558dccbaca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f437fe640 0 client.16503 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 --2- 172.21.15.5:0/849224081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558dccb8bf60 0x558dccbaf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 >> 172.21.15.5:0/849224081 conn(0x558dccb86cb0 msgr2=0x558dccb85d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 shutdown_connections 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 1 -- 172.21.15.5:0/849224081 wait complete. 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.120+0000 7f6f6cf81740 20 client.16503 trim_cache size 0 max 16384 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:28:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "9d9f0772-7e2a-43cf-aaef-7c50e7349b31": { 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9d9f0772-7e2a-43cf-aaef-7c50e7349b31", 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:28:55.123 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6f4c0a3d20 msgr2=0x7f6f4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009fd0 tx=0x7f6f5c013b60 comp rx=0 tx=0).stop 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 msgr2=0x7f6f6419e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 0x7f6f6419e9a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9c0 tx=0x7f6f5800ee80 comp rx=0 tx=0).stop 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 shutdown_connections 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f64106170 0x7f6f6419e9a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f64198610 0x7f6f64198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 --2- 172.21.15.5:0/770913786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f641010a0 0x7f6f6419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 >> 172.21.15.5:0/770913786 conn(0x7f6f64077f40 msgr2=0x7f6f6406aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 shutdown_connections 2024-04-22T13:28:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:28:55.121+0000 7f6f6a398640 1 -- 172.21.15.5:0/770913786 wait complete. 2024-04-22T13:28:55.127 INFO:tasks.cephfs.filesystem:scrub status for tag:9d9f0772-7e2a-43cf-aaef-7c50e7349b31 - {'path': '/', 'tag': '9d9f0772-7e2a-43cf-aaef-7c50e7349b31', 'options': 'recursive,force'} 2024-04-22T13:28:56.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:56 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:28:56.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:56 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:56.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:56 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:28:56.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:56 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:56.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:56 smithi005 ceph-mon[28006]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-22T13:28:56.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:56 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:28:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:56 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:28:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:56 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:57.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:56 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:28:57.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:56 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:57.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:56 smithi049 ceph-mon[31379]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-22T13:28:57.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:56 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:28:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:56 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:28:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:56 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:56 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:28:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:56 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:56 smithi107 ceph-mon[31859]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-22T13:28:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:56 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:28:58.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:28:58 smithi005 ceph-mon[28006]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-22T13:28:59.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:28:58 smithi049 ceph-mon[31379]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-22T13:28:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:28:58 smithi107 ceph-mon[31859]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-22T13:29:00.836 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:00 smithi107 ceph-mon[31859]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-22T13:29:01.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:00 smithi049 ceph-mon[31379]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-22T13:29:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:00 smithi005 ceph-mon[28006]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-22T13:29:02.823 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:02 smithi049 ceph-mon[31379]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:29:03.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:02 smithi107 ceph-mon[31859]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:29:03.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:02 smithi005 ceph-mon[28006]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:29:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 -- 172.21.15.5:0/1842904426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6381031c0 msgr2=0x7fe6381035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 --2- 172.21.15.5:0/1842904426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6381031c0 0x7fe6381035a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe628009900 tx=0x7fe62802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 -- 172.21.15.5:0/1842904426 shutdown_connections 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 --2- 172.21.15.5:0/1842904426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63810c590 0x7fe63810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 --2- 172.21.15.5:0/1842904426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe638103ae0 0x7fe63810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 --2- 172.21.15.5:0/1842904426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6381031c0 0x7fe6381035a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 -- 172.21.15.5:0/1842904426 >> 172.21.15.5:0/1842904426 conn(0x7fe638077f70 msgr2=0x7fe638078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 -- 172.21.15.5:0/1842904426 shutdown_connections 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.463+0000 7fe63e7db640 1 -- 172.21.15.5:0/1842904426 wait complete. 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 Processor -- start 2024-04-22T13:29:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 -- start start 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6381031c0 0x7fe63819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 0x7fe63819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe63810c590 0x7fe6381985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381121e0 con 0x7fe638103ae0 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe638112060 con 0x7fe63810c590 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe63e7db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe638112360 con 0x7fe6381031c0 2024-04-22T13:29:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.464+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 0x7fe63819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 0x7fe63819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34526/0 (socket says 172.21.15.5:34526) 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 -- 172.21.15.5:0/2680710150 learned_addr learned my addr 172.21.15.5:0/2680710150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe63cd78640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe63810c590 0x7fe6381985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe637fff640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6381031c0 0x7fe63819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 -- 172.21.15.5:0/2680710150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6381031c0 msgr2=0x7fe63819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6381031c0 0x7fe63819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 -- 172.21.15.5:0/2680710150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe63810c590 msgr2=0x7fe6381985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe63810c590 0x7fe6381985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 -- 172.21.15.5:0/2680710150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe638198d60 con 0x7fe638103ae0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe63cd78640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe63810c590 0x7fe6381985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6377fe640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 0x7fe63819e970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e9f0 tx=0x7fe62c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6357fa640 1 -- 172.21.15.5:0/2680710150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c00cd80 con 0x7fe638103ae0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6357fa640 1 -- 172.21.15.5:0/2680710150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c0108e0 con 0x7fe638103ae0 2024-04-22T13:29:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe6357fa640 1 -- 172.21.15.5:0/2680710150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c018980 con 0x7fe638103ae0 2024-04-22T13:29:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.465+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe638199050 con 0x7fe638103ae0 2024-04-22T13:29:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.466+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6381a52f0 con 0x7fe638103ae0 2024-04-22T13:29:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6357fa640 1 -- 172.21.15.5:0/2680710150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe62c010430 con 0x7fe638103ae0 2024-04-22T13:29:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 Processor -- start 2024-04-22T13:29:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 -- start start 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x555942030620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942030fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x55594203c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555941f958d0 con 0x55594203a0e0 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555941f32e80 con 0x555942030b60 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6413d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555941f91890 con 0x5559420318d0 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.468+0000 7fe6357fa640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6200a3d10 0x7fe6200a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x55594203c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe63cd78640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6200a3d10 0x7fe6200a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x55594203c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34540/0 (socket says 172.21.15.5:34540) 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 -- 172.21.15.5:0/626588796 learned_addr learned my addr 172.21.15.5:0/626588796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe63cd78640 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x555942030620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe637fff640 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942030fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6357fa640 1 -- 172.21.15.5:0/2680710150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe62c0d0c10 con 0x7fe638103ae0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 -- 172.21.15.5:0/626588796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 msgr2=0x555942030620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x555942030620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 -- 172.21.15.5:0/626588796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 msgr2=0x555942030fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942030fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 -- 172.21.15.5:0/626588796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555941f54320 con 0x55594203a0e0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe637fff640 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942030fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe63cd78640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6200a3d10 0x7fe6200a61d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe63806a850 tx=0x7fe630009db0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe6377fe640 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x55594203c4d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0ce3f0 tx=0x7fe62c06e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770c5b74bc61112e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe61e7fc640 1 -- 172.21.15.5:0/626588796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c09cbe0 con 0x55594203a0e0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe61e7fc640 1 -- 172.21.15.5:0/626588796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c0a0f90 con 0x55594203a0e0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.469+0000 7fe61e7fc640 1 -- 172.21.15.5:0/626588796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c099250 con 0x55594203a0e0 2024-04-22T13:29:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 -- 172.21.15.5:0/626588796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 msgr2=0x55594203c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x55594203c4d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0ce3f0 tx=0x7fe62c06e6e0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 -- 172.21.15.5:0/626588796 shutdown_connections 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x55594203c4d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942030fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 --2- 172.21.15.5:0/626588796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x555942030620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 -- 172.21.15.5:0/626588796 >> 172.21.15.5:0/626588796 conn(0x555941ff7c90 msgr2=0x55594202b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 -- 172.21.15.5:0/626588796 shutdown_connections 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 -- 172.21.15.5:0/626588796 wait complete. 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.470+0000 7fe6413d0740 1 -- start start 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 Processor -- start 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942039ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 0x555942036e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942037450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55594203ce10 con 0x5559420318d0 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555941f1deb0 con 0x555942030b60 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55594203cf90 con 0x55594203a0e0 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6413d0740 10 client.0 _ll_register_callbacks cb 0x555941f4ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 0x555942036e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 0x555942036e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34556/0 (socket says 172.21.15.5:34556) 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 -- 172.21.15.5:0/3236050498 learned_addr learned my addr 172.21.15.5:0/3236050498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe63cd78640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942037450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe637fff640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942039ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 -- 172.21.15.5:0/3236050498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 msgr2=0x555942037450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942037450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 -- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 msgr2=0x555942039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942039ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.471+0000 7fe6377fe640 1 -- 172.21.15.5:0/3236050498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555941ca8210 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe637fff640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942039ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6377fe640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 0x555942036e40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e9c0 tx=0x7fe62c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18694c6cf08c3933 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6177fe640 1 -- 172.21.15.5:0/3236050498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0a5850 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6177fe640 1 -- 172.21.15.5:0/3236050498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe62c09c030 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe62c0a0f90 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6177fe640 1 -- 172.21.15.5:0/3236050498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe62c09c270 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 10 client.16515 fetch_fsmap learned FSMap version 27 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555941d74620 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6177fe640 1 -- 172.21.15.5:0/3236050498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c0a0f90 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6177fe640 1 -- 172.21.15.5:0/3236050498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe62c0ceae0 con 0x5559420318d0 2024-04-22T13:29:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 10 client.16515 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 20 client.16515 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555942049fd0 0x55594204c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 4 client.16515 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.472+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555941fa9880 con 0x555942049fd0 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.473+0000 7fe637fff640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555942049fd0 0x55594204c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.473+0000 7fe6177fe640 10 client.16515 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.473+0000 7fe637fff640 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555942049fd0 0x55594204c3c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d250f20239a33a server_cookie=f23561b35470e2f8 in_seq=0 out_seq=0 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6177fe640 1 -- 172.21.15.5:0/3236050498 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555941fa9880 con 0x555942049fd0 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6177fe640 10 client.16515 handle_command_reply: tid=0 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 client.16515 shutdown 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 msgr2=0x555942036e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 0x555942036e40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e9c0 tx=0x7fe62c000bc0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 shutdown_connections 2024-04-22T13:29:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555942049fd0 0x55594204c3c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559420318d0 0x555942036e40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942037450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6177fe640 0 client.16515 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3236050498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x555942039ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.474+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 >> 172.21.15.5:0/3236050498 conn(0x555941ff7c90 msgr2=0x55594202d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.475+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 shutdown_connections 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.475+0000 7fe6413d0740 1 -- 172.21.15.5:0/3236050498 wait complete. 2024-04-22T13:29:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.475+0000 7fe6413d0740 20 client.16515 trim_cache size 0 max 16384 2024-04-22T13:29:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 Processor -- start 2024-04-22T13:29:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 -- start start 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 0x555942039570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559420318d0 0x555942039ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942031fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381adaa0 con 0x555942030b60 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381af5e0 con 0x5559420318d0 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6413d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381ae170 con 0x55594203a0e0 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 0x555942039570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 0x555942039570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34566/0 (socket says 172.21.15.5:34566) 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 -- 172.21.15.5:0/1866225976 learned_addr learned my addr 172.21.15.5:0/1866225976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe6377fe640 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559420318d0 0x555942039ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe63cd78640 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942031fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 -- 172.21.15.5:0/1866225976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 msgr2=0x555942031fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942031fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 -- 172.21.15.5:0/1866225976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559420318d0 msgr2=0x555942039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559420318d0 0x555942039ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 -- 172.21.15.5:0/1866225976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594203c910 con 0x555942030b60 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.490+0000 7fe637fff640 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 0x555942039570 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe628009900 tx=0x7fe628009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93316d94d75f4e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe61dffb640 1 -- 172.21.15.5:0/1866225976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62802c880 con 0x555942030b60 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe61dffb640 1 -- 172.21.15.5:0/1866225976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe628048070 con 0x555942030b60 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe61dffb640 1 -- 172.21.15.5:0/1866225976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628043440 con 0x555942030b60 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 -- 172.21.15.5:0/1866225976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 msgr2=0x555942039570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 0x555942039570 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe628009900 tx=0x7fe628009260 comp rx=0 tx=0).stop 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 -- 172.21.15.5:0/1866225976 shutdown_connections 2024-04-22T13:29:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55594203a0e0 0x555942031fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559420318d0 0x555942039ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 --2- 172.21.15.5:0/1866225976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555942030b60 0x555942039570 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 -- 172.21.15.5:0/1866225976 >> 172.21.15.5:0/1866225976 conn(0x555941ff7c90 msgr2=0x555942035710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 -- 172.21.15.5:0/1866225976 shutdown_connections 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.491+0000 7fe6413d0740 1 -- 172.21.15.5:0/1866225976 wait complete. 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 -- start start 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 Processor -- start 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x55594204f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x55594204f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x5559420332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381aef10 con 0x55594203a0e0 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381ae840 con 0x555942030b60 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381aba80 con 0x5559420318d0 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe6413d0740 10 client.0 _ll_register_callbacks cb 0x7fe630012750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe63cd78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x5559420332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe63cd78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x5559420332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34572/0 (socket says 172.21.15.5:34572) 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.492+0000 7fe63cd78640 1 -- 172.21.15.5:0/3328901415 learned_addr learned my addr 172.21.15.5:0/3328901415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6377fe640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x55594204f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe637fff640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x55594204f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe63cd78640 1 -- 172.21.15.5:0/3328901415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 msgr2=0x55594204f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe63cd78640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x55594204f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe63cd78640 1 -- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 msgr2=0x55594204f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe63cd78640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x55594204f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe63cd78640 1 -- 172.21.15.5:0/3328901415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555942033ae0 con 0x55594203a0e0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe63cd78640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x5559420332b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe630010de0 tx=0x7fe63001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7620b742e36b679 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6177fe640 1 -- 172.21.15.5:0/3328901415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63002e060 con 0x55594203a0e0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6177fe640 1 -- 172.21.15.5:0/3328901415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe63001b040 con 0x55594203a0e0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe628048070 con 0x55594203a0e0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6177fe640 1 -- 172.21.15.5:0/3328901415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe630010e10 con 0x55594203a0e0 2024-04-22T13:29:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6413d0740 10 client.16521 fetch_fsmap learned FSMap version 27 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.493+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559420afc50 con 0x55594203a0e0 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6177fe640 1 -- 172.21.15.5:0/3328901415 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe628048070 con 0x55594203a0e0 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6177fe640 1 -- 172.21.15.5:0/3328901415 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe63001f050 con 0x55594203a0e0 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6413d0740 10 client.16521 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6413d0740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6413d0740 20 client.16521 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5559420b0400 0x5559420b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6413d0740 4 client.16521 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555941f353d0 con 0x5559420b0400 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe637fff640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5559420b0400 0x5559420b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe637fff640 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5559420b0400 0x5559420b2820 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0d991bef47f799 server_cookie=c16a4e84ceebd6af in_seq=0 out_seq=0 2024-04-22T13:29:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:03.494+0000 7fe6177fe640 10 client.16521 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:05.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:04 smithi049 ceph-mon[31379]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:29:05.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:04 smithi107 ceph-mon[31859]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:29:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:04 smithi005 ceph-mon[28006]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6177fe640 1 -- 172.21.15.5:0/3328901415 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555941f353d0 con 0x5559420b0400 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6177fe640 10 client.16521 handle_command_reply: tid=0 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 client.16521 shutdown 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 msgr2=0x5559420332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x5559420332b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe630010de0 tx=0x7fe63001c000 comp rx=0 tx=0).stop 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 shutdown_connections 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5559420b0400 0x5559420b2820 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55594203a0e0 0x5559420332b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559420318d0 0x55594204f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 --2- 172.21.15.5:0/3328901415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555942030b60 0x55594204f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6177fe640 0 client.16521 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 >> 172.21.15.5:0/3328901415 conn(0x555941ff7c90 msgr2=0x5559420354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 shutdown_connections 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 1 -- 172.21.15.5:0/3328901415 wait complete. 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.580+0000 7fe6413d0740 20 client.16521 trim_cache size 0 max 16384 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:29:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:29:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6200a3d10 msgr2=0x7fe6200a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6200a3d10 0x7fe6200a61d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe63806a850 tx=0x7fe630009db0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 msgr2=0x7fe63819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 0x7fe63819e970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e9f0 tx=0x7fe62c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 shutdown_connections 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe638103ae0 0x7fe63819e970 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe6200a3d10 0x7fe6200a61d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe63810c590 0x7fe6381985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 --2- 172.21.15.5:0/2680710150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6381031c0 0x7fe63819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 >> 172.21.15.5:0/2680710150 conn(0x7fe638077f70 msgr2=0x7fe6380ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 shutdown_connections 2024-04-22T13:29:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:05.581+0000 7fe63e7db640 1 -- 172.21.15.5:0/2680710150 wait complete. 2024-04-22T13:29:05.925 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: *** Caught signal (Segmentation fault) ** 2024-04-22T13:29:05.925 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: in thread 7f16b39b0700 thread_name:md_log_replay 2024-04-22T13:29:05.925 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:29:05.925 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f16c4d71d20] 2024-04-22T13:29:05.925 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55982df5638f] 2024-04-22T13:29:05.925 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55982df59b23] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 4: (MDLog::_replay_thread()+0x617) [0x55982dfb4647] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55982dfbdd31] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f16c4d671ca] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 7: clone() 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.589+0000 7f16b39b0700 -1 *** Caught signal (Segmentation fault) ** 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: in thread 7f16b39b0700 thread_name:md_log_replay 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f16c4d71d20] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55982df5638f] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55982df59b23] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 4: (MDLog::_replay_thread()+0x617) [0x55982dfb4647] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55982dfbdd31] 2024-04-22T13:29:05.926 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f16c4d671ca] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 7: clone() 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 0> 2024-04-22T13:29:05.589+0000 7f16b39b0700 -1 *** Caught signal (Segmentation fault) ** 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: in thread 7f16b39b0700 thread_name:md_log_replay 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f16c4d71d20] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55982df5638f] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55982df59b23] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 4: (MDLog::_replay_thread()+0x617) [0x55982dfb4647] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55982dfbdd31] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f16c4d671ca] 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 7: clone() 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 0> 2024-04-22T13:29:05.589+0000 7f16b39b0700 -1 *** Caught signal (Segmentation fault) ** 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: in thread 7f16b39b0700 thread_name:md_log_replay 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f16c4d71d20] 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55982df5638f] 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55982df59b23] 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 4: (MDLog::_replay_thread()+0x617) [0x55982dfb4647] 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55982dfbdd31] 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f16c4d671ca] 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 7: clone() 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[62965]: 2024-04-22T13:29:05.928 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:05 smithi107 podman[66643]: 2024-04-22 13:29:05.654523371 +0000 UTC m=+0.022433121 container died 31f1c4059595dc0e0ebad726dbcd41c3c298e91881fa6278fa2135b6a83f811b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T13:29:06.534 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:06 smithi107 podman[66643]: 2024-04-22 13:29:06.255098099 +0000 UTC m=+0.623007845 container remove 31f1c4059595dc0e0ebad726dbcd41c3c298e91881fa6278fa2135b6a83f811b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:29:06.534 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:06 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-22T13:29:06.534 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:06 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-22T13:29:06.534 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:06 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.014s CPU time. 2024-04-22T13:29:06.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:06.994 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:06 smithi005 ceph-mon[28006]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:29:07.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:06 smithi049 ceph-mon[31379]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:29:07.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:06 smithi107 ceph-mon[31859]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 -- 172.21.15.5:0/316166447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa050107270 msgr2=0x7fa05010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 --2- 172.21.15.5:0/316166447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa050107270 0x7fa05010ee40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa048009ca0 tx=0x7fa04802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 -- 172.21.15.5:0/316166447 shutdown_connections 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 --2- 172.21.15.5:0/316166447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa050107270 0x7fa05010ee40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 --2- 172.21.15.5:0/316166447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa050102b50 0x7fa050106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 --2- 172.21.15.5:0/316166447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0501021a0 0x7fa050102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 -- 172.21.15.5:0/316166447 >> 172.21.15.5:0/316166447 conn(0x7fa0500793e0 msgr2=0x7fa0500797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 -- 172.21.15.5:0/316166447 shutdown_connections 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.088+0000 7fa0576c0640 1 -- 172.21.15.5:0/316166447 wait complete. 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 Processor -- start 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 -- start start 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 0x7fa0501986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa050107270 0x7fa050198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa05019cfa0 0x7fa05019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501121e0 con 0x7fa0501021a0 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa050112060 con 0x7fa05019cfa0 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa0576c0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa050112360 con 0x7fa050107270 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 0x7fa0501986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 0x7fa0501986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34584/0 (socket says 172.21.15.5:34584) 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 -- 172.21.15.5:0/435807430 learned_addr learned my addr 172.21.15.5:0/435807430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa054c5b640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa050107270 0x7fa050198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa055c5d640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa05019cfa0 0x7fa05019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 -- 172.21.15.5:0/435807430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa050107270 msgr2=0x7fa050198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa050107270 0x7fa050198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 -- 172.21.15.5:0/435807430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa05019cfa0 msgr2=0x7fa05019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa05019cfa0 0x7fa05019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 -- 172.21.15.5:0/435807430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05019da90 con 0x7fa0501021a0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa054c5b640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa050107270 0x7fa050198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.089+0000 7fa05545c640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 0x7fa0501986e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9f0 tx=0x7fa04c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.090+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00cd80 con 0x7fa0501021a0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.090+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c0054d0 con 0x7fa0501021a0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.090+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05019dd80 con 0x7fa0501021a0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.090+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010690 con 0x7fa0501021a0 2024-04-22T13:29:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.090+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0501a5520 con 0x7fa0501021a0 2024-04-22T13:29:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.091+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa050069250 con 0x7fa0501021a0 2024-04-22T13:29:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.093+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa04c004e50 con 0x7fa0501021a0 2024-04-22T13:29:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.093+0000 7fa0467fc640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.093+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa04c0d0f70 con 0x7fa0501021a0 2024-04-22T13:29:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.093+0000 7fa054c5b640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.094+0000 7fa054c5b640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa0501017f0 tx=0x7fa040005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.097+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa04c053ff0 con 0x7fa0501021a0 2024-04-22T13:29:07.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.327+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa050199a30 con 0x7fa0501021a0 2024-04-22T13:29:07.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.328+0000 7fa0467fc640 1 -- 172.21.15.5:0/435807430 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7fa04c09d640 con 0x7fa0501021a0 2024-04-22T13:29:07.324 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:29:07.324 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:29:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:29:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa03c0a3d20 msgr2=0x7fa03c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa0501017f0 tx=0x7fa040005f90 comp rx=0 tx=0).stop 2024-04-22T13:29:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 msgr2=0x7fa0501986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 0x7fa0501986e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9f0 tx=0x7fa04c00edf0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 shutdown_connections 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa05019cfa0 0x7fa05019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa050107270 0x7fa050198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 --2- 172.21.15.5:0/435807430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501021a0 0x7fa0501986e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 >> 172.21.15.5:0/435807430 conn(0x7fa0500793e0 msgr2=0x7fa0500ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 shutdown_connections 2024-04-22T13:29:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.330+0000 7fa0576c0640 1 -- 172.21.15.5:0/435807430 wait complete. 2024-04-22T13:29:07.338 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db --set-id ce13b8c6 --release --await' 2024-04-22T13:29:07.338 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db --set-id ce13b8c6 --release --await 2024-04-22T13:29:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 -- 172.21.15.5:0/3182361523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 msgr2=0x7fc89410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 --2- 172.21.15.5:0/3182361523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89410e9a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009ca0 tx=0x7fc88c02c180 comp rx=0 tx=0).stop 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 -- 172.21.15.5:0/3182361523 shutdown_connections 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 --2- 172.21.15.5:0/3182361523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89410e9a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 --2- 172.21.15.5:0/3182361523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 0x7fc89410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 --2- 172.21.15.5:0/3182361523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8941031c0 0x7fc8941035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 -- 172.21.15.5:0/3182361523 >> 172.21.15.5:0/3182361523 conn(0x7fc894078000 msgr2=0x7fc894078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 -- 172.21.15.5:0/3182361523 shutdown_connections 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.491+0000 7fc899de6640 1 -- 172.21.15.5:0/3182361523 wait complete. 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 Processor -- start 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 -- start start 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8941031c0 0x7fc894198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 0x7fc894198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc894112170 con 0x7fc89410c500 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc894111ff0 con 0x7fc8941031c0 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc899de6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941122f0 con 0x7fc894103ae0 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc893fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc893fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34614/0 (socket says 172.21.15.5:34614) 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc893fff640 1 -- 172.21.15.5:0/700343296 learned_addr learned my addr 172.21.15.5:0/700343296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.492+0000 7fc8937fe640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8941031c0 0x7fc894198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc892ffd640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 0x7fc894198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc893fff640 1 -- 172.21.15.5:0/700343296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 msgr2=0x7fc894198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc893fff640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 0x7fc894198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc893fff640 1 -- 172.21.15.5:0/700343296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8941031c0 msgr2=0x7fc894198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc893fff640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8941031c0 0x7fc894198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc893fff640 1 -- 172.21.15.5:0/700343296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89419d680 con 0x7fc89410c500 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc892ffd640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 0x7fc894198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc893fff640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89419cf50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc88c02c790 tx=0x7fc88c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc890ff9640 1 -- 172.21.15.5:0/700343296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c039040 con 0x7fc89410c500 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc890ff9640 1 -- 172.21.15.5:0/700343296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c044070 con 0x7fc89410c500 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc890ff9640 1 -- 172.21.15.5:0/700343296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c032850 con 0x7fc89410c500 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89419d910 con 0x7fc89410c500 2024-04-22T13:29:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.493+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8941a5200 con 0x7fc89410c500 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.495+0000 7fc890ff9640 1 -- 172.21.15.5:0/700343296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc88c004240 con 0x7fc89410c500 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 Processor -- start 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 -- start start 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 0x5564eb8036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb80c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb80ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5564eb768500 con 0x5564eb803310 2024-04-22T13:29:07.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5564eb70e6a0 con 0x5564eb80c810 2024-04-22T13:29:07.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5564eb7644c0 con 0x5564eb803c30 2024-04-22T13:29:07.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc890ff9640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc87c0a3da0 0x7fc87c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 0x5564eb8036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 0x5564eb8036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34626/0 (socket says 172.21.15.5:34626) 2024-04-22T13:29:07.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc892ffd640 1 -- 172.21.15.5:0/2405419835 learned_addr learned my addr 172.21.15.5:0/2405419835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb80c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc893fff640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc87c0a3da0 0x7fc87c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc892ffd640 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb80ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc890ff9640 1 -- 172.21.15.5:0/700343296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc88c0f2be0 con 0x7fc89410c500 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc8937fe640 1 -- 172.21.15.5:0/2405419835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 msgr2=0x5564eb80ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb80ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.496+0000 7fc8937fe640 1 -- 172.21.15.5:0/2405419835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 msgr2=0x5564eb8036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 0x5564eb8036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc8937fe640 1 -- 172.21.15.5:0/2405419835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564eb72d5e0 con 0x5564eb803c30 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc892ffd640 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 0x5564eb8036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb80c2d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc88400bbf0 tx=0x7fc8840107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a8f1180a38c91c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc893fff640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc87c0a3da0 0x7fc87c0a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0be920 tx=0x7fc88c0b40b0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc879ffb640 1 -- 172.21.15.5:0/2405419835 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884004030 con 0x5564eb803c30 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc879ffb640 1 -- 172.21.15.5:0/2405419835 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8840035f0 con 0x5564eb803c30 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.497+0000 7fc879ffb640 1 -- 172.21.15.5:0/2405419835 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884011680 con 0x5564eb803c30 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2405419835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 msgr2=0x5564eb80c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb80c2d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc88400bbf0 tx=0x7fc8840107a0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2405419835 shutdown_connections 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb80ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb80c2d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2405419835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803310 0x5564eb8036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2405419835 >> 172.21.15.5:0/2405419835 conn(0x5564eb7fda40 msgr2=0x5564eb7fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2405419835 shutdown_connections 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2405419835 wait complete. 2024-04-22T13:29:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 -- start start 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 Processor -- start 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803310 0x5564eb809670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb809bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.498+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb80a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5564eb80f600 con 0x5564eb80c810 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5564eb721a60 con 0x5564eb803310 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5564eb80f780 con 0x5564eb803c30 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 10 client.0 _ll_register_callbacks cb 0x5564eb47a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb80a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb80a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34628/0 (socket says 172.21.15.5:34628) 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 -- 172.21.15.5:0/2518316916 learned_addr learned my addr 172.21.15.5:0/2518316916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803310 0x5564eb809670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb809bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 -- 172.21.15.5:0/2518316916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 msgr2=0x5564eb809bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb809bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 -- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803310 msgr2=0x5564eb809670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803310 0x5564eb809670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 -- 172.21.15.5:0/2518316916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564eb47a920 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc892ffd640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb80a0f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc888005d80 tx=0x7fc888009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf99db1f1dd6fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc872ffd640 1 -- 172.21.15.5:0/2518316916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888004020 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc872ffd640 1 -- 172.21.15.5:0/2518316916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc8880041c0 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8840035f0 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc872ffd640 1 -- 172.21.15.5:0/2518316916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc888026050 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 10 client.16539 fetch_fsmap learned FSMap version 27 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.499+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564eb546d30 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc872ffd640 1 -- 172.21.15.5:0/2518316916 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8840035f0 con 0x5564eb80c810 2024-04-22T13:29:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc872ffd640 1 -- 172.21.15.5:0/2518316916 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc88800c070 con 0x5564eb80c810 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc89c9d4740 10 client.16539 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc89c9d4740 10 client.16539 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc89c9d4740 20 client.16539 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb81ca50 0x5564eb81ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc89c9d4740 4 client.16539 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564eb73ba10 con 0x5564eb81ca50 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.500+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb81ca50 0x5564eb81ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.501+0000 7fc872ffd640 10 client.16539 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.501+0000 7fc8937fe640 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb81ca50 0x5564eb81ee80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9d6e5560fa616a server_cookie=2650ba9fe2d63b34 in_seq=0 out_seq=0 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.501+0000 7fc872ffd640 1 -- 172.21.15.5:0/2518316916 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564eb73ba10 con 0x5564eb81ca50 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.501+0000 7fc872ffd640 10 client.16539 handle_command_reply: tid=0 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.501+0000 7fc89c9d4740 1 client.16539 shutdown 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 msgr2=0x5564eb80a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb80a0f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc888005d80 tx=0x7fc888009320 comp rx=0 tx=0).stop 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 shutdown_connections 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb81ca50 0x5564eb81ee80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb80a0f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc872ffd640 0 client.16539 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803c30 0x5564eb809bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/2518316916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803310 0x5564eb809670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 >> 172.21.15.5:0/2518316916 conn(0x5564eb7fda40 msgr2=0x5564eb7ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 shutdown_connections 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 1 -- 172.21.15.5:0/2518316916 wait complete. 2024-04-22T13:29:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.502+0000 7fc89c9d4740 20 client.16539 trim_cache size 0 max 16384 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 Processor -- start 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 -- start start 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb8184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803c30 0x5564eb824750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb824c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941adb00 con 0x5564eb80c810 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941af640 con 0x5564eb803c30 2024-04-22T13:29:07.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ae1d0 con 0x5564eb803310 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb824c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb824c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34642/0 (socket says 172.21.15.5:34642) 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc892ffd640 1 -- 172.21.15.5:0/412685763 learned_addr learned my addr 172.21.15.5:0/412685763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.519+0000 7fc892ffd640 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb8184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc8937fe640 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803c30 0x5564eb824750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc892ffd640 1 -- 172.21.15.5:0/412685763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 msgr2=0x5564eb8184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc892ffd640 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb8184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc892ffd640 1 -- 172.21.15.5:0/412685763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803c30 msgr2=0x5564eb824750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc892ffd640 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803c30 0x5564eb824750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc892ffd640 1 -- 172.21.15.5:0/412685763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564eb7fe7b0 con 0x5564eb80c810 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc892ffd640 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb824c90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc888005fd0 tx=0x7fc8880149c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e349b7deec69bd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc8797fa640 1 -- 172.21.15.5:0/412685763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88802f070 con 0x5564eb80c810 2024-04-22T13:29:07.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc8797fa640 1 -- 172.21.15.5:0/412685763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc888014e50 con 0x5564eb80c810 2024-04-22T13:29:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc8797fa640 1 -- 172.21.15.5:0/412685763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888022d10 con 0x5564eb80c810 2024-04-22T13:29:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc89c9d4740 1 -- 172.21.15.5:0/412685763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 msgr2=0x5564eb824c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.520+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb824c90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc888005fd0 tx=0x7fc8880149c0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- 172.21.15.5:0/412685763 shutdown_connections 2024-04-22T13:29:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb80c810 0x5564eb824c90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb803c30 0x5564eb824750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/412685763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb8184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- 172.21.15.5:0/412685763 >> 172.21.15.5:0/412685763 conn(0x5564eb7fda40 msgr2=0x5564eb7fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- 172.21.15.5:0/412685763 shutdown_connections 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- 172.21.15.5:0/412685763 wait complete. 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- start start 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 Processor -- start 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb87b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 0x5564eb87b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb810da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941aef70 con 0x5564eb803c30 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ae8a0 con 0x5564eb80c810 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941abae0 con 0x5564eb803310 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc89c9d4740 10 client.0 _ll_register_callbacks cb 0x7fc888014910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc8937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 0x5564eb87b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc8937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 0x5564eb87b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34644/0 (socket says 172.21.15.5:34644) 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.521+0000 7fc8937fe640 1 -- 172.21.15.5:0/3018733824 learned_addr learned my addr 172.21.15.5:0/3018733824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc892ffd640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb810da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc892ffd640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb87b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc8937fe640 1 -- 172.21.15.5:0/3018733824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 msgr2=0x5564eb87b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc8937fe640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb87b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc8937fe640 1 -- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 msgr2=0x5564eb810da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc8937fe640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb810da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc8937fe640 1 -- 172.21.15.5:0/3018733824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564eb811490 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc892ffd640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb810da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc8937fe640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 0x5564eb87b800 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc884010e20 tx=0x7fc88400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726292f181b1cbe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc872ffd640 1 -- 172.21.15.5:0/3018733824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88402e060 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc872ffd640 1 -- 172.21.15.5:0/3018733824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc884029630 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc888014e50 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc872ffd640 1 -- 172.21.15.5:0/3018733824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc884029830 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc89c9d4740 10 client.16545 fetch_fsmap learned FSMap version 27 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564eb883c60 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.522+0000 7fc872ffd640 1 -- 172.21.15.5:0/3018733824 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc888014e50 con 0x5564eb803c30 2024-04-22T13:29:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc872ffd640 1 -- 172.21.15.5:0/3018733824 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc884005500 con 0x5564eb803c30 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc89c9d4740 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc89c9d4740 10 client.16545 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc89c9d4740 20 client.16545 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb884100 0x5564eb8864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc89c9d4740 4 client.16545 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ce13b8c6", "release": true, "await": true}] 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ce13b8c6", "release": true, "await": true}) -- 0x5564eb775f20 con 0x5564eb884100 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc892ffd640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb884100 0x5564eb8864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc872ffd640 10 client.16545 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.523+0000 7fc892ffd640 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb884100 0x5564eb8864f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c38080e3457b23 server_cookie=ed810f26321298bd in_seq=0 out_seq=0 2024-04-22T13:29:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.527+0000 7fc872ffd640 1 -- 172.21.15.5:0/3018733824 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5564eb775f20 con 0x5564eb884100 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.527+0000 7fc872ffd640 10 client.16545 handle_command_reply: tid=0 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 client.16545 shutdown 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 msgr2=0x5564eb87b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 0x5564eb87b800 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc884010e20 tx=0x7fc88400bf70 comp rx=0 tx=0).stop 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 shutdown_connections 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5564eb884100 0x5564eb8864f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564eb803c30 0x5564eb87b800 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5564eb80c810 0x5564eb810da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 --2- 172.21.15.5:0/3018733824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564eb803310 0x5564eb87b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc872ffd640 0 client.16545 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 >> 172.21.15.5:0/3018733824 conn(0x5564eb7fda40 msgr2=0x5564eb7fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 shutdown_connections 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 1 -- 172.21.15.5:0/3018733824 wait complete. 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc89c9d4740 20 client.16545 trim_cache size 0 max 16384 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 12, 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "ce13b8c6": { 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 12, 2024-04-22T13:29:07.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 86.5, 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:29:07.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:07.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:07.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:07.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:07.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:07.531 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc87c0a3da0 msgr2=0x7fc87c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc87c0a3da0 0x7fc87c0a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0be920 tx=0x7fc88c0b40b0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 msgr2=0x7fc89419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89419cf50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc88c02c790 tx=0x7fc88c003c20 comp rx=0 tx=0).stop 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 shutdown_connections 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc89410c500 0x7fc89419cf50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc87c0a3da0 0x7fc87c0a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc894103ae0 0x7fc894198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 --2- 172.21.15.5:0/700343296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8941031c0 0x7fc894198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.528+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 >> 172.21.15.5:0/700343296 conn(0x7fc894078000 msgr2=0x7fc8940fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.529+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 shutdown_connections 2024-04-22T13:29:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.529+0000 7fc899de6640 1 -- 172.21.15.5:0/700343296 wait complete. 2024-04-22T13:29:07.533 INFO:tasks.workunit.client.0.smithi005.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-04-22T13:29:07.536 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: ce13b8c6, seconds elapsed: 27.7 2024-04-22T13:29:07.536 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-04-22T13:29:07.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:07.551 INFO:tasks.workunit.client.0.smithi005.stderr:--2024-04-22 13:29:07-- http://download.ceph.com/qa/fsync-tester.c 2024-04-22T13:29:07.560 INFO:tasks.workunit.client.0.smithi005.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-22T13:29:07.583 INFO:tasks.workunit.client.0.smithi005.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-22T13:29:07.602 INFO:tasks.workunit.client.0.smithi005.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-22T13:29:07.602 INFO:tasks.workunit.client.0.smithi005.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-04-22T13:29:07.603 INFO:tasks.workunit.client.0.smithi005.stderr:Saving to: ‘fsync-tester.c’ 2024-04-22T13:29:07.606 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:29:07.606 INFO:tasks.workunit.client.0.smithi005.stderr: 0K .. 100% 506M=0s 2024-04-22T13:29:07.606 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:29:07.606 INFO:tasks.workunit.client.0.smithi005.stderr:2024-04-22 13:29:07 (506 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-04-22T13:29:07.607 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-22T13:29:07.607 INFO:tasks.workunit.client.0.smithi005.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-04-22T13:29:07.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- 172.21.15.5:0/1469376522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014106140 msgr2=0x7f501410dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- 172.21.15.5:0/1469376522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014106140 0x7f501410dd10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f500c009ca0 tx=0x7f500c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- 172.21.15.5:0/1469376522 shutdown_connections 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- 172.21.15.5:0/1469376522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014106140 0x7f501410dd10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- 172.21.15.5:0/1469376522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5014101a20 0x7f5014105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- 172.21.15.5:0/1469376522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014101070 0x7f5014101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- 172.21.15.5:0/1469376522 >> 172.21.15.5:0/1469376522 conn(0x7f50140771d0 msgr2=0x7f50140775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- 172.21.15.5:0/1469376522 shutdown_connections 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- 172.21.15.5:0/1469376522 wait complete. 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 Processor -- start 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- start start 2024-04-22T13:29:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5014101070 0x7f5014198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014106140 0x7f5014198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 0x7f501419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50141121c0 con 0x7f501419cf70 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5014112040 con 0x7f5014101070 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.644+0000 7f501ad50640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5014112340 con 0x7f5014106140 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 0x7f501419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 0x7f501419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34658/0 (socket says 172.21.15.5:34658) 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 -- 172.21.15.5:0/598518762 learned_addr learned my addr 172.21.15.5:0/598518762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f5018aec640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5014101070 0x7f5014198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f500bfff640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014106140 0x7f5014198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 -- 172.21.15.5:0/598518762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014106140 msgr2=0x7f5014198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014106140 0x7f5014198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 -- 172.21.15.5:0/598518762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5014101070 msgr2=0x7f5014198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5014101070 0x7f5014198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 -- 172.21.15.5:0/598518762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501419da60 con 0x7f501419cf70 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f500bfff640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014106140 0x7f5014198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f50192ed640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 0x7f501419d420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f500c005e30 tx=0x7f500c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c041020 con 0x7f501419cf70 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f500c032970 con 0x7f501419cf70 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501419dcf0 con 0x7f501419cf70 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c039040 con 0x7f501419cf70 2024-04-22T13:29:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.645+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50141a54b0 con 0x7f501419cf70 2024-04-22T13:29:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.647+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5014068e00 con 0x7f501419cf70 2024-04-22T13:29:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.648+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f500c044020 con 0x7f501419cf70 2024-04-22T13:29:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.648+0000 7f5009ffb640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4ffc0a3d20 0x7f4ffc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.648+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f500c0f2490 con 0x7f501419cf70 2024-04-22T13:29:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.649+0000 7f5018aec640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4ffc0a3d20 0x7f4ffc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.649+0000 7f5018aec640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4ffc0a3d20 0x7f4ffc0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5010005fd0 tx=0x7f501000b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.653+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f500c04b020 con 0x7f501419cf70 2024-04-22T13:29:07.683 INFO:tasks.workunit.client.0.smithi005.stderr:+ ./fsync-tester 2024-04-22T13:29:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.886+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5014101a20 con 0x7f501419cf70 2024-04-22T13:29:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.886+0000 7f5009ffb640 1 -- 172.21.15.5:0/598518762 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11905 (secure 0 0 0) 0x7f500c0bec70 con 0x7f501419cf70 2024-04-22T13:29:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:29:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:27:07.748442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2603097758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2603097758},{"type":"v1","addr":"172.21.15.107:6835","nonce":2603097758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24496,24485,24469]},"id":1}]} 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4ffc0a3d20 msgr2=0x7f4ffc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4ffc0a3d20 0x7f4ffc0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5010005fd0 tx=0x7f501000b390 comp rx=0 tx=0).stop 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 msgr2=0x7f501419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 0x7f501419d420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f500c005e30 tx=0x7f500c034670 comp rx=0 tx=0).stop 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 shutdown_connections 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f501419cf70 0x7f501419d420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4ffc0a3d20 0x7f4ffc0a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5014106140 0x7f5014198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 --2- 172.21.15.5:0/598518762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5014101070 0x7f5014198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 >> 172.21.15.5:0/598518762 conn(0x7f50140771d0 msgr2=0x7f5014100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 shutdown_connections 2024-04-22T13:29:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:07.889+0000 7f501ad50640 1 -- 172.21.15.5:0/598518762 wait complete. 2024-04-22T13:29:07.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:07 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/435807430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:07.896 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db / --timeout 60 --expiration 86.5 --await' 2024-04-22T13:29:07.897 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db / --timeout 60 --expiration 86.5 --await 2024-04-22T13:29:08.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:07 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/435807430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:08.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:07 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/435807430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 -- 172.21.15.5:0/2406919928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c1031c0 msgr2=0x7f1e4c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2406919928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c1035a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009900 tx=0x7f1e4002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 -- 172.21.15.5:0/2406919928 shutdown_connections 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2406919928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 0x7f1e4c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2406919928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2406919928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c1035a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 -- 172.21.15.5:0/2406919928 >> 172.21.15.5:0/2406919928 conn(0x7f1e4c078030 msgr2=0x7f1e4c078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 -- 172.21.15.5:0/2406919928 shutdown_connections 2024-04-22T13:29:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 -- 172.21.15.5:0/2406919928 wait complete. 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.057+0000 7f1e510e4640 1 Processor -- start 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 -- start start 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 0x7f1e4c101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1121d0 con 0x7f1e4c103ae0 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c112050 con 0x7f1e4c1031c0 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e510e4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c112350 con 0x7f1e4c10c590 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34684/0 (socket says 172.21.15.5:34684) 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2443907176 learned_addr learned my addr 172.21.15.5:0/2443907176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 0x7f1e4c101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2443907176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 msgr2=0x7f1e4c101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 0x7f1e4c101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2443907176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1031c0 msgr2=0x7f1e4c100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2443907176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c1a6c10 con 0x7f1e4c103ae0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 0x7f1e4c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.058+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.059+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c101350 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00d950 tx=0x7f1e3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.059+0000 7f1e33fff640 1 -- 172.21.15.5:0/2443907176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c017070 con 0x7f1e4c103ae0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.059+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1a6f00 con 0x7f1e4c103ae0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.059+0000 7f1e33fff640 1 -- 172.21.15.5:0/2443907176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c009c30 con 0x7f1e4c103ae0 2024-04-22T13:29:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.059+0000 7f1e33fff640 1 -- 172.21.15.5:0/2443907176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00f620 con 0x7f1e4c103ae0 2024-04-22T13:29:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.059+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a73c0 con 0x7f1e4c103ae0 2024-04-22T13:29:08.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.061+0000 7f1e33fff640 1 -- 172.21.15.5:0/2443907176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1e3c010460 con 0x7f1e4c103ae0 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.062+0000 7f1e33fff640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e340a3d20 0x7f1e340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.062+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e340a3d20 0x7f1e340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.062+0000 7f1e33fff640 1 -- 172.21.15.5:0/2443907176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1e3c0d04a0 con 0x7f1e4c103ae0 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.062+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e340a3d20 0x7f1e340a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005ab0 tx=0x7f1e40005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 Processor -- start 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 -- start start 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdbc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a6ecd110b0 con 0x55a6ecdb12e0 2024-04-22T13:29:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a6eccc2680 con 0x55a6ecdb09c0 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e53db2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a6ecccec60 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34694/0 (socket says 172.21.15.5:34694) 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2597377548 learned_addr learned my addr 172.21.15.5:0/2597377548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdbc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.063+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2597377548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb9ec0 msgr2=0x55a6ecdbc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdbc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2597377548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 msgr2=0x55a6ecdb0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4a59c640 1 -- 172.21.15.5:0/2597377548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ecaf3c90 con 0x55a6ecdb12e0 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdbc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb9980 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0988d0 tx=0x7f1e3c09ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f909755d53ce971d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e317fa640 1 -- 172.21.15.5:0/2597377548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c020070 con 0x55a6ecdb12e0 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e317fa640 1 -- 172.21.15.5:0/2597377548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c0a0820 con 0x55a6ecdb12e0 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.064+0000 7f1e317fa640 1 -- 172.21.15.5:0/2597377548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0d0230 con 0x55a6ecdb12e0 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 -- 172.21.15.5:0/2597377548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 msgr2=0x55a6ecdb9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb9980 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0988d0 tx=0x7f1e3c09ccb0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 -- 172.21.15.5:0/2597377548 shutdown_connections 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb9980 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdbc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 --2- 172.21.15.5:0/2597377548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 -- 172.21.15.5:0/2597377548 >> 172.21.15.5:0/2597377548 conn(0x55a6ecdab0f0 msgr2=0x55a6ecdab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 -- 172.21.15.5:0/2597377548 shutdown_connections 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.065+0000 7f1e53db2740 1 -- 172.21.15.5:0/2597377548 wait complete. 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 -- start start 2024-04-22T13:29:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 Processor -- start 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a6ecd14400 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a6eccd0840 con 0x55a6ecdb12e0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a6ecd19720 con 0x55a6ecdb09c0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e53db2740 10 client.0 _ll_register_callbacks cb 0x55a6ecd21ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e4b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e4b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34700/0 (socket says 172.21.15.5:34700) 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.066+0000 7f1e4b59e640 1 -- 172.21.15.5:0/3665812234 learned_addr learned my addr 172.21.15.5:0/3665812234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4b59e640 1 -- 172.21.15.5:0/3665812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 msgr2=0x55a6ecdb6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4b59e640 1 -- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb12e0 msgr2=0x55a6ecdb74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4b59e640 1 -- 172.21.15.5:0/3665812234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ecb54420 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb7a00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1e440129f0 tx=0x7f1e44012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73f2263c6dbe610 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e2eff5640 1 -- 172.21.15.5:0/3665812234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011cd0 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e2eff5640 1 -- 172.21.15.5:0/3665812234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1e4400f460 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c0a0820 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e2eff5640 1 -- 172.21.15.5:0/3665812234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e44011e70 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e53db2740 10 client.16563 fetch_fsmap learned FSMap version 27 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.067+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a6ec9e7580 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e2eff5640 1 -- 172.21.15.5:0/3665812234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c0a0820 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e2eff5640 1 -- 172.21.15.5:0/3665812234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1e4401c7e0 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e53db2740 10 client.16563 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e53db2740 10 client.16563 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e53db2740 20 client.16563 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e53db2740 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ecdc9d30 0x55a6ecdcc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e53db2740 4 client.16563 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6ecd22b10 con 0x55a6ecdc9d30 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.068+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ecdc9d30 0x55a6ecdcc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ecdc9d30 0x55a6ecdcc120 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412a12aeddb9fb61 server_cookie=64253079fa90203c in_seq=0 out_seq=0 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e2eff5640 10 client.16563 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e2eff5640 1 -- 172.21.15.5:0/3665812234 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6ecd22b10 con 0x55a6ecdc9d30 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e2eff5640 10 client.16563 handle_command_reply: tid=0 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e53db2740 1 client.16563 shutdown 2024-04-22T13:29:08.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 msgr2=0x55a6ecdb7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e53db2740 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb7a00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1e440129f0 tx=0x7f1e44012eb0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 shutdown_connections 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.069+0000 7f1e53db2740 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ecdc9d30 0x55a6ecdcc120 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb7a00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e2eff5640 0 client.16563 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 1 --2- 172.21.15.5:0/3665812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 >> 172.21.15.5:0/3665812234 conn(0x55a6ecdab0f0 msgr2=0x55a6ecdace40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 shutdown_connections 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 1 -- 172.21.15.5:0/3665812234 wait complete. 2024-04-22T13:29:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.070+0000 7f1e53db2740 20 client.16563 trim_cache size 0 max 16384 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.102+0000 7f1e53db2740 1 Processor -- start 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 -- start start 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1b1850 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1b03e0 con 0x55a6ecdb09c0 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e53db2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1b0ab0 con 0x55a6ecdb12e0 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdc5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdc5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34710/0 (socket says 172.21.15.5:34710) 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 -- 172.21.15.5:0/189793281 learned_addr learned my addr 172.21.15.5:0/189793281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 -- 172.21.15.5:0/189793281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb12e0 msgr2=0x55a6ecdb5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 -- 172.21.15.5:0/189793281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 msgr2=0x55a6ecdb96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 -- 172.21.15.5:0/189793281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ecdc8830 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.103+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdc5810 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1e4401ccd0 tx=0x7f1e440118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f10f2014700df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.104+0000 7f1e30ff9640 1 -- 172.21.15.5:0/189793281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44029040 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.104+0000 7f1e30ff9640 1 -- 172.21.15.5:0/189793281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4401c420 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.104+0000 7f1e30ff9640 1 -- 172.21.15.5:0/189793281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440052a0 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 -- 172.21.15.5:0/189793281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 msgr2=0x55a6ecdc5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdc5810 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1e4401ccd0 tx=0x7f1e440118c0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 -- 172.21.15.5:0/189793281 shutdown_connections 2024-04-22T13:29:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdc5810 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 --2- 172.21.15.5:0/189793281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 -- 172.21.15.5:0/189793281 >> 172.21.15.5:0/189793281 conn(0x55a6ecdab0f0 msgr2=0x55a6ecdb77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 -- 172.21.15.5:0/189793281 shutdown_connections 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 -- 172.21.15.5:0/189793281 wait complete. 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 -- start start 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.105+0000 7f1e53db2740 1 Processor -- start 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1adcf0 con 0x55a6ecdb12e0 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1b1180 con 0x55a6ecdb9ec0 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ae360 con 0x55a6ecdb09c0 2024-04-22T13:29:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e53db2740 10 client.0 _ll_register_callbacks cb 0x7f1e44009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34718/0 (socket says 172.21.15.5:34718) 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 -- 172.21.15.5:0/1041212091 learned_addr learned my addr 172.21.15.5:0/1041212091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4b59e640 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 -- 172.21.15.5:0/1041212091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 msgr2=0x55a6ecdb3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 -- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb9ec0 msgr2=0x55a6ecdb4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 -- 172.21.15.5:0/1041212091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ece2f360 con 0x55a6ecdb12e0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e4a59c640 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb42e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c09cd20 tx=0x7f1e3c0ce0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422b64af5511f20e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.106+0000 7f1e2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e2eff5640 1 -- 172.21.15.5:0/1041212091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c020040 con 0x55a6ecdb12e0 2024-04-22T13:29:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e2eff5640 1 -- 172.21.15.5:0/1041212091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f1e3c0bb2e0 con 0x55a6ecdb12e0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4401c420 con 0x55a6ecdb12e0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e2eff5640 1 -- 172.21.15.5:0/1041212091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c0a0820 con 0x55a6ecdb12e0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 10 client.16569 fetch_fsmap learned FSMap version 27 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a6ece30620 con 0x55a6ecdb12e0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e2eff5640 1 -- 172.21.15.5:0/1041212091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4401c420 con 0x55a6ecdb12e0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e2eff5640 1 -- 172.21.15.5:0/1041212091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1e3c0982e0 con 0x55a6ecdb12e0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 10 client.16569 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 10 client.16569 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 20 client.16569 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ece30960 0x55a6ece32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 4 client.16569 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.5, "await": true}] 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.5, "await": true}) -- 0x55a6ecd1c980 con 0x55a6ece30960 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.107+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ece30960 0x55a6ece32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.108+0000 7f1e4ad9d640 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ece30960 0x55a6ece32d50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930985485ff9101 server_cookie=44c4d84b1b9633d1 in_seq=0 out_seq=0 2024-04-22T13:29:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.108+0000 7f1e2eff5640 10 client.16569 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.132+0000 7f1e2eff5640 1 -- 172.21.15.5:0/1041212091 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a6ecd1c980 con 0x55a6ece30960 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.132+0000 7f1e2eff5640 10 client.16569 handle_command_reply: tid=0 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.132+0000 7f1e53db2740 1 client.16569 shutdown 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 msgr2=0x55a6ecdb42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb42e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c09cd20 tx=0x7f1e3c0ce0d0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 shutdown_connections 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a6ece30960 0x55a6ece32d50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ecdb12e0 0x55a6ecdb42e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e2eff5640 0 client.16569 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a6ecdb9ec0 0x55a6ecdb4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 --2- 172.21.15.5:0/1041212091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a6ecdb09c0 0x55a6ecdb3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 >> 172.21.15.5:0/1041212091 conn(0x55a6ecdab0f0 msgr2=0x55a6ecdb6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 shutdown_connections 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 1 -- 172.21.15.5:0/1041212091 wait complete. 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e53db2740 20 client.16569 trim_cache size 0 max 16384 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:29:08.130 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 14, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "d7c67eb": { 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 14, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 86.5, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e340a3d20 msgr2=0x7f1e340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e340a3d20 0x7f1e340a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005ab0 tx=0x7f1e40005a20 comp rx=0 tx=0).stop 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 msgr2=0x7f1e4c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c101350 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00d950 tx=0x7f1e3c00de10 comp rx=0 tx=0).stop 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 shutdown_connections 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e4c103ae0 0x7f1e4c101350 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e340a3d20 0x7f1e340a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e4c10c590 0x7f1e4c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 --2- 172.21.15.5:0/2443907176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4c1031c0 0x7f1e4c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.133+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 >> 172.21.15.5:0/2443907176 conn(0x7f1e4c078030 msgr2=0x7f1e4c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.134+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 shutdown_connections 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:08.134+0000 7f1e510e4640 1 -- 172.21.15.5:0/2443907176 wait complete. 2024-04-22T13:29:08.140 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: d7c67eb, quiesce took 0.6 seconds. Will release after: 26.5 2024-04-22T13:29:09.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:08 smithi049 ceph-mon[31379]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:29:09.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:08 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/598518762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:08 smithi107 ceph-mon[31859]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:29:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:08 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/598518762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:08 smithi005 ceph-mon[28006]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:29:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:08 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/598518762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:10 smithi049 ceph-mon[31379]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:29:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:10 smithi107 ceph-mon[31859]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:29:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:10 smithi005 ceph-mon[28006]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T13:29:13.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:12 smithi049 ceph-mon[31379]: pgmap v64: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 114 KiB/s wr, 29 op/s 2024-04-22T13:29:13.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:12 smithi107 ceph-mon[31859]: pgmap v64: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 114 KiB/s wr, 29 op/s 2024-04-22T13:29:13.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:12 smithi005 ceph-mon[28006]: pgmap v64: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 114 KiB/s wr, 29 op/s 2024-04-22T13:29:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:14 smithi049 ceph-mon[31379]: pgmap v65: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 113 KiB/s wr, 24 op/s 2024-04-22T13:29:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:14 smithi107 ceph-mon[31859]: pgmap v65: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 113 KiB/s wr, 24 op/s 2024-04-22T13:29:15.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:14 smithi005 ceph-mon[28006]: pgmap v65: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 113 KiB/s wr, 24 op/s 2024-04-22T13:29:16.792 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:16 smithi107 ceph-mon[31859]: pgmap v66: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 33 op/s 2024-04-22T13:29:16.792 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:29:16.792 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:16.792 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:16.792 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:16.792 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:16 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:16.793 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:16 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-04-22T13:29:16.793 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:16 smithi107 systemd[1]: Stopped Ceph mds.f for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:29:16.793 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:16 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.014s CPU time. 2024-04-22T13:29:16.793 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:16 smithi107 systemd[1]: Starting Ceph mds.f for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:29:16.793 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:16 smithi107 podman[66893]: 2024-04-22 13:29:16.668452751 +0000 UTC m=+0.018443983 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:29:17.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:16 smithi049 ceph-mon[31379]: pgmap v66: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 33 op/s 2024-04-22T13:29:17.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:29:17.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:16 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:16 smithi005 ceph-mon[28006]: pgmap v66: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 33 op/s 2024-04-22T13:29:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:29:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:16 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:17.061 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:16 smithi107 podman[66893]: 2024-04-22 13:29:16.796744241 +0000 UTC m=+0.146735474 container create bf959c5c5c81b5032f18365ba6b9ae954aefd94c66a88391d80ae81ee1d0d324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:29:17.061 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:17 smithi107 podman[66893]: 2024-04-22 13:29:17.041180739 +0000 UTC m=+0.391171970 container init bf959c5c5c81b5032f18365ba6b9ae954aefd94c66a88391d80ae81ee1d0d324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-22T13:29:17.062 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:17 smithi107 podman[66893]: 2024-04-22 13:29:17.04388877 +0000 UTC m=+0.393880010 container start bf959c5c5c81b5032f18365ba6b9ae954aefd94c66a88391d80ae81ee1d0d324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-22T13:29:17.330 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mds[66943]: -- 172.21.15.107:0/473805421 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56550a62ba40 con 0x56550a5fec00 2024-04-22T13:29:17.330 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[66920]: starting mds.f at 2024-04-22T13:29:17.330 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:17 smithi107 bash[66893]: bf959c5c5c81b5032f18365ba6b9ae954aefd94c66a88391d80ae81ee1d0d324 2024-04-22T13:29:17.330 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:29:17 smithi107 systemd[1]: Started Ceph mds.f for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:29:18.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: daemon mds.f restarted 2024-04-22T13:29:18.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:29:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:29:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T13:29:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6834/603974888,v1:172.21.15.107:6835/603974888] up:boot 2024-04-22T13:29:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:29:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: daemon mds.f restarted 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6834/603974888,v1:172.21.15.107:6835/603974888] up:boot 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:29:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: daemon mds.f restarted 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6834/603974888,v1:172.21.15.107:6835/603974888] up:boot 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:29:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.682+0000 7f353eb96640 1 -- 172.21.15.5:0/3439980391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538107290 msgr2=0x7f353810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.682+0000 7f353eb96640 1 --2- 172.21.15.5:0/3439980391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538107290 0x7f353810ee60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3530009ca0 tx=0x7f353002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 -- 172.21.15.5:0/3439980391 shutdown_connections 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 --2- 172.21.15.5:0/3439980391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538107290 0x7f353810ee60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 --2- 172.21.15.5:0/3439980391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538102b70 0x7f3538106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 --2- 172.21.15.5:0/3439980391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35381021c0 0x7f35381025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 -- 172.21.15.5:0/3439980391 >> 172.21.15.5:0/3439980391 conn(0x7f35380fda40 msgr2=0x7f35380ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 -- 172.21.15.5:0/3439980391 shutdown_connections 2024-04-22T13:29:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 -- 172.21.15.5:0/3439980391 wait complete. 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 Processor -- start 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.683+0000 7f353eb96640 1 -- start start 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35381021c0 0x7f353819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538107290 0x7f353819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 0x7f3538198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3538112200 con 0x7f3538198610 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3538112080 con 0x7f35381021c0 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3538112380 con 0x7f3538107290 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 0x7f3538198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 0x7f3538198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58054/0 (socket says 172.21.15.5:58054) 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 -- 172.21.15.5:0/3594018203 learned_addr learned my addr 172.21.15.5:0/3594018203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f3537fff640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538107290 0x7f353819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353c932640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35381021c0 0x7f353819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 -- 172.21.15.5:0/3594018203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538107290 msgr2=0x7f353819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538107290 0x7f353819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 -- 172.21.15.5:0/3594018203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35381021c0 msgr2=0x7f353819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35381021c0 0x7f353819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 -- 172.21.15.5:0/3594018203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3538199130 con 0x7f3538198610 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353d133640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 0x7f3538198ac0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3530005e30 tx=0x7f35300346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f3535ffb640 1 -- 172.21.15.5:0/3594018203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530041020 con 0x7f3538198610 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f3535ffb640 1 -- 172.21.15.5:0/3594018203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3530032970 con 0x7f3538198610 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.684+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35381993c0 con 0x7f3538198610 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.685+0000 7f3535ffb640 1 -- 172.21.15.5:0/3594018203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530039040 con 0x7f3538198610 2024-04-22T13:29:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.685+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35381a5610 con 0x7f3538198610 2024-04-22T13:29:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.687+0000 7f3535ffb640 1 -- 172.21.15.5:0/3594018203 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3530044020 con 0x7f3538198610 2024-04-22T13:29:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.687+0000 7f3541785740 1 Processor -- start 2024-04-22T13:29:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.687+0000 7f3541785740 1 -- start start 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3541785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3d95620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3541785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d95fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3541785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3541785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3cfa8d0 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3541785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3c97e80 con 0x55b2b3d95b60 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3541785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3cf6890 con 0x55b2b3d968d0 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3535ffb640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35200a3d90 0x7f35200a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58060/0 (socket says 172.21.15.5:58060) 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 -- 172.21.15.5:0/156951302 learned_addr learned my addr 172.21.15.5:0/156951302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f353d133640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35200a3d90 0x7f35200a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f353c932640 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d95fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3535ffb640 1 -- 172.21.15.5:0/3594018203 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f35300f2200 con 0x7f3538198610 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3d95620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 -- 172.21.15.5:0/156951302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 msgr2=0x55b2b3d95620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3d95620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 -- 172.21.15.5:0/156951302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 msgr2=0x55b2b3d95fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d95fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 -- 172.21.15.5:0/156951302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b3cb9320 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f353c932640 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d95fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f353d133640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35200a3d90 0x7f35200a6250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35300eff80 tx=0x7f353002e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.688+0000 7f3537fff640 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da14d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f352c009eb0 tx=0x7f352c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f1d1fb345e36b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f351effd640 1 -- 172.21.15.5:0/156951302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c004050 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f351effd640 1 -- 172.21.15.5:0/156951302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c0054d0 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f351effd640 1 -- 172.21.15.5:0/156951302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c004cc0 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 -- 172.21.15.5:0/156951302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 msgr2=0x55b2b3da14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da14d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f352c009eb0 tx=0x7f352c0096a0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 -- 172.21.15.5:0/156951302 shutdown_connections 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da14d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d95fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 --2- 172.21.15.5:0/156951302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3d95620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 -- 172.21.15.5:0/156951302 >> 172.21.15.5:0/156951302 conn(0x55b2b3d5cc90 msgr2=0x55b2b3d90850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 -- 172.21.15.5:0/156951302 shutdown_connections 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.689+0000 7f3541785740 1 -- 172.21.15.5:0/156951302 wait complete. 2024-04-22T13:29:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 -- start start 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 Processor -- start 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3da1e10 con 0x55b2b3d968d0 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3c82eb0 con 0x55b2b3d95b60 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3da1f90 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f3541785740 10 client.0 _ll_register_callbacks cb 0x55b2b3cafb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f353c932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f353c932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58066/0 (socket says 172.21.15.5:58066) 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.690+0000 7f353c932640 1 -- 172.21.15.5:0/727166643 learned_addr learned my addr 172.21.15.5:0/727166643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3537fff640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3537fff640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f353c932640 1 -- 172.21.15.5:0/727166643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 msgr2=0x55b2b3d9c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f353c932640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f353c932640 1 -- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 msgr2=0x55b2b3d9eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f353c932640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f353c932640 1 -- 172.21.15.5:0/727166643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b3a0d210 con 0x55b2b3d968d0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3537fff640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f353c932640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9be50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3528012740 tx=0x7f3528012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becef5828f77d1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3517fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3517fff640 1 -- 172.21.15.5:0/727166643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004030 con 0x55b2b3d968d0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3517fff640 1 -- 172.21.15.5:0/727166643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f35280041d0 con 0x55b2b3d968d0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f352c0054d0 con 0x55b2b3d968d0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3517fff640 1 -- 172.21.15.5:0/727166643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35280043d0 con 0x55b2b3d968d0 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3541785740 10 client.16584 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.691+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2b3ad9620 con 0x55b2b3d968d0 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3517fff640 1 -- 172.21.15.5:0/727166643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c0054d0 con 0x55b2b3d968d0 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3517fff640 1 -- 172.21.15.5:0/727166643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3528003b10 con 0x55b2b3d968d0 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3541785740 10 client.16584 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3541785740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3541785740 20 client.16584 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3541785740 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3daf1c0 0x55b2b3db15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3541785740 4 client.16584 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b3d0e880 con 0x55b2b3daf1c0 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.692+0000 7f3537fff640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3daf1c0 0x55b2b3db15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3537fff640 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3daf1c0 0x55b2b3db15b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e5463ac53948e7 server_cookie=21f52af445d9eca9 in_seq=0 out_seq=0 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3517fff640 10 client.16584 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3517fff640 1 -- 172.21.15.5:0/727166643 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2b3d0e880 con 0x55b2b3daf1c0 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3517fff640 10 client.16584 handle_command_reply: tid=0 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3541785740 1 client.16584 shutdown 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 msgr2=0x55b2b3d9be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3541785740 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9be50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3528012740 tx=0x7f3528012c00 comp rx=0 tx=0).stop 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 shutdown_connections 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.693+0000 7f3541785740 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3daf1c0 0x55b2b3db15b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9be50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3517fff640 0 client.16584 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 1 --2- 172.21.15.5:0/727166643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 >> 172.21.15.5:0/727166643 conn(0x55b2b3d5cc90 msgr2=0x55b2b3d925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 shutdown_connections 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 1 -- 172.21.15.5:0/727166643 wait complete. 2024-04-22T13:29:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.694+0000 7f3541785740 20 client.16584 trim_cache size 0 max 16384 2024-04-22T13:29:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 Processor -- start 2024-04-22T13:29:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 -- start start 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3daad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35381adb90 con 0x55b2b3d968d0 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35381af6d0 con 0x55b2b3d95b60 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3541785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ae260 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58070/0 (socket says 172.21.15.5:58070) 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 -- 172.21.15.5:0/610440379 learned_addr learned my addr 172.21.15.5:0/610440379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3537fff640 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3daad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 -- 172.21.15.5:0/610440379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 msgr2=0x55b2b3d9e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 -- 172.21.15.5:0/610440379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 msgr2=0x55b2b3daad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3daad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f353c932640 1 -- 172.21.15.5:0/610440379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b3d90a60 con 0x55b2b3d968d0 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.709+0000 7f3537fff640 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3daad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f353c932640 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9ae30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3528005810 tx=0x7f3528005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db645001c00fcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f351e7fc640 1 -- 172.21.15.5:0/610440379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528029040 con 0x55b2b3d968d0 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f351e7fc640 1 -- 172.21.15.5:0/610440379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528012e50 con 0x55b2b3d968d0 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f351e7fc640 1 -- 172.21.15.5:0/610440379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352800fb80 con 0x55b2b3d968d0 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 -- 172.21.15.5:0/610440379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 msgr2=0x55b2b3d9ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9ae30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3528005810 tx=0x7f3528005840 comp rx=0 tx=0).stop 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 -- 172.21.15.5:0/610440379 shutdown_connections 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d968d0 0x55b2b3d9ae30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3d9e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 --2- 172.21.15.5:0/610440379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3daad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.710+0000 7f3541785740 1 -- 172.21.15.5:0/610440379 >> 172.21.15.5:0/610440379 conn(0x55b2b3d5cc90 msgr2=0x55b2b3d9a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 -- 172.21.15.5:0/610440379 shutdown_connections 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 -- 172.21.15.5:0/610440379 wait complete. 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 -- start start 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 Processor -- start 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3da01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35381af000 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ae930 con 0x55b2b3d95b60 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35381abb90 con 0x55b2b3d968d0 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3541785740 10 client.0 _ll_register_callbacks cb 0x7f3528009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3537fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3537fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58078/0 (socket says 172.21.15.5:58078) 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.711+0000 7f3537fff640 1 -- 172.21.15.5:0/4173622993 learned_addr learned my addr 172.21.15.5:0/4173622993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f353c932640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3da01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 -- 172.21.15.5:0/4173622993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 msgr2=0x55b2b3da01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3da01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 -- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 msgr2=0x55b2b3d9fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 -- 172.21.15.5:0/4173622993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b3d9c7d0 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3537fff640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da0710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f352c009e80 tx=0x7f352c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114d46380b454796 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3517fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3517fff640 1 -- 172.21.15.5:0/4173622993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c027040 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3517fff640 1 -- 172.21.15.5:0/4173622993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f352c004220 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528012e50 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3517fff640 1 -- 172.21.15.5:0/4173622993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f352c004420 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3541785740 10 client.16590 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.712+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2b3e15790 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3517fff640 1 -- 172.21.15.5:0/4173622993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528012e50 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3517fff640 1 -- 172.21.15.5:0/4173622993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f352c02f2d0 con 0x55b2b3d9f0e0 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3541785740 10 client.16590 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3541785740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3541785740 20 client.16590 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3541785740 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3e15d60 0x55b2b3e18150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3541785740 4 client.16590 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:29:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2b3c9a3d0 con 0x55b2b3e15d60 2024-04-22T13:29:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.713+0000 7f353c932640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3e15d60 0x55b2b3e18150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.714+0000 7f353c932640 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3e15d60 0x55b2b3e18150 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac312d34f097efe server_cookie=3079a041c02ace1a in_seq=0 out_seq=0 2024-04-22T13:29:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.714+0000 7f3517fff640 10 client.16590 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3517fff640 1 -- 172.21.15.5:0/4173622993 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2b3c9a3d0 con 0x55b2b3e15d60 2024-04-22T13:29:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3517fff640 10 client.16590 handle_command_reply: tid=0 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 client.16590 shutdown 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 msgr2=0x55b2b3da0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da0710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f352c009e80 tx=0x7f352c01a040 comp rx=0 tx=0).stop 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 shutdown_connections 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b2b3e15d60 0x55b2b3e18150 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2b3d9f0e0 0x55b2b3da0710 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2b3d968d0 0x55b2b3da01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3517fff640 0 client.16590 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 --2- 172.21.15.5:0/4173622993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b2b3d95b60 0x55b2b3d9fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 >> 172.21.15.5:0/4173622993 conn(0x55b2b3d5cc90 msgr2=0x55b2b3d9afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 shutdown_connections 2024-04-22T13:29:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.722+0000 7f3541785740 1 -- 172.21.15.5:0/4173622993 wait complete. 2024-04-22T13:29:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f3541785740 20 client.16590 trim_cache size 0 max 16384 2024-04-22T13:29:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:29:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:29:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:29:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35200a3d90 msgr2=0x7f35200a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35200a3d90 0x7f35200a6250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35300eff80 tx=0x7f353002e040 comp rx=0 tx=0).stop 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 msgr2=0x7f3538198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 0x7f3538198ac0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3530005e30 tx=0x7f35300346d0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 shutdown_connections 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3538198610 0x7f3538198ac0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35200a3d90 0x7f35200a6250 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538107290 0x7f353819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 --2- 172.21.15.5:0/3594018203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35381021c0 0x7f353819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 >> 172.21.15.5:0/3594018203 conn(0x7f35380fda40 msgr2=0x7f35380ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 shutdown_connections 2024-04-22T13:29:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:18.723+0000 7f353eb96640 1 -- 172.21.15.5:0/3594018203 wait complete. 2024-04-22T13:29:19.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:18 smithi049 ceph-mon[31379]: pgmap v68: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-04-22T13:29:19.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:18 smithi107 ceph-mon[31859]: pgmap v68: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-04-22T13:29:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:18 smithi005 ceph-mon[28006]: pgmap v68: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-04-22T13:29:21.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:20 smithi049 ceph-mon[31379]: pgmap v69: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-04-22T13:29:21.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:20 smithi107 ceph-mon[31859]: pgmap v69: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-04-22T13:29:21.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:20 smithi005 ceph-mon[28006]: pgmap v69: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-04-22T13:29:23.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:22 smithi049 ceph-mon[31379]: pgmap v70: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-04-22T13:29:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:22 smithi107 ceph-mon[31859]: pgmap v70: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-04-22T13:29:23.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:22 smithi005 ceph-mon[28006]: pgmap v70: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-04-22T13:29:25.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:24 smithi049 ceph-mon[31379]: pgmap v71: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-04-22T13:29:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:24 smithi107 ceph-mon[31859]: pgmap v71: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-04-22T13:29:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:24 smithi005 ceph-mon[28006]: pgmap v71: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-04-22T13:29:25.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:25.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 -- 172.21.15.5:0/730787289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46410e8f0 msgr2=0x7ff464110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 --2- 172.21.15.5:0/730787289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46410e8f0 0x7ff464110ce0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009ca0 tx=0x7ff45c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 -- 172.21.15.5:0/730787289 shutdown_connections 2024-04-22T13:29:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 --2- 172.21.15.5:0/730787289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff46410e8f0 0x7ff464110ce0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 --2- 172.21.15.5:0/730787289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff464069c20 0x7ff46410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 --2- 172.21.15.5:0/730787289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff464069300 0x7ff4640696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.249+0000 7ff46bf75640 1 -- 172.21.15.5:0/730787289 >> 172.21.15.5:0/730787289 conn(0x7ff464077fa0 msgr2=0x7ff4640783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 -- 172.21.15.5:0/730787289 shutdown_connections 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 -- 172.21.15.5:0/730787289 wait complete. 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 Processor -- start 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 -- start start 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 0x7ff4641a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff464069c20 0x7ff4641a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff46410e8f0 0x7ff46419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff464114340 con 0x7ff464069300 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641141c0 con 0x7ff464069c20 2024-04-22T13:29:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff46bf75640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641144c0 con 0x7ff46410e8f0 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.250+0000 7ff469d11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 0x7ff4641a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 0x7ff4641a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39010/0 (socket says 172.21.15.5:39010) 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 -- 172.21.15.5:0/3113140945 learned_addr learned my addr 172.21.15.5:0/3113140945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469510640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff464069c20 0x7ff4641a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff46a512640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff46410e8f0 0x7ff46419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 -- 172.21.15.5:0/3113140945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff46410e8f0 msgr2=0x7ff46419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff46410e8f0 0x7ff46419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 -- 172.21.15.5:0/3113140945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff464069c20 msgr2=0x7ff4641a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff464069c20 0x7ff4641a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 -- 172.21.15.5:0/3113140945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419afa0 con 0x7ff464069300 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff469d11640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 0x7ff4641a05d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff46000d950 tx=0x7ff46000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff45affd640 1 -- 172.21.15.5:0/3113140945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460017070 con 0x7ff464069300 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff45affd640 1 -- 172.21.15.5:0/3113140945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff460009c30 con 0x7ff464069300 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46419b290 con 0x7ff464069300 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff45affd640 1 -- 172.21.15.5:0/3113140945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46000f620 con 0x7ff464069300 2024-04-22T13:29:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.251+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a7550 con 0x7ff464069300 2024-04-22T13:29:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.254+0000 7ff45affd640 1 -- 172.21.15.5:0/3113140945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff460010460 con 0x7ff464069300 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.254+0000 7ff46eb67740 1 Processor -- start 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 -- start start 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e957960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957ea0 0x559d4e960540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e962e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559d4e8bc1e0 con 0x559d4e957580 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559d4e859390 con 0x559d4e957ea0 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46eb67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559d4e8b81a0 con 0x559d4e960a80 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e957960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e957960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39024/0 (socket says 172.21.15.5:39024) 2024-04-22T13:29:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 -- 172.21.15.5:0/983833835 learned_addr learned my addr 172.21.15.5:0/983833835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff469510640 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e962e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff45affd640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4500a3da0 0x7ff4500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff469d11640 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957ea0 0x559d4e960540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 -- 172.21.15.5:0/983833835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 msgr2=0x559d4e962e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e962e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 -- 172.21.15.5:0/983833835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957ea0 msgr2=0x559d4e960540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957ea0 0x559d4e960540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.255+0000 7ff46a512640 1 -- 172.21.15.5:0/983833835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4e6987d0 con 0x559d4e957580 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.256+0000 7ff46a512640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4500a3da0 0x7ff4500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.256+0000 7ff45affd640 1 -- 172.21.15.5:0/3113140945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff4600d0360 con 0x7ff464069300 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.256+0000 7ff46a512640 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e957960 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff45c004af0 tx=0x7ff45c005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3761c43109a88545 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.256+0000 7ff44ffff640 1 -- 172.21.15.5:0/983833835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c039040 con 0x559d4e957580 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.256+0000 7ff44ffff640 1 -- 172.21.15.5:0/983833835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45c032e80 con 0x559d4e957580 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.256+0000 7ff44ffff640 1 -- 172.21.15.5:0/983833835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c004710 con 0x559d4e957580 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 -- 172.21.15.5:0/983833835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 msgr2=0x559d4e957960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46a512640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4500a3da0 0x7ff4500a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff45c04c000 tx=0x7ff45c045000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e957960 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff45c004af0 tx=0x7ff45c005f40 comp rx=0 tx=0).stop 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 -- 172.21.15.5:0/983833835 shutdown_connections 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e962e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957ea0 0x559d4e960540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 --2- 172.21.15.5:0/983833835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e957960 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 -- 172.21.15.5:0/983833835 >> 172.21.15.5:0/983833835 conn(0x559d4e951cb0 msgr2=0x559d4e952090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 -- 172.21.15.5:0/983833835 shutdown_connections 2024-04-22T13:29:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 -- 172.21.15.5:0/983833835 wait complete. 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 -- start start 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.257+0000 7ff46eb67740 1 Processor -- start 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957580 0x559d4e95da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e95e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559d4e963870 con 0x559d4e957ea0 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559d4e8752f0 con 0x559d4e957580 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559d4e9639f0 con 0x559d4e960a80 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46eb67740 10 client.0 _ll_register_callbacks cb 0x559d4e882580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39032/0 (socket says 172.21.15.5:39032) 2024-04-22T13:29:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 -- 172.21.15.5:0/2846483002 learned_addr learned my addr 172.21.15.5:0/2846483002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469d11640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957580 0x559d4e95da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff46a512640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e95e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 -- 172.21.15.5:0/2846483002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 msgr2=0x559d4e95e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e95e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 -- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957580 msgr2=0x559d4e95da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957580 0x559d4e95da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 -- 172.21.15.5:0/2846483002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d4e5ce3f0 con 0x559d4e957ea0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469d11640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957580 0x559d4e95da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff469510640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95dfd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff454012700 tx=0x7ff454012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3d95e00b82a9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff44d7fa640 1 -- 172.21.15.5:0/2846483002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400f8c0 con 0x559d4e957ea0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.258+0000 7ff44d7fa640 1 -- 172.21.15.5:0/2846483002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff45400fa60 con 0x559d4e957ea0 2024-04-22T13:29:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c032e80 con 0x559d4e957ea0 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff44d7fa640 1 -- 172.21.15.5:0/2846483002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff45400fc60 con 0x559d4e957ea0 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 10 client.16602 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d4e58e0f0 con 0x559d4e957ea0 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff44d7fa640 1 -- 172.21.15.5:0/2846483002 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45c032e80 con 0x559d4e957ea0 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff44d7fa640 1 -- 172.21.15.5:0/2846483002 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff4540254e0 con 0x559d4e957ea0 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 10 client.16602 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 20 client.16602 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e970a30 0x559d4e972e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 4 client.16602 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d4e8c9c00 con 0x559d4e970a30 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.259+0000 7ff469d11640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e970a30 0x559d4e972e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.260+0000 7ff469d11640 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e970a30 0x559d4e972e60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d847020cdaab310 server_cookie=a34ba45168f5a71a in_seq=0 out_seq=0 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.260+0000 7ff44d7fa640 10 client.16602 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff44d7fa640 1 -- 172.21.15.5:0/2846483002 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d4e8c9c00 con 0x559d4e970a30 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff44d7fa640 10 client.16602 handle_command_reply: tid=0 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 client.16602 shutdown 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 msgr2=0x559d4e95dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95dfd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff454012700 tx=0x7ff454012bc0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 shutdown_connections 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e970a30 0x559d4e972e60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95dfd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e960a80 0x559d4e95e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff44d7fa640 0 client.16602 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 --2- 172.21.15.5:0/2846483002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e957580 0x559d4e95da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 >> 172.21.15.5:0/2846483002 conn(0x559d4e951cb0 msgr2=0x559d4e953a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 shutdown_connections 2024-04-22T13:29:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 1 -- 172.21.15.5:0/2846483002 wait complete. 2024-04-22T13:29:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.261+0000 7ff46eb67740 20 client.16602 trim_cache size 0 max 16384 2024-04-22T13:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 Processor -- start 2024-04-22T13:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 -- start start 2024-04-22T13:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e9601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957ea0 0x559d4e96c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e950bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641afd50 con 0x559d4e957580 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641b1890 con 0x559d4e960a80 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46eb67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641b0420 con 0x559d4e957ea0 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e9601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e9601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39044/0 (socket says 172.21.15.5:39044) 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 -- 172.21.15.5:0/1726690323 learned_addr learned my addr 172.21.15.5:0/1726690323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469510640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957ea0 0x559d4e96c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46a512640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e950bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 -- 172.21.15.5:0/1726690323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957ea0 msgr2=0x559d4e96c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957ea0 0x559d4e96c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 -- 172.21.15.5:0/1726690323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 msgr2=0x559d4e950bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e950bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469d11640 1 -- 172.21.15.5:0/1726690323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4e957c90 con 0x559d4e957580 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff469510640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957ea0 0x559d4e96c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.279+0000 7ff46a512640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e950bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff469d11640 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e9601c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff4600987e0 tx=0x7ff460005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04163e7c90908fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff44f7fe640 1 -- 172.21.15.5:0/1726690323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460020040 con 0x559d4e957580 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff44f7fe640 1 -- 172.21.15.5:0/1726690323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff46009a680 con 0x559d4e957580 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff44f7fe640 1 -- 172.21.15.5:0/1726690323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4600a0120 con 0x559d4e957580 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 -- 172.21.15.5:0/1726690323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 msgr2=0x559d4e9601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e9601c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff4600987e0 tx=0x7ff460005880 comp rx=0 tx=0).stop 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 -- 172.21.15.5:0/1726690323 shutdown_connections 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e950bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957ea0 0x559d4e96c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 --2- 172.21.15.5:0/1726690323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957580 0x559d4e9601c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 -- 172.21.15.5:0/1726690323 >> 172.21.15.5:0/1726690323 conn(0x559d4e951cb0 msgr2=0x559d4e9504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 -- 172.21.15.5:0/1726690323 shutdown_connections 2024-04-22T13:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.280+0000 7ff46eb67740 1 -- 172.21.15.5:0/1726690323 wait complete. 2024-04-22T13:29:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 -- start start 2024-04-22T13:29:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 Processor -- start 2024-04-22T13:29:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957580 0x559d4e95ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e9d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641b11c0 con 0x559d4e957ea0 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641b0af0 con 0x559d4e960a80 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641add30 con 0x559d4e957580 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46eb67740 10 client.0 _ll_register_callbacks cb 0x7ff454009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff469510640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff469510640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39050/0 (socket says 172.21.15.5:39050) 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff469510640 1 -- 172.21.15.5:0/564759169 learned_addr learned my addr 172.21.15.5:0/564759169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff469d11640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957580 0x559d4e95ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff46a512640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e9d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.281+0000 7ff469510640 1 -- 172.21.15.5:0/564759169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957580 msgr2=0x559d4e95ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff469510640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957580 0x559d4e95ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff469510640 1 -- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 msgr2=0x559d4e9d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff469510640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e9d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff469510640 1 -- 172.21.15.5:0/564759169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d4e9d6520 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46a512640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e9d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff469510640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95b0a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x559d4e95f060 tx=0x7ff454009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2804f78a1c1bd83d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff44d7fa640 1 -- 172.21.15.5:0/564759169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454025d00 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff44d7fa640 1 -- 172.21.15.5:0/564759169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff454025450 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff46009a680 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff44d7fa640 1 -- 172.21.15.5:0/564759169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff454033050 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 10 client.16608 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d4e9d7650 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff44d7fa640 1 -- 172.21.15.5:0/564759169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff46009a680 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff44d7fa640 1 -- 172.21.15.5:0/564759169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff4540035f0 con 0x559d4e957ea0 2024-04-22T13:29:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 10 client.16608 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 20 client.16608 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e9d7e50 0x559d4e9da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 4 client.16608 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.282+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d4e8c3a70 con 0x559d4e9d7e50 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff469d11640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e9d7e50 0x559d4e9da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff469d11640 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e9d7e50 0x559d4e9da270 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec87dcd7e56f451 server_cookie=91ad75f05f8089fe in_seq=0 out_seq=0 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff44d7fa640 10 client.16608 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff44d7fa640 1 -- 172.21.15.5:0/564759169 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d4e8c3a70 con 0x559d4e9d7e50 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff44d7fa640 10 client.16608 handle_command_reply: tid=0 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 client.16608 shutdown 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 msgr2=0x559d4e95b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95b0a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x559d4e95f060 tx=0x7ff454009d20 comp rx=0 tx=0).stop 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 shutdown_connections 2024-04-22T13:29:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559d4e9d7e50 0x559d4e9da270 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d4e957ea0 0x559d4e95b0a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559d4e960a80 0x559d4e9d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 --2- 172.21.15.5:0/564759169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d4e957580 0x559d4e95ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff44d7fa640 0 client.16608 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.283+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 >> 172.21.15.5:0/564759169 conn(0x559d4e951cb0 msgr2=0x559d4e953970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 shutdown_connections 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46eb67740 1 -- 172.21.15.5:0/564759169 wait complete. 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46eb67740 20 client.16608 trim_cache size 0 max 16384 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:29:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:29:25.289 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4500a3da0 msgr2=0x7ff4500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4500a3da0 0x7ff4500a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff45c04c000 tx=0x7ff45c045000 comp rx=0 tx=0).stop 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 msgr2=0x7ff4641a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 0x7ff4641a05d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff46000d950 tx=0x7ff46000de10 comp rx=0 tx=0).stop 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 shutdown_connections 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4500a3da0 0x7ff4500a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff46410e8f0 0x7ff46419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff464069c20 0x7ff4641a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 --2- 172.21.15.5:0/3113140945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff464069300 0x7ff4641a05d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 >> 172.21.15.5:0/3113140945 conn(0x7ff464077fa0 msgr2=0x7ff46410fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 shutdown_connections 2024-04-22T13:29:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.284+0000 7ff46bf75640 1 -- 172.21.15.5:0/3113140945 wait complete. 2024-04-22T13:29:25.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:29:25.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 -- 172.21.15.5:0/4175180084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f78069f10 msgr2=0x7f2f78106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 --2- 172.21.15.5:0/4175180084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f78069f10 0x7f2f78106be0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009900 tx=0x7f2f6c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 -- 172.21.15.5:0/4175180084 shutdown_connections 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 --2- 172.21.15.5:0/4175180084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f78069f10 0x7f2f78106be0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 --2- 172.21.15.5:0/4175180084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f781072c0 0x7f2f7810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 --2- 172.21.15.5:0/4175180084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f78069560 0x7f2f78069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 -- 172.21.15.5:0/4175180084 >> 172.21.15.5:0/4175180084 conn(0x7f2f780771d0 msgr2=0x7f2f780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 -- 172.21.15.5:0/4175180084 shutdown_connections 2024-04-22T13:29:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 -- 172.21.15.5:0/4175180084 wait complete. 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 Processor -- start 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.398+0000 7f2f7e515640 1 -- start start 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7e515640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f78069560 0x7f2f781a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7e515640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 0x7f2f781a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7e515640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f7819a760 0x7f2f7819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7e515640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f78112180 con 0x7f2f781072c0 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7e515640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2f78112000 con 0x7f2f78069560 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7e515640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f78112300 con 0x7f2f7819a760 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 0x7f2f781a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 0x7f2f781a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39068/0 (socket says 172.21.15.5:39068) 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 -- 172.21.15.5:0/556254239 learned_addr learned my addr 172.21.15.5:0/556254239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f77fff640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f78069560 0x7f2f781a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f7cab2640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f7819a760 0x7f2f7819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 -- 172.21.15.5:0/556254239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f7819a760 msgr2=0x7f2f7819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f7819a760 0x7f2f7819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 -- 172.21.15.5:0/556254239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f78069560 msgr2=0x7f2f781a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f78069560 0x7f2f781a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 -- 172.21.15.5:0/556254239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7819b360 con 0x7f2f781072c0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.399+0000 7f2f777fe640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 0x7f2f781a0be0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c036040 tx=0x7f2f6c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.400+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c043020 con 0x7f2f781072c0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.400+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7819b5f0 con 0x7f2f781072c0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.400+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f6c02cbe0 con 0x7f2f781072c0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.400+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c003c30 con 0x7f2f781072c0 2024-04-22T13:29:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.400+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f781a77e0 con 0x7f2f781072c0 2024-04-22T13:29:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.401+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f78103650 con 0x7f2f781072c0 2024-04-22T13:29:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.402+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2f6c046020 con 0x7f2f781072c0 2024-04-22T13:29:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.402+0000 7f2f757fa640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2f600a3d20 0x7f2f600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.402+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2f6c0f2d50 con 0x7f2f781072c0 2024-04-22T13:29:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.403+0000 7f2f77fff640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2f600a3d20 0x7f2f600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.403+0000 7f2f77fff640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2f600a3d20 0x7f2f600a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f680058c0 tx=0x7f2f6800a250 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.406+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f6c04d020 con 0x7f2f781072c0 2024-04-22T13:29:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.632+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f78069fb0 con 0x7f2f781072c0 2024-04-22T13:29:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.633+0000 7f2f757fa640 1 -- 172.21.15.5:0/556254239 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f2f6c0bf530 con 0x7f2f781072c0 2024-04-22T13:29:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:29:25.630 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:29:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2f600a3d20 msgr2=0x7f2f600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2f600a3d20 0x7f2f600a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f680058c0 tx=0x7f2f6800a250 comp rx=0 tx=0).stop 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 msgr2=0x7f2f781a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 0x7f2f781a0be0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c036040 tx=0x7f2f6c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 shutdown_connections 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f781072c0 0x7f2f781a0be0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2f600a3d20 0x7f2f600a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f7819a760 0x7f2f7819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 --2- 172.21.15.5:0/556254239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f78069560 0x7f2f781a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 >> 172.21.15.5:0/556254239 conn(0x7f2f780771d0 msgr2=0x7f2f780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 shutdown_connections 2024-04-22T13:29:25.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.635+0000 7f2f7e515640 1 -- 172.21.15.5:0/556254239 wait complete. 2024-04-22T13:29:25.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:29:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 -- 172.21.15.5:0/2419685511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8810c6d0 msgr2=0x7f9e8810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/2419685511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8810c6d0 0x7f9e8810eb60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ca0 tx=0x7f9e8002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 -- 172.21.15.5:0/2419685511 shutdown_connections 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/2419685511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8810c6d0 0x7f9e8810eb60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/2419685511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e88103b70 0x7f9e8810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/2419685511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e881031c0 0x7f9e881035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 -- 172.21.15.5:0/2419685511 >> 172.21.15.5:0/2419685511 conn(0x7f9e88077f70 msgr2=0x7f9e88078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 -- 172.21.15.5:0/2419685511 shutdown_connections 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.750+0000 7f9e8cc69640 1 -- 172.21.15.5:0/2419685511 wait complete. 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 Processor -- start 2024-04-22T13:29:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 -- start start 2024-04-22T13:29:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e881031c0 0x7f9e88100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e8810c6d0 0x7f9e88101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 0x7f9e881a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881123a0 con 0x7f9e88101890 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112220 con 0x7f9e881031c0 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8cc69640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112520 con 0x7f9e8810c6d0 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e86d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 0x7f9e881a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e86d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 0x7f9e881a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39082/0 (socket says 172.21.15.5:39082) 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e86d9d640 1 -- 172.21.15.5:0/1226930662 learned_addr learned my addr 172.21.15.5:0/1226930662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.751+0000 7f9e8659c640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e881031c0 0x7f9e88100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e8810c6d0 0x7f9e88101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e86d9d640 1 -- 172.21.15.5:0/1226930662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e8810c6d0 msgr2=0x7f9e88101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e86d9d640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e8810c6d0 0x7f9e88101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e86d9d640 1 -- 172.21.15.5:0/1226930662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e881031c0 msgr2=0x7f9e88100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e86d9d640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e881031c0 0x7f9e88100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e86d9d640 1 -- 172.21.15.5:0/1226930662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e881a71d0 con 0x7f9e88101890 2024-04-22T13:29:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e86d9d640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 0x7f9e881a6b30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005e30 tx=0x7f9e80034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/1226930662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80039040 con 0x7f9e88101890 2024-04-22T13:29:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/1226930662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e80044070 con 0x7f9e88101890 2024-04-22T13:29:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e881a7460 con 0x7f9e88101890 2024-04-22T13:29:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/1226930662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80003e60 con 0x7f9e88101890 2024-04-22T13:29:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.752+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881a79a0 con 0x7f9e88101890 2024-04-22T13:29:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.754+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/1226930662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9e800324c0 con 0x7f9e88101890 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 Processor -- start 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 -- start start 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5614a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa56153430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa560af1f0 con 0x55fa56153a00 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fa5604c4b0 con 0x55fa5614a4d0 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8f857740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa560ab1b0 con 0x55fa5614ae80 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39098/0 (socket says 172.21.15.5:39098) 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2770826676 learned_addr learned my addr 172.21.15.5:0/2770826676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e7f7fe640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e740a3da0 0x7f9e740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e8659c640 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa56153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5614a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.755+0000 7f9e86d9d640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e740a3da0 0x7f9e740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7f7fe640 1 -- 172.21.15.5:0/1226930662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f9e800f28d0 con 0x7f9e88101890 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2770826676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 msgr2=0x55fa56153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa56153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2770826676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 msgr2=0x55fa5614a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5614a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2770826676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5603e1b0 con 0x55fa56153a00 2024-04-22T13:29:25.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56155e50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9e70013580 tx=0x7f9e7000f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfde79cf0ea7da9b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e86d9d640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e740a3da0 0x7f9e740a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e800f0210 tx=0x7f9e80032a70 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7cff9640 1 -- 172.21.15.5:0/2770826676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70019030 con 0x55fa56153a00 2024-04-22T13:29:25.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7cff9640 1 -- 172.21.15.5:0/2770826676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e700054d0 con 0x55fa56153a00 2024-04-22T13:29:25.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.756+0000 7f9e7cff9640 1 -- 172.21.15.5:0/2770826676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e700116e0 con 0x55fa56153a00 2024-04-22T13:29:25.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 -- 172.21.15.5:0/2770826676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 msgr2=0x55fa56155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56155e50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9e70013580 tx=0x7f9e7000f620 comp rx=0 tx=0).stop 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 -- 172.21.15.5:0/2770826676 shutdown_connections 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56155e50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa56153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2770826676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5614a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 -- 172.21.15.5:0/2770826676 >> 172.21.15.5:0/2770826676 conn(0x55fa56144ba0 msgr2=0x55fa56144f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 -- 172.21.15.5:0/2770826676 shutdown_connections 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 -- 172.21.15.5:0/2770826676 wait complete. 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 -- start start 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 Processor -- start 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa56153320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.757+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa561506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8f857740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa56156970 con 0x55fa56153a00 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8f857740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fa56068500 con 0x55fa5614a4d0 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8f857740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa56156af0 con 0x55fa5614ae80 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8f857740 10 client.0 _ll_register_callbacks cb 0x55fa56036b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39100/0 (socket says 172.21.15.5:39100) 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2000366219 learned_addr learned my addr 172.21.15.5:0/2000366219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8659c640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa561506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa56153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 msgr2=0x55fa561506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa561506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2000366219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 msgr2=0x55fa56153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa56153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 -- 172.21.15.5:0/2000366219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa55dc13f0 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56150c50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9e70005d00 tx=0x7f9e70005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38dcab655bb7628 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e6dffb640 1 -- 172.21.15.5:0/2000366219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70019040 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e6dffb640 1 -- 172.21.15.5:0/2000366219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f9e70005120 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.758+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e700054d0 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e6dffb640 1 -- 172.21.15.5:0/2000366219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e70005320 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 10 client.16626 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa55e8b7d0 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e6dffb640 1 -- 172.21.15.5:0/2000366219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e700054d0 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e6dffb640 1 -- 172.21.15.5:0/2000366219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9e70003c70 con 0x55fa56153a00 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 10 client.16626 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 10 client.16626 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:29:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 20 client.16626 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa56163a80 0x55fa56165e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 4 client.16626 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5604e930 con 0x55fa56163a80 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.759+0000 7f9e8659c640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa56163a80 0x55fa56165e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.760+0000 7f9e6dffb640 10 client.16626 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.760+0000 7f9e8659c640 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa56163a80 0x55fa56165e70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=191d3a093725fea9 server_cookie=b91b54b7fe437040 in_seq=0 out_seq=0 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.760+0000 7f9e6dffb640 1 -- 172.21.15.5:0/2000366219 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa5604e930 con 0x55fa56163a80 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.760+0000 7f9e6dffb640 10 client.16626 handle_command_reply: tid=0 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 client.16626 shutdown 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 msgr2=0x55fa56150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56150c50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9e70005d00 tx=0x7f9e70005c50 comp rx=0 tx=0).stop 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 shutdown_connections 2024-04-22T13:29:25.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa56163a80 0x55fa56165e70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa56150c50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa561506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 --2- 172.21.15.5:0/2000366219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa56153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e6dffb640 0 client.16626 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 >> 172.21.15.5:0/2000366219 conn(0x55fa56144ba0 msgr2=0x55fa561468f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 shutdown_connections 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 1 -- 172.21.15.5:0/2000366219 wait complete. 2024-04-22T13:29:25.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.761+0000 7f9e8f857740 20 client.16626 trim_cache size 0 max 16384 2024-04-22T13:29:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 Processor -- start 2024-04-22T13:29:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 -- start start 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 0x55fa56143a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa56153a00 0x55fa56152de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881afd00 con 0x55fa5614ae80 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b1840 con 0x55fa5614a4d0 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8f857740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b03d0 con 0x55fa56153a00 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.774+0000 7f9e8659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 0x55fa56143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 0x55fa56143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39104/0 (socket says 172.21.15.5:39104) 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 -- 172.21.15.5:0/1902207119 learned_addr learned my addr 172.21.15.5:0/1902207119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa56153a00 0x55fa56152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 -- 172.21.15.5:0/1902207119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa56153a00 msgr2=0x55fa56152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa56153a00 0x55fa56152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 -- 172.21.15.5:0/1902207119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 msgr2=0x55fa5615f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 -- 172.21.15.5:0/1902207119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5614ac20 con 0x55fa5614ae80 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e8659c640 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 0x55fa56143a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55fa56149fa0 tx=0x7f9e7800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08832a4194b415 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e6ffff640 1 -- 172.21.15.5:0/1902207119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78005220 con 0x55fa5614ae80 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e6ffff640 1 -- 172.21.15.5:0/1902207119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78024030 con 0x55fa5614ae80 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.775+0000 7f9e6ffff640 1 -- 172.21.15.5:0/1902207119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78020420 con 0x55fa5614ae80 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- 172.21.15.5:0/1902207119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 msgr2=0x55fa56143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 0x55fa56143a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55fa56149fa0 tx=0x7f9e7800a250 comp rx=0 tx=0).stop 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- 172.21.15.5:0/1902207119 shutdown_connections 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa5614ae80 0x55fa56143a80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa56153a00 0x55fa56152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- 172.21.15.5:0/1902207119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- 172.21.15.5:0/1902207119 >> 172.21.15.5:0/1902207119 conn(0x55fa56144ba0 msgr2=0x55fa56143370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- 172.21.15.5:0/1902207119 shutdown_connections 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- 172.21.15.5:0/1902207119 wait complete. 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- start start 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 Processor -- start 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa5614d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa5614d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b1170 con 0x55fa56153a00 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b0aa0 con 0x55fa5614a4d0 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881add00 con 0x55fa5614ae80 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e8f857740 10 client.0 _ll_register_callbacks cb 0x7f9e70004e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.776+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa5614d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa5614d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39118/0 (socket says 172.21.15.5:39118) 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 -- 172.21.15.5:0/3698014889 learned_addr learned my addr 172.21.15.5:0/3698014889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e8659c640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa5614d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 -- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 msgr2=0x55fa5614d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa5614d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 -- 172.21.15.5:0/3698014889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 msgr2=0x55fa5615c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 -- 172.21.15.5:0/3698014889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5614dee0 con 0x55fa56153a00 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e7ffff640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa5614d6b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006010 tx=0x7f9e700049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337126944573dc7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e6dffb640 1 -- 172.21.15.5:0/3698014889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70004b70 con 0x55fa56153a00 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e6dffb640 1 -- 172.21.15.5:0/3698014889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f9e70027030 con 0x55fa56153a00 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e78024030 con 0x55fa56153a00 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e6dffb640 1 -- 172.21.15.5:0/3698014889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e7002d0d0 con 0x55fa56153a00 2024-04-22T13:29:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e8f857740 10 client.16632 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.777+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa561c9dd0 con 0x55fa56153a00 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e6dffb640 1 -- 172.21.15.5:0/3698014889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78024030 con 0x55fa56153a00 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e6dffb640 1 -- 172.21.15.5:0/3698014889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9e70005050 con 0x55fa56153a00 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8f857740 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8f857740 10 client.16632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8f857740 20 client.16632 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8f857740 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa561ca4c0 0x55fa561cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8f857740 4 client.16632 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa560bcc10 con 0x55fa561ca4c0 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8659c640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa561ca4c0 0x55fa561cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e8659c640 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa561ca4c0 0x55fa561cc8e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f44e05170b2c0a61 server_cookie=a35b41eeeb20cf31 in_seq=0 out_seq=0 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.778+0000 7f9e6dffb640 10 client.16632 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e6dffb640 1 -- 172.21.15.5:0/3698014889 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa560bcc10 con 0x55fa561ca4c0 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e6dffb640 10 client.16632 handle_command_reply: tid=0 2024-04-22T13:29:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 client.16632 shutdown 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 msgr2=0x55fa5614d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa5614d6b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006010 tx=0x7f9e700049d0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 shutdown_connections 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa561ca4c0 0x55fa561cc8e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa56153a00 0x55fa5614d6b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa5614ae80 0x55fa5614d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e6dffb640 0 client.16632 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 --2- 172.21.15.5:0/3698014889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa5614a4d0 0x55fa5615c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 >> 172.21.15.5:0/3698014889 conn(0x55fa56144ba0 msgr2=0x55fa56143c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 shutdown_connections 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 1 -- 172.21.15.5:0/3698014889 wait complete. 2024-04-22T13:29:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8f857740 20 client.16632 trim_cache size 0 max 16384 2024-04-22T13:29:25.784 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e740a3da0 msgr2=0x7f9e740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e740a3da0 0x7f9e740a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e800f0210 tx=0x7f9e80032a70 comp rx=0 tx=0).stop 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 msgr2=0x7f9e881a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 0x7f9e881a6b30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005e30 tx=0x7f9e80034760 comp rx=0 tx=0).stop 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 shutdown_connections 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e88101890 0x7f9e881a6b30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e740a3da0 0x7f9e740a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e8810c6d0 0x7f9e88101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 --2- 172.21.15.5:0/1226930662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e881031c0 0x7f9e88100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.779+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 >> 172.21.15.5:0/1226930662 conn(0x7f9e88077f70 msgr2=0x7f9e880fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.780+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 shutdown_connections 2024-04-22T13:29:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.780+0000 7f9e8cc69640 1 -- 172.21.15.5:0/1226930662 wait complete. 2024-04-22T13:29:25.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:29:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.895+0000 7f3ebef2e640 1 -- 172.21.15.5:0/459831121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8101240 msgr2=0x7f3eb810d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.895+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/459831121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8101240 0x7f3eb810d530 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009ca0 tx=0x7f3eb002c180 comp rx=0 tx=0).stop 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- 172.21.15.5:0/459831121 shutdown_connections 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/459831121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8101240 0x7f3eb810d530 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/459831121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3eb8100880 0x7f3eb8100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/459831121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb8102510 0x7f3eb81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- 172.21.15.5:0/459831121 >> 172.21.15.5:0/459831121 conn(0x7f3eb80760d0 msgr2=0x7f3eb80764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- 172.21.15.5:0/459831121 shutdown_connections 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- 172.21.15.5:0/459831121 wait complete. 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 Processor -- start 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- start start 2024-04-22T13:29:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 0x7f3eb8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb8101240 0x7f3eb8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3eb8102510 0x7f3eb819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81111f0 con 0x7f3eb8100880 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8111070 con 0x7f3eb8102510 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebef2e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8111370 con 0x7f3eb8101240 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebccca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 0x7f3eb8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.896+0000 7f3ebccca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 0x7f3eb8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39132/0 (socket says 172.21.15.5:39132) 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 -- 172.21.15.5:0/3779643813 learned_addr learned my addr 172.21.15.5:0/3779643813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb8101240 0x7f3eb8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3eb8102510 0x7f3eb819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 -- 172.21.15.5:0/3779643813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb8101240 msgr2=0x7f3eb8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb8101240 0x7f3eb8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 -- 172.21.15.5:0/3779643813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3eb8102510 msgr2=0x7f3eb819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3eb8102510 0x7f3eb819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 -- 172.21.15.5:0/3779643813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb819d750 con 0x7f3eb8100880 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebccca640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 0x7f3eb8198760 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d770 tx=0x7f3ea800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3eb5ffb640 1 -- 172.21.15.5:0/3779643813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8017070 con 0x7f3eb8100880 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3eb5ffb640 1 -- 172.21.15.5:0/3779643813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea80054d0 con 0x7f3eb8100880 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb819da40 con 0x7f3eb8100880 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3eb5ffb640 1 -- 172.21.15.5:0/3779643813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8004c30 con 0x7f3eb8100880 2024-04-22T13:29:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.897+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb81a52d0 con 0x7f3eb8100880 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.900+0000 7f3eb5ffb640 1 -- 172.21.15.5:0/3779643813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3ea8004e10 con 0x7f3eb8100880 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.900+0000 7f3ec1bf5740 1 Processor -- start 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.900+0000 7f3ec1bf5740 1 -- start start 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.900+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3e52121f0 con 0x55f3e52ad4d0 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f3e51af4b0 con 0x55f3e52ade80 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f3e520e1b0 con 0x55f3e52b6a00 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39140/0 (socket says 172.21.15.5:39140) 2024-04-22T13:29:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/2258985886 learned_addr learned my addr 172.21.15.5:0/2258985886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebccca640 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3eb5ffb640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3ea00a3da0 0x7f3ea00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/2258985886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 msgr2=0x55f3e52b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/2258985886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 msgr2=0x55f3e52b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/2258985886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e51a11b0 con 0x55f3e52ad4d0 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3eb5ffb640 1 -- 172.21.15.5:0/3779643813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3ea80d0220 con 0x7f3eb8100880 2024-04-22T13:29:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3ea00a3da0 0x7f3ea00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.901+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52ad8b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009ca0 tx=0x7f3eb0032400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7530bb91d0157a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3e9effd640 1 -- 172.21.15.5:0/2258985886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0039040 con 0x55f3e52ad4d0 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3e9effd640 1 -- 172.21.15.5:0/2258985886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb0032560 con 0x55f3e52ad4d0 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3e9effd640 1 -- 172.21.15.5:0/2258985886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0034680 con 0x55f3e52ad4d0 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/2258985886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 msgr2=0x55f3e52ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3ea00a3da0 0x7f3ea00a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3eb004b000 tx=0x7f3eb0044000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52ad8b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009ca0 tx=0x7f3eb0032400 comp rx=0 tx=0).stop 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/2258985886 shutdown_connections 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/2258985886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52ad8b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/2258985886 >> 172.21.15.5:0/2258985886 conn(0x55f3e52a7ba0 msgr2=0x55f3e52a7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/2258985886 shutdown_connections 2024-04-22T13:29:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.902+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/2258985886 wait complete. 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 -- start start 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 Processor -- start 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ad4d0 0x55f3e52b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 0x55f3e52b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3e52b9970 con 0x55f3e52b6a00 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f3e51cb500 con 0x55f3e52ade80 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f3e52b9af0 con 0x55f3e52ad4d0 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ec1bf5740 10 client.0 _ll_register_callbacks cb 0x55f3e5199b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ebd4cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 0x55f3e52b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ebd4cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 0x55f3e52b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39152/0 (socket says 172.21.15.5:39152) 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.903+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/4109003930 learned_addr learned my addr 172.21.15.5:0/4109003930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebccca640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ad4d0 0x55f3e52b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/4109003930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ad4d0 msgr2=0x55f3e52b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ad4d0 0x55f3e52b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 msgr2=0x55f3e52b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebd4cb640 1 -- 172.21.15.5:0/4109003930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e4f243f0 con 0x55f3e52b6a00 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 0x55f3e52b3bd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005dc0 tx=0x7f3eb0004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d0aee199d24369 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3e97fff640 1 -- 172.21.15.5:0/4109003930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0059060 con 0x55f3e52b6a00 2024-04-22T13:29:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eb0032560 con 0x55f3e52b6a00 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3e97fff640 1 -- 172.21.15.5:0/4109003930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3eb0034440 con 0x55f3e52b6a00 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3e97fff640 1 -- 172.21.15.5:0/4109003930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3eb0034640 con 0x55f3e52b6a00 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ec1bf5740 10 client.16644 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3e4fee7d0 con 0x55f3e52b6a00 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.904+0000 7f3e97fff640 1 -- 172.21.15.5:0/4109003930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb0032560 con 0x55f3e52b6a00 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3e97fff640 1 -- 172.21.15.5:0/4109003930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3eb003f070 con 0x55f3e52b6a00 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ec1bf5740 10 client.16644 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ec1bf5740 10 client.16644 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ec1bf5740 20 client.16644 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e52c6a60 0x55f3e52c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ec1bf5740 4 client.16644 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3e51b1930 con 0x55f3e52c6a60 2024-04-22T13:29:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ebccca640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e52c6a60 0x55f3e52c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3e97fff640 10 client.16644 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.905+0000 7f3ebccca640 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e52c6a60 0x55f3e52c8e50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d61a4c86d8c6a021 server_cookie=28e140b9953c4f5a in_seq=0 out_seq=0 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3e97fff640 1 -- 172.21.15.5:0/4109003930 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3e51b1930 con 0x55f3e52c6a60 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3e97fff640 10 client.16644 handle_command_reply: tid=0 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 client.16644 shutdown 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 msgr2=0x55f3e52b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 0x55f3e52b3bd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005dc0 tx=0x7f3eb0004980 comp rx=0 tx=0).stop 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 shutdown_connections 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e52c6a60 0x55f3e52c8e50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52b6a00 0x55f3e52b3bd0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3e97fff640 0 client.16644 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/4109003930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ad4d0 0x55f3e52b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 >> 172.21.15.5:0/4109003930 conn(0x55f3e52a7ba0 msgr2=0x55f3e52a98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 shutdown_connections 2024-04-22T13:29:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.906+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/4109003930 wait complete. 2024-04-22T13:29:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.907+0000 7f3ec1bf5740 20 client.16644 trim_cache size 0 max 16384 2024-04-22T13:29:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 Processor -- start 2024-04-22T13:29:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 -- start start 2024-04-22T13:29:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ade80 0x55f3e52b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52b6a00 0x55f3e52a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81ada30 con 0x55f3e52ad4d0 2024-04-22T13:29:25.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81af570 con 0x55f3e52b6a00 2024-04-22T13:29:25.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81ae100 con 0x55f3e52ade80 2024-04-22T13:29:25.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39158/0 (socket says 172.21.15.5:39158) 2024-04-22T13:29:25.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 -- 172.21.15.5:0/250502286 learned_addr learned my addr 172.21.15.5:0/250502286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ade80 0x55f3e52b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52b6a00 0x55f3e52a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 -- 172.21.15.5:0/250502286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ade80 msgr2=0x55f3e52b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ade80 0x55f3e52b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 -- 172.21.15.5:0/250502286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52b6a00 msgr2=0x55f3e52a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52b6a00 0x55f3e52a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 -- 172.21.15.5:0/250502286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e52b1c00 con 0x55f3e52ad4d0 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52b6a00 0x55f3e52a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.920+0000 7f3ebccca640 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52c2830 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8098650 tx=0x7f3ea80d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e43d4a4b589628 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3e9e7fc640 1 -- 172.21.15.5:0/250502286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80b4070 con 0x55f3e52ad4d0 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3e9e7fc640 1 -- 172.21.15.5:0/250502286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8005680 con 0x55f3e52ad4d0 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3e9e7fc640 1 -- 172.21.15.5:0/250502286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8020040 con 0x55f3e52ad4d0 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/250502286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 msgr2=0x55f3e52c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52c2830 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8098650 tx=0x7f3ea80d2040 comp rx=0 tx=0).stop 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/250502286 shutdown_connections 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52b6a00 0x55f3e52a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52ade80 0x55f3e52b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/250502286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52c2830 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/250502286 >> 172.21.15.5:0/250502286 conn(0x55f3e52a7ba0 msgr2=0x55f3e52a8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/250502286 shutdown_connections 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.921+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/250502286 wait complete. 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 -- start start 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 Processor -- start 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81aeea0 con 0x55f3e52ad4d0 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81ae7d0 con 0x55f3e52ade80 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81aba10 con 0x55f3e52b6a00 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ec1bf5740 10 client.0 _ll_register_callbacks cb 0x7f3eb0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52bf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39172/0 (socket says 172.21.15.5:39172) 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 -- 172.21.15.5:0/3183257745 learned_addr learned my addr 172.21.15.5:0/3183257745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebd4cb640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 -- 172.21.15.5:0/3183257745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 msgr2=0x55f3e52b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 -- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 msgr2=0x55f3e52b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3ebccca640 1 -- 172.21.15.5:0/3183257745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e52b0ec0 con 0x55f3e52ad4d0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.922+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ebccca640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52bf220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80cdfa0 tx=0x7f3ea80a4e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb19852e24bf2d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3e97fff640 1 -- 172.21.15.5:0/3183257745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80b9020 con 0x55f3e52ad4d0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3e97fff640 1 -- 172.21.15.5:0/3183257745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3ea80cdc20 con 0x55f3e52ad4d0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea8005680 con 0x55f3e52ad4d0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3e97fff640 1 -- 172.21.15.5:0/3183257745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ea80b4030 con 0x55f3e52ad4d0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 10 client.16650 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3e532cd00 con 0x55f3e52ad4d0 2024-04-22T13:29:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3e97fff640 1 -- 172.21.15.5:0/3183257745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8005680 con 0x55f3e52ad4d0 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3e97fff640 1 -- 172.21.15.5:0/3183257745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3ea80a0170 con 0x55f3e52ad4d0 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 10 client.16650 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 10 client.16650 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 20 client.16650 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e532d4d0 0x55f3e532f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 4 client.16650 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3e521fc10 con 0x55f3e532d4d0 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.923+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e532d4d0 0x55f3e532f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3eb7fff640 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e532d4d0 0x55f3e532f8f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5870ccd2496c4daa server_cookie=52a245de6caa692d in_seq=0 out_seq=0 2024-04-22T13:29:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3e97fff640 10 client.16650 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3e97fff640 1 -- 172.21.15.5:0/3183257745 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3e521fc10 con 0x55f3e532d4d0 2024-04-22T13:29:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3e97fff640 10 client.16650 handle_command_reply: tid=0 2024-04-22T13:29:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 client.16650 shutdown 2024-04-22T13:29:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 msgr2=0x55f3e52bf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52bf220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80cdfa0 tx=0x7f3ea80a4e60 comp rx=0 tx=0).stop 2024-04-22T13:29:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 shutdown_connections 2024-04-22T13:29:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55f3e532d4d0 0x55f3e532f8f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3e52b6a00 0x55f3e52b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3e52ade80 0x55f3e52b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3e97fff640 0 client.16650 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 --2- 172.21.15.5:0/3183257745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3e52ad4d0 0x55f3e52bf220 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 >> 172.21.15.5:0/3183257745 conn(0x55f3e52a7ba0 msgr2=0x55f3e52a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 shutdown_connections 2024-04-22T13:29:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 1 -- 172.21.15.5:0/3183257745 wait complete. 2024-04-22T13:29:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.924+0000 7f3ec1bf5740 20 client.16650 trim_cache size 0 max 16384 2024-04-22T13:29:25.938 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3ea00a3da0 msgr2=0x7f3ea00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3ea00a3da0 0x7f3ea00a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3eb004b000 tx=0x7f3eb0044000 comp rx=0 tx=0).stop 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 msgr2=0x7f3eb8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 0x7f3eb8198760 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d770 tx=0x7f3ea800dc30 comp rx=0 tx=0).stop 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 shutdown_connections 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3ea00a3da0 0x7f3ea00a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3eb8102510 0x7f3eb819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb8101240 0x7f3eb8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 --2- 172.21.15.5:0/3779643813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb8100880 0x7f3eb8198760 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 >> 172.21.15.5:0/3779643813 conn(0x7f3eb80760d0 msgr2=0x7f3eb81055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 shutdown_connections 2024-04-22T13:29:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:25.925+0000 7f3ebef2e640 1 -- 172.21.15.5:0/3779643813 wait complete. 2024-04-22T13:29:25.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:29:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 -- 172.21.15.5:0/4279011036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434102180 msgr2=0x7f8434102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 --2- 172.21.15.5:0/4279011036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434102180 0x7f8434102560 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8424009900 tx=0x7f842402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 -- 172.21.15.5:0/4279011036 shutdown_connections 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 --2- 172.21.15.5:0/4279011036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8434107250 0x7f843410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 --2- 172.21.15.5:0/4279011036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8434102b30 0x7f8434106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 --2- 172.21.15.5:0/4279011036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434102180 0x7f8434102560 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 -- 172.21.15.5:0/4279011036 >> 172.21.15.5:0/4279011036 conn(0x7f84340fda00 msgr2=0x7f84340ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 -- 172.21.15.5:0/4279011036 shutdown_connections 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 -- 172.21.15.5:0/4279011036 wait complete. 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.048+0000 7f843a1b6640 1 Processor -- start 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 -- start start 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8434102180 0x7f843419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 0x7f843419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84341985e0 0x7f8434198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84341121c0 con 0x7f8434107250 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112040 con 0x7f8434102180 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f843a1b6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112340 con 0x7f84341985e0 2024-04-22T13:29:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 0x7f843419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 0x7f843419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39198/0 (socket says 172.21.15.5:39198) 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 -- 172.21.15.5:0/3147392961 learned_addr learned my addr 172.21.15.5:0/3147392961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8433fff640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84341985e0 0x7f8434198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f84337fe640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8434102180 0x7f843419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 -- 172.21.15.5:0/3147392961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84341985e0 msgr2=0x7f8434198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84341985e0 0x7f8434198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 -- 172.21.15.5:0/3147392961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8434102180 msgr2=0x7f843419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8434102180 0x7f843419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 -- 172.21.15.5:0/3147392961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8434199100 con 0x7f8434107250 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.049+0000 7f8432ffd640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 0x7f843419e970 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8428005fd0 tx=0x7f842800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.050+0000 7f8430ff9640 1 -- 172.21.15.5:0/3147392961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428017070 con 0x7f8434107250 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.050+0000 7f8430ff9640 1 -- 172.21.15.5:0/3147392961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8428009c30 con 0x7f8434107250 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.050+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341993f0 con 0x7f8434107250 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.050+0000 7f8430ff9640 1 -- 172.21.15.5:0/3147392961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428003d30 con 0x7f8434107250 2024-04-22T13:29:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.050+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a5640 con 0x7f8434107250 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.052+0000 7f8430ff9640 1 -- 172.21.15.5:0/3147392961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8428003ed0 con 0x7f8434107250 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.052+0000 7f843cdad740 1 Processor -- start 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.052+0000 7f843cdad740 1 -- start start 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.052+0000 7f843cdad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.052+0000 7f843cdad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.052+0000 7f843cdad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f843cdad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623964471f0 con 0x5623964eba00 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f843cdad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5623963e44b0 con 0x5623964e24d0 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f843cdad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623964431b0 con 0x5623964e2e80 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39206/0 (socket says 172.21.15.5:39206) 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8432ffd640 1 -- 172.21.15.5:0/1960747274 learned_addr learned my addr 172.21.15.5:0/1960747274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8430ff9640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f841c0a3d20 0x7f841c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f84337fe640 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8430ff9640 1 -- 172.21.15.5:0/3147392961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f84280d04d0 con 0x7f8434107250 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f841c0a3d20 0x7f841c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 -- 172.21.15.5:0/1960747274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 msgr2=0x5623964eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 -- 172.21.15.5:0/1960747274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 msgr2=0x5623964ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8433fff640 1 -- 172.21.15.5:0/1960747274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623963d61b0 con 0x5623964e24d0 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.053+0000 7f8432ffd640 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.054+0000 7f8433fff640 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964e28b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f842c009f90 tx=0x7f842c0149a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa57d4eccf5ca66f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.054+0000 7f8419ffb640 1 -- 172.21.15.5:0/1960747274 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c027070 con 0x5623964e24d0 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.054+0000 7f8419ffb640 1 -- 172.21.15.5:0/1960747274 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c019450 con 0x5623964e24d0 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.054+0000 7f8419ffb640 1 -- 172.21.15.5:0/1960747274 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c00c5c0 con 0x5623964e24d0 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.054+0000 7f8433fff640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f841c0a3d20 0x7f841c0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f842c024000 tx=0x7f842c01f000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 -- 172.21.15.5:0/1960747274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 msgr2=0x5623964e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964e28b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f842c009f90 tx=0x7f842c0149a0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 -- 172.21.15.5:0/1960747274 shutdown_connections 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 --2- 172.21.15.5:0/1960747274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964e28b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 -- 172.21.15.5:0/1960747274 >> 172.21.15.5:0/1960747274 conn(0x5623964dcba0 msgr2=0x5623964dcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 -- 172.21.15.5:0/1960747274 shutdown_connections 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 -- 172.21.15.5:0/1960747274 wait complete. 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 -- start start 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.055+0000 7f843cdad740 1 Processor -- start 2024-04-22T13:29:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e24d0 0x5623964eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e2e80 0x5623964e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623964ee970 con 0x5623964eba00 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562396400500 con 0x5623964e2e80 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623964eeaf0 con 0x5623964e24d0 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 10 client.0 _ll_register_callbacks cb 0x5623963ceb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964e8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39218/0 (socket says 172.21.15.5:39218) 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 -- 172.21.15.5:0/911748211 learned_addr learned my addr 172.21.15.5:0/911748211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8432ffd640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e2e80 0x5623964e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f84337fe640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e24d0 0x5623964eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 -- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e24d0 msgr2=0x5623964eb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e24d0 0x5623964eb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 -- 172.21.15.5:0/911748211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e2e80 msgr2=0x5623964e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e2e80 0x5623964e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 -- 172.21.15.5:0/911748211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623961593f0 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8433fff640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964e8c50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f842c015010 tx=0x7f842c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a747d0741a964a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8412ffd640 1 -- 172.21.15.5:0/911748211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c027040 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f8412ffd640 1 -- 172.21.15.5:0/911748211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f842c004c00 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.056+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842c019450 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f8412ffd640 1 -- 172.21.15.5:0/911748211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f842c004e00 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 10 client.16662 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623962237d0 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f8412ffd640 1 -- 172.21.15.5:0/911748211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c019450 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f8412ffd640 1 -- 172.21.15.5:0/911748211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f842c020040 con 0x5623964eba00 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 10 client.16662 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 10 client.16662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 20 client.16662 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5623964fbb60 0x5623964fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 4 client.16662 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623963e6930 con 0x5623964fbb60 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.057+0000 7f84337fe640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5623964fbb60 0x5623964fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.058+0000 7f8412ffd640 10 client.16662 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.058+0000 7f84337fe640 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5623964fbb60 0x5623964fdf50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9abf59caa6453ff5 server_cookie=a397afa668bd6cfa in_seq=0 out_seq=0 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f8412ffd640 1 -- 172.21.15.5:0/911748211 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623963e6930 con 0x5623964fbb60 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f8412ffd640 10 client.16662 handle_command_reply: tid=0 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 client.16662 shutdown 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 msgr2=0x5623964e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964e8c50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f842c015010 tx=0x7f842c009c90 comp rx=0 tx=0).stop 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 shutdown_connections 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5623964fbb60 0x5623964fdf50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964e8c50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e2e80 0x5623964e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f8412ffd640 0 client.16662 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 --2- 172.21.15.5:0/911748211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e24d0 0x5623964eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 >> 172.21.15.5:0/911748211 conn(0x5623964dcba0 msgr2=0x5623964de8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 shutdown_connections 2024-04-22T13:29:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 1 -- 172.21.15.5:0/911748211 wait complete. 2024-04-22T13:29:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.059+0000 7f843cdad740 20 client.16662 trim_cache size 0 max 16384 2024-04-22T13:29:26.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 Processor -- start 2024-04-22T13:29:26.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 -- start start 2024-04-22T13:29:26.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ab9f0 con 0x5623964eba00 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ad530 con 0x5623964e24d0 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.072+0000 7f843cdad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ac0c0 con 0x5623964e2e80 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964eade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39224/0 (socket says 172.21.15.5:39224) 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 -- 172.21.15.5:0/2528333260 learned_addr learned my addr 172.21.15.5:0/2528333260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8432ffd640 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 -- 172.21.15.5:0/2528333260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 msgr2=0x5623964e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f84337fe640 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 -- 172.21.15.5:0/2528333260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 msgr2=0x5623964f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964f7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 -- 172.21.15.5:0/2528333260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623964fe4e0 con 0x5623964eba00 2024-04-22T13:29:26.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f8433fff640 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964eade0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f842c014ea0 tx=0x7f842c012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a190cf634dfe4e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f84197fa640 1 -- 172.21.15.5:0/2528333260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c049070 con 0x5623964eba00 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f84197fa640 1 -- 172.21.15.5:0/2528333260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c02f070 con 0x5623964eba00 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.073+0000 7f84197fa640 1 -- 172.21.15.5:0/2528333260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c027040 con 0x5623964eba00 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 -- 172.21.15.5:0/2528333260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 msgr2=0x5623964eade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964eade0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f842c014ea0 tx=0x7f842c012f80 comp rx=0 tx=0).stop 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 -- 172.21.15.5:0/2528333260 shutdown_connections 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964eba00 0x5623964eade0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964e2e80 0x5623964e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 --2- 172.21.15.5:0/2528333260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 -- 172.21.15.5:0/2528333260 >> 172.21.15.5:0/2528333260 conn(0x5623964dcba0 msgr2=0x5623964de660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 -- 172.21.15.5:0/2528333260 shutdown_connections 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 -- 172.21.15.5:0/2528333260 wait complete. 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 -- start start 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.074+0000 7f843cdad740 1 Processor -- start 2024-04-22T13:29:26.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 0x5623964efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964eba00 0x5623964f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ace60 con 0x5623964e2e80 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ac790 con 0x5623964e24d0 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84341a9a10 con 0x5623964eba00 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 10 client.0 _ll_register_callbacks cb 0x7f8424009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 0x5623964efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 0x5623964efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39228/0 (socket says 172.21.15.5:39228) 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 -- 172.21.15.5:0/2635091666 learned_addr learned my addr 172.21.15.5:0/2635091666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f84337fe640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8433fff640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964eba00 0x5623964f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 -- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964eba00 msgr2=0x5623964f01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964eba00 0x5623964f01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 -- 172.21.15.5:0/2635091666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 msgr2=0x5623964ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 -- 172.21.15.5:0/2635091666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562396560f80 con 0x5623964e2e80 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8432ffd640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 0x5623964efc60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f842809c1a0 tx=0x7f8428098080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a7cfac23c95d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8412ffd640 1 -- 172.21.15.5:0/2635091666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428098740 con 0x5623964e2e80 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f8412ffd640 1 -- 172.21.15.5:0/2635091666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8428098b10 con 0x5623964e2e80 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.075+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842c02f070 con 0x5623964e2e80 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f8412ffd640 1 -- 172.21.15.5:0/2635091666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8428009de0 con 0x5623964e2e80 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 10 client.16668 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562396562230 con 0x5623964e2e80 2024-04-22T13:29:26.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f8412ffd640 1 -- 172.21.15.5:0/2635091666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c02f070 con 0x5623964e2e80 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f8412ffd640 1 -- 172.21.15.5:0/2635091666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f84280041c0 con 0x5623964e2e80 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 10 client.16668 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 10 client.16668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 20 client.16668 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562396562a00 0x562396564e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 4 client.16668 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562396454c10 con 0x562396562a00 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.076+0000 7f84337fe640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562396562a00 0x562396564e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f8412ffd640 10 client.16668 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f84337fe640 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562396562a00 0x562396564e20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61c95422ca41a605 server_cookie=e3bca299dbbf489b in_seq=0 out_seq=0 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f8412ffd640 1 -- 172.21.15.5:0/2635091666 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562396454c10 con 0x562396562a00 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f8412ffd640 10 client.16668 handle_command_reply: tid=0 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 client.16668 shutdown 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 msgr2=0x5623964efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 0x5623964efc60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f842809c1a0 tx=0x7f8428098080 comp rx=0 tx=0).stop 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 shutdown_connections 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562396562a00 0x562396564e20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623964e2e80 0x5623964efc60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623964eba00 0x5623964f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f8412ffd640 0 client.16668 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 --2- 172.21.15.5:0/2635091666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623964e24d0 0x5623964ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 >> 172.21.15.5:0/2635091666 conn(0x5623964dcba0 msgr2=0x5623964f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 shutdown_connections 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 1 -- 172.21.15.5:0/2635091666 wait complete. 2024-04-22T13:29:26.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.077+0000 7f843cdad740 20 client.16668 trim_cache size 0 max 16384 2024-04-22T13:29:26.082 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:26.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f841c0a3d20 msgr2=0x7f841c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f841c0a3d20 0x7f841c0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f842c024000 tx=0x7f842c01f000 comp rx=0 tx=0).stop 2024-04-22T13:29:26.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 msgr2=0x7f843419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 0x7f843419e970 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8428005fd0 tx=0x7f842800dde0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 shutdown_connections 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8434107250 0x7f843419e970 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f841c0a3d20 0x7f841c0a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84341985e0 0x7f8434198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 --2- 172.21.15.5:0/3147392961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8434102180 0x7f843419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 >> 172.21.15.5:0/3147392961 conn(0x7f84340fda00 msgr2=0x7f84340fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 shutdown_connections 2024-04-22T13:29:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.078+0000 7f843a1b6640 1 -- 172.21.15.5:0/3147392961 wait complete. 2024-04-22T13:29:26.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:29:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 -- 172.21.15.5:0/848990711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 msgr2=0x7f0c0410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/848990711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c0410dd30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 -- 172.21.15.5:0/848990711 shutdown_connections 2024-04-22T13:29:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/848990711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c0410dd30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/848990711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c04101a40 0x7f0c04105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/848990711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c04101090 0x7f0c04101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 -- 172.21.15.5:0/848990711 >> 172.21.15.5:0/848990711 conn(0x7f0c04078000 msgr2=0x7f0c04078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 -- 172.21.15.5:0/848990711 shutdown_connections 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 -- 172.21.15.5:0/848990711 wait complete. 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 Processor -- start 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.193+0000 7f0c0c08c640 1 -- start start 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0c08c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c04101090 0x7f0c041986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0c08c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c04198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0c08c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c0419cfb0 0x7f0c0419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0c08c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041110d0 con 0x7f0c04106160 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0c08c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04110f50 con 0x7f0c0419cfb0 2024-04-22T13:29:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0c08c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111250 con 0x7f0c04101090 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c04198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c04198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39250/0 (socket says 172.21.15.5:39250) 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 -- 172.21.15.5:0/4065526313 learned_addr learned my addr 172.21.15.5:0/4065526313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09e28640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c04101090 0x7f0c041986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0a629640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c0419cfb0 0x7f0c0419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 -- 172.21.15.5:0/4065526313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c04101090 msgr2=0x7f0c041986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c04101090 0x7f0c041986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 -- 172.21.15.5:0/4065526313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c0419cfb0 msgr2=0x7f0c0419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c0419cfb0 0x7f0c0419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 -- 172.21.15.5:0/4065526313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0419daa0 con 0x7f0c04106160 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c0a629640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c0419cfb0 0x7f0c0419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0c09627640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c04198c30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400e9f0 tx=0x7f0bf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.194+0000 7f0bfaffd640 1 -- 172.21.15.5:0/4065526313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400ccc0 con 0x7f0c04106160 2024-04-22T13:29:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.195+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0419dd90 con 0x7f0c04106160 2024-04-22T13:29:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.195+0000 7f0bfaffd640 1 -- 172.21.15.5:0/4065526313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf40054d0 con 0x7f0c04106160 2024-04-22T13:29:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.195+0000 7f0bfaffd640 1 -- 172.21.15.5:0/4065526313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4010430 con 0x7f0c04106160 2024-04-22T13:29:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.195+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041a5530 con 0x7f0c04106160 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0bfaffd640 1 -- 172.21.15.5:0/4065526313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0bf40046c0 con 0x7f0c04106160 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 Processor -- start 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 -- start start 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b2009788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200981430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200981a00 0x55b200983e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2008dd1f0 con 0x55b2009784d0 2024-04-22T13:29:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b20087a4b0 con 0x55b200978e80 2024-04-22T13:29:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2008d91b0 con 0x55b200981a00 2024-04-22T13:29:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0a629640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b2009788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0a629640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b2009788b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39254/0 (socket says 172.21.15.5:39254) 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.197+0000 7f0c0a629640 1 -- 172.21.15.5:0/3633328210 learned_addr learned my addr 172.21.15.5:0/3633328210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c09627640 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200981a00 0x55b200983e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c09e28640 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200981430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0bfaffd640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 -- 172.21.15.5:0/3633328210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200981a00 msgr2=0x55b200983e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200981a00 0x55b200983e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 -- 172.21.15.5:0/3633328210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 msgr2=0x55b200981430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200981430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 -- 172.21.15.5:0/3633328210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b20086c1b0 con 0x55b2009784d0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0bfaffd640 1 -- 172.21.15.5:0/4065526313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0bf40d0b80 con 0x7f0c04106160 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b2009788b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7e65c2889f46b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0beffff640 1 -- 172.21.15.5:0/3633328210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc039040 con 0x55b2009784d0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0beffff640 1 -- 172.21.15.5:0/3633328210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc032e60 con 0x55b2009784d0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0beffff640 1 -- 172.21.15.5:0/3633328210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004850 con 0x55b2009784d0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.198+0000 7f0c0a629640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc04b000 tx=0x7f0bfc044000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3633328210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 msgr2=0x55b2009788b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b2009788b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc009950 comp rx=0 tx=0).stop 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3633328210 shutdown_connections 2024-04-22T13:29:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200981a00 0x55b200983e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200981430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3633328210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b2009788b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3633328210 >> 172.21.15.5:0/3633328210 conn(0x55b200972ba0 msgr2=0x55b200972f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3633328210 shutdown_connections 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3633328210 wait complete. 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.199+0000 7f0c0ec82740 1 -- start start 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 Processor -- start 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b200981340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200978e80 0x55b20097e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b20097ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b200984970 con 0x55b2009784d0 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b200896500 con 0x55b200981a00 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b200984af0 con 0x55b200978e80 2024-04-22T13:29:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0ec82740 10 client.0 _ll_register_callbacks cb 0x55b200864b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b200981340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b200981340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39256/0 (socket says 172.21.15.5:39256) 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 -- 172.21.15.5:0/3556620464 learned_addr learned my addr 172.21.15.5:0/3556620464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c0a629640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b20097ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09627640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200978e80 0x55b20097e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 -- 172.21.15.5:0/3556620464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200978e80 msgr2=0x55b20097e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200978e80 0x55b20097e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 -- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 msgr2=0x55b20097ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b20097ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.200+0000 7f0c09e28640 1 -- 172.21.15.5:0/3556620464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2005ef3f0 con 0x55b2009784d0 2024-04-22T13:29:26.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c09e28640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b200981340 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000bf50 tx=0x7f0c000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d49ace72acbf22 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0bed7fa640 1 -- 172.21.15.5:0/3556620464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00004020 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc032e60 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0bed7fa640 1 -- 172.21.15.5:0/3556620464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0c000041c0 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0bed7fa640 1 -- 172.21.15.5:0/3556620464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c000043c0 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 10 client.16680 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2006b97d0 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0bed7fa640 1 -- 172.21.15.5:0/3556620464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc032e60 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0bed7fa640 1 -- 172.21.15.5:0/3556620464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0c000039c0 con 0x55b2009784d0 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 10 client.16680 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 10 client.16680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 20 client.16680 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b200991cd0 0x55b200994140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 4 client.16680 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.201+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b20087c930 con 0x55b200991cd0 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.202+0000 7f0c09627640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b200991cd0 0x55b200994140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.202+0000 7f0c09627640 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b200991cd0 0x55b200994140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbea935f7b3bd9c3 server_cookie=68355f62ef3eeec1 in_seq=0 out_seq=0 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.202+0000 7f0bed7fa640 10 client.16680 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0bed7fa640 1 -- 172.21.15.5:0/3556620464 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b20087c930 con 0x55b200991cd0 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0bed7fa640 10 client.16680 handle_command_reply: tid=0 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 client.16680 shutdown 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 msgr2=0x55b200981340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b200981340 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000bf50 tx=0x7f0c000118d0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 shutdown_connections 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b200991cd0 0x55b200994140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b20097ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b200978e80 0x55b20097e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/3556620464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2009784d0 0x55b200981340 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0bed7fa640 0 client.16680 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 >> 172.21.15.5:0/3556620464 conn(0x55b200972ba0 msgr2=0x55b2009748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 shutdown_connections 2024-04-22T13:29:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 1 -- 172.21.15.5:0/3556620464 wait complete. 2024-04-22T13:29:26.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.203+0000 7f0c0ec82740 20 client.16680 trim_cache size 0 max 16384 2024-04-22T13:29:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 Processor -- start 2024-04-22T13:29:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 -- start start 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b20098d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 0x55b200974630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b200980e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041adae0 con 0x55b200978e80 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041af620 con 0x55b200981a00 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041ae1b0 con 0x55b2009784d0 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.216+0000 7f0c09627640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 0x55b200974630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 0x55b200974630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39270/0 (socket says 172.21.15.5:39270) 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 -- 172.21.15.5:0/2774739446 learned_addr learned my addr 172.21.15.5:0/2774739446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c0a629640 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b200980e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09e28640 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b20098d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 -- 172.21.15.5:0/2774739446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 msgr2=0x55b20098d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b20098d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 -- 172.21.15.5:0/2774739446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 msgr2=0x55b200980e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b200980e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 -- 172.21.15.5:0/2774739446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b20097cc20 con 0x55b200978e80 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c0a629640 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b200980e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0c09627640 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 0x55b200974630 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c6a0 tx=0x7f0bf40bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f841c2cee9b60d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0bef7fe640 1 -- 172.21.15.5:0/2774739446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4005680 con 0x55b200978e80 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0bef7fe640 1 -- 172.21.15.5:0/2774739446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf409d250 con 0x55b200978e80 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.217+0000 7f0bef7fe640 1 -- 172.21.15.5:0/2774739446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40ce580 con 0x55b200978e80 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- 172.21.15.5:0/2774739446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 msgr2=0x55b200974630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 0x55b200974630 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c6a0 tx=0x7f0bf40bd040 comp rx=0 tx=0).stop 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- 172.21.15.5:0/2774739446 shutdown_connections 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200978e80 0x55b200974630 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200981a00 0x55b200980e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/2774739446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b20098d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- 172.21.15.5:0/2774739446 >> 172.21.15.5:0/2774739446 conn(0x55b200972ba0 msgr2=0x55b200973f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- 172.21.15.5:0/2774739446 shutdown_connections 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- 172.21.15.5:0/2774739446 wait complete. 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- start start 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 Processor -- start 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b200985840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200985d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 0x55b2009862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041aef50 con 0x55b200981a00 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041ae880 con 0x55b200978e80 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.218+0000 7f0c0ec82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041abae0 con 0x55b2009784d0 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0ec82740 10 client.0 _ll_register_callbacks cb 0x7f0c0000ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 0x55b2009862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 0x55b2009862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39284/0 (socket says 172.21.15.5:39284) 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 -- 172.21.15.5:0/1268920942 learned_addr learned my addr 172.21.15.5:0/1268920942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c09e28640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b200985840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c09627640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200985d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 -- 172.21.15.5:0/1268920942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 msgr2=0x55b200985840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b200985840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 -- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 msgr2=0x55b200985d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200985d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 -- 172.21.15.5:0/1268920942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2009f7330 con 0x55b200981a00 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0a629640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 0x55b2009862c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c064dee53eb54f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0bed7fa640 1 -- 172.21.15.5:0/1268920942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc032e60 con 0x55b200981a00 2024-04-22T13:29:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0bed7fa640 1 -- 172.21.15.5:0/1268920942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0bfc004040 con 0x55b200981a00 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf409d250 con 0x55b200981a00 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0bed7fa640 1 -- 172.21.15.5:0/1268920942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc062050 con 0x55b200981a00 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0ec82740 10 client.16686 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.219+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2009f8560 con 0x55b200981a00 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0bed7fa640 1 -- 172.21.15.5:0/1268920942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf409d250 con 0x55b200981a00 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0bed7fa640 1 -- 172.21.15.5:0/1268920942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0bfc053040 con 0x55b200981a00 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c0ec82740 10 client.16686 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c0ec82740 10 client.16686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c0ec82740 20 client.16686 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b2009f8890 0x55b2009fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c0ec82740 4 client.16686 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2008eac10 con 0x55b2009f8890 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.220+0000 7f0c09e28640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b2009f8890 0x55b2009fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c09e28640 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b2009f8890 0x55b2009fac80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2aeae00298ba42f server_cookie=41a21f9aef0a7c14 in_seq=0 out_seq=0 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0bed7fa640 10 client.16686 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0bed7fa640 1 -- 172.21.15.5:0/1268920942 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2008eac10 con 0x55b2009f8890 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0bed7fa640 10 client.16686 handle_command_reply: tid=0 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 client.16686 shutdown 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 msgr2=0x55b2009862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 0x55b2009862c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc003d50 comp rx=0 tx=0).stop 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 shutdown_connections 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b2009f8890 0x55b2009fac80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b200981a00 0x55b2009862c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b200978e80 0x55b200985d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 --2- 172.21.15.5:0/1268920942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2009784d0 0x55b200985840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0bed7fa640 0 client.16686 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 >> 172.21.15.5:0/1268920942 conn(0x55b200972ba0 msgr2=0x55b2009747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 shutdown_connections 2024-04-22T13:29:26.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 1 -- 172.21.15.5:0/1268920942 wait complete. 2024-04-22T13:29:26.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.221+0000 7f0c0ec82740 20 client.16686 trim_cache size 0 max 16384 2024-04-22T13:29:26.225 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0bf00a3d20 msgr2=0x7f0bf00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc04b000 tx=0x7f0bfc044000 comp rx=0 tx=0).stop 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 msgr2=0x7f0c04198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c04198c30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400e9f0 tx=0x7f0bf400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 shutdown_connections 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c04106160 0x7f0c04198c30 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c0419cfb0 0x7f0c0419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 --2- 172.21.15.5:0/4065526313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c04101090 0x7f0c041986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 >> 172.21.15.5:0/4065526313 conn(0x7f0c04078000 msgr2=0x7f0c040695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 shutdown_connections 2024-04-22T13:29:26.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.222+0000 7f0c0c08c640 1 -- 172.21.15.5:0/4065526313 wait complete. 2024-04-22T13:29:26.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:29:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 -- 172.21.15.5:0/2815643733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210101090 msgr2=0x7f3210101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 --2- 172.21.15.5:0/2815643733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210101090 0x7f3210101470 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3204009900 tx=0x7f320402bc50 comp rx=0 tx=0).stop 2024-04-22T13:29:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 -- 172.21.15.5:0/2815643733 shutdown_connections 2024-04-22T13:29:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 --2- 172.21.15.5:0/2815643733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3210106160 0x7f321010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 --2- 172.21.15.5:0/2815643733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210101a40 0x7f3210105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 --2- 172.21.15.5:0/2815643733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210101090 0x7f3210101470 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 -- 172.21.15.5:0/2815643733 >> 172.21.15.5:0/2815643733 conn(0x7f3210078000 msgr2=0x7f3210078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 -- 172.21.15.5:0/2815643733 shutdown_connections 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.337+0000 7f3216ae9640 1 -- 172.21.15.5:0/2815643733 wait complete. 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 Processor -- start 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 -- start start 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3210101090 0x7f3210199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210106160 0x7f3210199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 0x7f32101a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32101110a0 con 0x7f3210199fd0 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3210110f20 con 0x7f3210106160 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3216ae9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3210111220 con 0x7f3210101090 2024-04-22T13:29:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 0x7f32101a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 0x7f32101a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39308/0 (socket says 172.21.15.5:39308) 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 -- 172.21.15.5:0/3229467280 learned_addr learned my addr 172.21.15.5:0/3229467280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3214885640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3210101090 0x7f3210199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f320ffff640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210106160 0x7f3210199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 -- 172.21.15.5:0/3229467280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3210101090 msgr2=0x7f3210199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3210101090 0x7f3210199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 -- 172.21.15.5:0/3229467280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210106160 msgr2=0x7f3210199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210106160 0x7f3210199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f3215086640 1 -- 172.21.15.5:0/3229467280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32101a4d80 con 0x7f3210199fd0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.338+0000 7f320ffff640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210106160 0x7f3210199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.339+0000 7f3215086640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 0x7f32101a4580 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f320800cab0 tx=0x7f320800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.339+0000 7f320dffb640 1 -- 172.21.15.5:0/3229467280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800ae20 con 0x7f3210199fd0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.339+0000 7f320dffb640 1 -- 172.21.15.5:0/3229467280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f320800f900 con 0x7f3210199fd0 2024-04-22T13:29:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.339+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32101a5070 con 0x7f3210199fd0 2024-04-22T13:29:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.339+0000 7f320dffb640 1 -- 172.21.15.5:0/3229467280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080189f0 con 0x7f3210199fd0 2024-04-22T13:29:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.339+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101a55c0 con 0x7f3210199fd0 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f320dffb640 1 -- 172.21.15.5:0/3229467280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f320800fab0 con 0x7f3210199fd0 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 Processor -- start 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 -- start start 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686eb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f320dffb640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f32000a3da0 0x7f32000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686f4e40 0x5602686f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602686501f0 con 0x5602686eb330 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5602685ed4b0 con 0x5602686f4e40 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.341+0000 7f32197b0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56026864c1b0 con 0x5602686ebce0 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686eb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39322/0 (socket says 172.21.15.5:39322) 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 -- 172.21.15.5:0/1840770035 learned_addr learned my addr 172.21.15.5:0/1840770035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f3214885640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f32000a3da0 0x7f32000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320dffb640 1 -- 172.21.15.5:0/3229467280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f32080d0f90 con 0x7f3210199fd0 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f3215086640 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686f4e40 0x5602686f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 -- 172.21.15.5:0/1840770035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 msgr2=0x5602686f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 -- 172.21.15.5:0/1840770035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686f4e40 msgr2=0x5602686f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686f4e40 0x5602686f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 -- 172.21.15.5:0/1840770035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602685df1b0 con 0x5602686eb330 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f3215086640 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686f4e40 0x5602686f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f320ffff640 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686eb710 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005e00 tx=0x7f31fc009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ccca1e0ca116eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f3214885640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f32000a3da0 0x7f32000a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3204000c00 tx=0x7f3204004040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f31faffd640 1 -- 172.21.15.5:0/1840770035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0150f0 con 0x5602686eb330 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f31faffd640 1 -- 172.21.15.5:0/1840770035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc0035f0 con 0x5602686eb330 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.342+0000 7f31faffd640 1 -- 172.21.15.5:0/1840770035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc013700 con 0x5602686eb330 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 -- 172.21.15.5:0/1840770035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 msgr2=0x5602686eb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686eb710 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005e00 tx=0x7f31fc009670 comp rx=0 tx=0).stop 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 -- 172.21.15.5:0/1840770035 shutdown_connections 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686f4e40 0x5602686f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 --2- 172.21.15.5:0/1840770035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686eb710 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 -- 172.21.15.5:0/1840770035 >> 172.21.15.5:0/1840770035 conn(0x5602686e5ba0 msgr2=0x5602686e5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 -- 172.21.15.5:0/1840770035 shutdown_connections 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 -- 172.21.15.5:0/1840770035 wait complete. 2024-04-22T13:29:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 -- start start 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.343+0000 7f32197b0740 1 Processor -- start 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602686fbd60 con 0x5602686eb330 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560268609500 con 0x5602686ebce0 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5602686fbee0 con 0x5602686f4e40 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f32197b0740 10 client.0 _ll_register_callbacks cb 0x5602685d7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686fb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39334/0 (socket says 172.21.15.5:39334) 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 -- 172.21.15.5:0/1776150171 learned_addr learned my addr 172.21.15.5:0/1776150171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f320ffff640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3215086640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 -- 172.21.15.5:0/1776150171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 msgr2=0x5602686f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 -- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 msgr2=0x5602686f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.344+0000 7f3214885640 1 -- 172.21.15.5:0/1776150171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602683623f0 con 0x5602686eb330 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f3214885640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686fb210 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3204035000 tx=0x7f3204009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe5fb71c8296ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f31effff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f31effff640 1 -- 172.21.15.5:0/1776150171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320404e020 con 0x5602686eb330 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f31effff640 1 -- 172.21.15.5:0/1776150171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f32040048c0 con 0x5602686eb330 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31fc0035f0 con 0x5602686eb330 2024-04-22T13:29:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f31effff640 1 -- 172.21.15.5:0/1776150171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3204059050 con 0x5602686eb330 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 10 client.16698 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56026842c7d0 con 0x5602686eb330 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f31effff640 1 -- 172.21.15.5:0/1776150171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc0035f0 con 0x5602686eb330 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f31effff640 1 -- 172.21.15.5:0/1776150171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3204003b90 con 0x5602686eb330 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 10 client.16698 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 20 client.16698 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560268708f50 0x56026870b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 4 client.16698 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.345+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602685ef930 con 0x560268708f50 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.346+0000 7f320ffff640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560268708f50 0x56026870b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.346+0000 7f320ffff640 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560268708f50 0x56026870b3d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa9bd85230e98d2 server_cookie=86c80efe5582a537 in_seq=0 out_seq=0 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.346+0000 7f31effff640 10 client.16698 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f31effff640 1 -- 172.21.15.5:0/1776150171 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602685ef930 con 0x560268708f50 2024-04-22T13:29:26.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f31effff640 10 client.16698 handle_command_reply: tid=0 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 client.16698 shutdown 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 msgr2=0x5602686fb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686fb210 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3204035000 tx=0x7f3204009260 comp rx=0 tx=0).stop 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 shutdown_connections 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560268708f50 0x56026870b3d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 --2- 172.21.15.5:0/1776150171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x5602686fb210 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f31effff640 0 client.16698 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 >> 172.21.15.5:0/1776150171 conn(0x5602686e5ba0 msgr2=0x5602686e78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 shutdown_connections 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 1 -- 172.21.15.5:0/1776150171 wait complete. 2024-04-22T13:29:26.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.347+0000 7f32197b0740 20 client.16698 trim_cache size 0 max 16384 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.360+0000 7f32197b0740 1 Processor -- start 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.360+0000 7f32197b0740 1 -- start start 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.360+0000 7f32197b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686eb330 0x560268704c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f32197b0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f32197b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 0x5602686e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f32197b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32101ab850 con 0x5602686f4e40 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f32197b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32101ad390 con 0x5602686eb330 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f32197b0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32101abf20 con 0x5602686ebce0 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 0x5602686e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 0x5602686e4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39338/0 (socket says 172.21.15.5:39338) 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 -- 172.21.15.5:0/1913705429 learned_addr learned my addr 172.21.15.5:0/1913705429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3214885640 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686eb330 0x560268704c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f320ffff640 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 -- 172.21.15.5:0/1913705429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 msgr2=0x5602686f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 -- 172.21.15.5:0/1913705429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686eb330 msgr2=0x560268704c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686eb330 0x560268704c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 -- 172.21.15.5:0/1913705429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602686ea560 con 0x5602686f4e40 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f3215086640 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 0x5602686e4a80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3208099360 tx=0x7f3208018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901e4d49aa51531e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f31fa7fc640 1 -- 172.21.15.5:0/1913705429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320809cf00 con 0x5602686f4e40 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f31fa7fc640 1 -- 172.21.15.5:0/1913705429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32080a12b0 con 0x5602686f4e40 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.361+0000 7f31fa7fc640 1 -- 172.21.15.5:0/1913705429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080ce930 con 0x5602686f4e40 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 -- 172.21.15.5:0/1913705429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 msgr2=0x5602686e4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 0x5602686e4a80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3208099360 tx=0x7f3208018400 comp rx=0 tx=0).stop 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 -- 172.21.15.5:0/1913705429 shutdown_connections 2024-04-22T13:29:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686f4e40 0x5602686e4a80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686ebce0 0x5602686f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 --2- 172.21.15.5:0/1913705429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686eb330 0x560268704c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 -- 172.21.15.5:0/1913705429 >> 172.21.15.5:0/1913705429 conn(0x5602686e5ba0 msgr2=0x5602686e4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 -- 172.21.15.5:0/1913705429 shutdown_connections 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 -- 172.21.15.5:0/1913705429 wait complete. 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 -- start start 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.362+0000 7f32197b0740 1 Processor -- start 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x560268701360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32101accc0 con 0x5602686eb330 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32101ac5f0 con 0x5602686ebce0 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32101a9870 con 0x5602686f4e40 2024-04-22T13:29:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 10 client.0 _ll_register_callbacks cb 0x7f32040045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x560268701360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x560268701360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39344/0 (socket says 172.21.15.5:39344) 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 -- 172.21.15.5:0/4196764467 learned_addr learned my addr 172.21.15.5:0/4196764467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f320ffff640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 -- 172.21.15.5:0/4196764467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 msgr2=0x5602686fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3215086640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686fa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 -- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 msgr2=0x5602686f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 -- 172.21.15.5:0/4196764467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602686fa690 con 0x5602686eb330 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f320ffff640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f3214885640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x560268701360 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f32040409a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1f3c1608c909fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f31effff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:26.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f31effff640 1 -- 172.21.15.5:0/4196764467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320404e040 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f31effff640 1 -- 172.21.15.5:0/4196764467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f32040035f0 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.363+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32080a12b0 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f31effff640 1 -- 172.21.15.5:0/4196764467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f320406d050 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 10 client.16704 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56026876f2e0 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f31effff640 1 -- 172.21.15.5:0/4196764467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32080a12b0 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f31effff640 1 -- 172.21.15.5:0/4196764467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32040046c0 con 0x5602686eb330 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 10 client.16704 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 20 client.16704 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56026876fab0 0x560268771ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 4 client.16704 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56026865dc10 con 0x56026876fab0 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.364+0000 7f320ffff640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56026876fab0 0x560268771ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f320ffff640 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56026876fab0 0x560268771ed0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e41524f6c37588 server_cookie=d993ad87767fef56 in_seq=0 out_seq=0 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f31effff640 10 client.16704 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f31effff640 1 -- 172.21.15.5:0/4196764467 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56026865dc10 con 0x56026876fab0 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f31effff640 10 client.16704 handle_command_reply: tid=0 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 client.16704 shutdown 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 msgr2=0x560268701360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x560268701360 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f32040409a0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 shutdown_connections 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56026876fab0 0x560268771ed0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602686f4e40 0x5602686fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5602686ebce0 0x5602686f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f31effff640 0 client.16704 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 --2- 172.21.15.5:0/4196764467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602686eb330 0x560268701360 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 >> 172.21.15.5:0/4196764467 conn(0x5602686e5ba0 msgr2=0x5602686e4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 shutdown_connections 2024-04-22T13:29:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 1 -- 172.21.15.5:0/4196764467 wait complete. 2024-04-22T13:29:26.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.365+0000 7f32197b0740 20 client.16704 trim_cache size 0 max 16384 2024-04-22T13:29:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f32000a3da0 msgr2=0x7f32000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f32000a3da0 0x7f32000a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3204000c00 tx=0x7f3204004040 comp rx=0 tx=0).stop 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 msgr2=0x7f32101a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 0x7f32101a4580 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f320800cab0 tx=0x7f320800cf70 comp rx=0 tx=0).stop 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 shutdown_connections 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3210199fd0 0x7f32101a4580 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f32000a3da0 0x7f32000a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3210106160 0x7f3210199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 --2- 172.21.15.5:0/3229467280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3210101090 0x7f3210199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 >> 172.21.15.5:0/3229467280 conn(0x7f3210078000 msgr2=0x7f32100ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 shutdown_connections 2024-04-22T13:29:26.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:26.366+0000 7f3216ae9640 1 -- 172.21.15.5:0/3229467280 wait complete. 2024-04-22T13:29:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:26 smithi049 ceph-mon[31379]: pgmap v72: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-22T13:29:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:26 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/556254239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:27.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:26 smithi107 ceph-mon[31859]: pgmap v72: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-22T13:29:27.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:26 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/556254239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:27.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:26 smithi005 ceph-mon[28006]: pgmap v72: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-22T13:29:27.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:26 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/556254239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:27.374 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:29:27.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:29:27.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:29:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 -- 172.21.15.5:0/339660839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc103b70 msgr2=0x7f8bdc10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 --2- 172.21.15.5:0/339660839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc103b70 0x7f8bdc10bfc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009900 tx=0x7f8bc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 -- 172.21.15.5:0/339660839 shutdown_connections 2024-04-22T13:29:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 --2- 172.21.15.5:0/339660839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc103b70 0x7f8bdc10bfc0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 --2- 172.21.15.5:0/339660839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bdc10c6d0 0x7f8bdc10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 --2- 172.21.15.5:0/339660839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bdc1031c0 0x7f8bdc1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.498+0000 7f8be1893640 1 -- 172.21.15.5:0/339660839 >> 172.21.15.5:0/339660839 conn(0x7f8bdc077f70 msgr2=0x7f8bdc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 -- 172.21.15.5:0/339660839 shutdown_connections 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 -- 172.21.15.5:0/339660839 wait complete. 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 Processor -- start 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 -- start start 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bdc1031c0 0x7f8bdc100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bdc10c6d0 0x7f8bdc101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 0x7f8bdc1a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc112330 con 0x7f8bdc101890 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1121b0 con 0x7f8bdc1031c0 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8be1893640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1124b0 con 0x7f8bdc10c6d0 2024-04-22T13:29:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.499+0000 7f8bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 0x7f8bdc1a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 0x7f8bdc1a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39366/0 (socket says 172.21.15.5:39366) 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 -- 172.21.15.5:0/1861355335 learned_addr learned my addr 172.21.15.5:0/1861355335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bdc10c6d0 0x7f8bdc101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bdc1031c0 0x7f8bdc100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 -- 172.21.15.5:0/1861355335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bdc10c6d0 msgr2=0x7f8bdc101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bdc10c6d0 0x7f8bdc101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 -- 172.21.15.5:0/1861355335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bdc1031c0 msgr2=0x7f8bdc100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bdc1031c0 0x7f8bdc100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 -- 172.21.15.5:0/1861355335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc1a7140 con 0x7f8bdc101890 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 0x7f8bdc1a6a40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400eaa0 tx=0x7f8bd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8be0891640 1 -- 172.21.15.5:0/1861355335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400cdb0 con 0x7f8bdc101890 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8be0891640 1 -- 172.21.15.5:0/1861355335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd4005020 con 0x7f8bdc101890 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc1a7430 con 0x7f8bdc101890 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8be0891640 1 -- 172.21.15.5:0/1861355335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40116d0 con 0x7f8bdc101890 2024-04-22T13:29:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.500+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc1a7950 con 0x7f8bdc101890 2024-04-22T13:29:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.502+0000 7f8be0891640 1 -- 172.21.15.5:0/1861355335 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8bd40051d0 con 0x7f8bdc101890 2024-04-22T13:29:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 Processor -- start 2024-04-22T13:29:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 -- start start 2024-04-22T13:29:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8301cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a830a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a830d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4a8265f00 con 0x55d4a8302290 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d4a8203590 con 0x55d4a830ae10 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be4566740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d4a8261ec0 con 0x55d4a83018e0 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be0891640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bc80a3dd0 0x7f8bc80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a830a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a830a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39370/0 (socket says 172.21.15.5:39370) 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 -- 172.21.15.5:0/202319825 learned_addr learned my addr 172.21.15.5:0/202319825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8301cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bc80a3dd0 0x7f8bc80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a830d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8be0891640 1 -- 172.21.15.5:0/1861355335 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bd40d09a0 con 0x7f8bdc101890 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 -- 172.21.15.5:0/202319825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 msgr2=0x55d4a8301cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8301cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 -- 172.21.15.5:0/202319825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 msgr2=0x55d4a830d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a830d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.503+0000 7f8bdaffd640 1 -- 172.21.15.5:0/202319825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4a80407e0 con 0x55d4a8302290 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.504+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a830a840 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00cc60 tx=0x7f8bcc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b9fbc7539bf13c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.504+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bc80a3dd0 0x7f8bc80a6290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40992e0 tx=0x7f8bd4005890 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.504+0000 7f8bd1ffb640 1 -- 172.21.15.5:0/202319825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc014070 con 0x55d4a8302290 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.504+0000 7f8bd1ffb640 1 -- 172.21.15.5:0/202319825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc0047a0 con 0x55d4a8302290 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.504+0000 7f8bd1ffb640 1 -- 172.21.15.5:0/202319825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc010690 con 0x55d4a8302290 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 -- 172.21.15.5:0/202319825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 msgr2=0x55d4a830a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a830a840 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00cc60 tx=0x7f8bcc00e720 comp rx=0 tx=0).stop 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 -- 172.21.15.5:0/202319825 shutdown_connections 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a830a840 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a830d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 --2- 172.21.15.5:0/202319825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8301cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 -- 172.21.15.5:0/202319825 >> 172.21.15.5:0/202319825 conn(0x55d4a82fbfb0 msgr2=0x55d4a82fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 -- 172.21.15.5:0/202319825 shutdown_connections 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 -- 172.21.15.5:0/202319825 wait complete. 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 -- start start 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 Processor -- start 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8307e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.505+0000 7f8be4566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a83083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8be4566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a83088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8be4566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4a830dd80 con 0x55d4a8302290 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8be4566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55d4a821f580 con 0x55d4a830ae10 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8be4566740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d4a830df00 con 0x55d4a83018e0 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8be4566740 10 client.0 _ll_register_callbacks cb 0x55d4a8225650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a83083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a83083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39376/0 (socket says 172.21.15.5:39376) 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1621607519 learned_addr learned my addr 172.21.15.5:0/1621607519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a83088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1621607519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 msgr2=0x55d4a8307e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8307e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 msgr2=0x55d4a83088e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a83088e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1621607519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4a80a0f70 con 0x55d4a8302290 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a83083a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc017000 tx=0x7f8bcc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64eafbdb6b0733d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bbaffd640 1 -- 172.21.15.5:0/1621607519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc027060 con 0x55d4a8302290 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bbaffd640 1 -- 172.21.15.5:0/1621607519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bcc005410 con 0x55d4a8302290 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc0047a0 con 0x55d4a8302290 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.506+0000 7f8bbaffd640 1 -- 172.21.15.5:0/1621607519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc014070 con 0x55d4a8302290 2024-04-22T13:29:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 10 client.16716 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4a7f38170 con 0x55d4a8302290 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8bbaffd640 1 -- 172.21.15.5:0/1621607519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc0047a0 con 0x55d4a8302290 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8bbaffd640 1 -- 172.21.15.5:0/1621607519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bcc0104a0 con 0x55d4a8302290 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 10 client.16716 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 20 client.16716 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a831b310 0x55d4a831d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 4 client.16716 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4a8273920 con 0x55d4a831b310 2024-04-22T13:29:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.507+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a831b310 0x55d4a831d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.508+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a831b310 0x55d4a831d700 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b32aa4889ee84 server_cookie=a9efb33ec808594 in_seq=0 out_seq=0 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.508+0000 7f8bbaffd640 10 client.16716 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.508+0000 7f8bbaffd640 1 -- 172.21.15.5:0/1621607519 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4a8273920 con 0x55d4a831b310 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.508+0000 7f8bbaffd640 10 client.16716 handle_command_reply: tid=0 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 client.16716 shutdown 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 msgr2=0x55d4a83083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a83083a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc017000 tx=0x7f8bcc00c9a0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 shutdown_connections 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a831b310 0x55d4a831d700 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a83083a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a830ae10 0x55d4a83088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8bbaffd640 0 client.16716 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 --2- 172.21.15.5:0/1621607519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a8307e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 >> 172.21.15.5:0/1621607519 conn(0x55d4a82fbfb0 msgr2=0x55d4a82fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 shutdown_connections 2024-04-22T13:29:27.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 1 -- 172.21.15.5:0/1621607519 wait complete. 2024-04-22T13:29:27.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.509+0000 7f8be4566740 20 client.16716 trim_cache size 0 max 16384 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 Processor -- start 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 -- start start 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a83018e0 0x55d4a830a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a8316c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a830ae10 0x55d4a82fae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1afd60 con 0x55d4a8302290 2024-04-22T13:29:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b18a0 con 0x55d4a83018e0 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8be4566740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b0430 con 0x55d4a830ae10 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a8316c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a8316c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39382/0 (socket says 172.21.15.5:39382) 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1300710209 learned_addr learned my addr 172.21.15.5:0/1300710209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1300710209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a830ae10 msgr2=0x55d4a82fae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a830ae10 0x55d4a82fae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1300710209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a83018e0 msgr2=0x55d4a830a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a83018e0 0x55d4a830a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 -- 172.21.15.5:0/1300710209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4a83063b0 con 0x55d4a8302290 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.527+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a8316c40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00efd0 tx=0x7f8bcc00c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332b961218654efd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8bd17fa640 1 -- 172.21.15.5:0/1300710209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc027040 con 0x55d4a8302290 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8bd17fa640 1 -- 172.21.15.5:0/1300710209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc004d40 con 0x55d4a8302290 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8bd17fa640 1 -- 172.21.15.5:0/1300710209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc02c480 con 0x55d4a8302290 2024-04-22T13:29:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 -- 172.21.15.5:0/1300710209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 msgr2=0x55d4a8316c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a8316c40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00efd0 tx=0x7f8bcc00c4d0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 -- 172.21.15.5:0/1300710209 shutdown_connections 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a8302290 0x55d4a8316c40 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a830ae10 0x55d4a82fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 --2- 172.21.15.5:0/1300710209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a83018e0 0x55d4a830a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 -- 172.21.15.5:0/1300710209 >> 172.21.15.5:0/1300710209 conn(0x55d4a82fbfb0 msgr2=0x55d4a82fa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 -- 172.21.15.5:0/1300710209 shutdown_connections 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.528+0000 7f8be4566740 1 -- 172.21.15.5:0/1300710209 wait complete. 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 -- start start 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 Processor -- start 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a83135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a8302290 0x55d4a830e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 0x55d4a830ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b11d0 con 0x55d4a830ae10 2024-04-22T13:29:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b0b00 con 0x55d4a8302290 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1add60 con 0x55d4a83018e0 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8be4566740 10 client.0 _ll_register_callbacks cb 0x7f8bc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 0x55d4a830ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 0x55d4a830ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39392/0 (socket says 172.21.15.5:39392) 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 -- 172.21.15.5:0/3765325051 learned_addr learned my addr 172.21.15.5:0/3765325051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a83135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a8302290 0x55d4a830e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 -- 172.21.15.5:0/3765325051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 msgr2=0x55d4a83135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a83135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 -- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a8302290 msgr2=0x55d4a830e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a8302290 0x55d4a830e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.529+0000 7f8bd3fff640 1 -- 172.21.15.5:0/3765325051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4a830f6c0 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bd3fff640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 0x55d4a830ee90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc06a2d0 tx=0x7f8bc402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed3d986918fc6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bbaffd640 1 -- 172.21.15.5:0/3765325051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc402ca80 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc004d40 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bbaffd640 1 -- 172.21.15.5:0/3765325051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bc402cc20 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bbaffd640 1 -- 172.21.15.5:0/3765325051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bc4005030 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 10 client.16722 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4a83816d0 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bbaffd640 1 -- 172.21.15.5:0/3765325051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc004d40 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8bbaffd640 1 -- 172.21.15.5:0/3765325051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bc4050030 con 0x55d4a830ae10 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 10 client.16722 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 20 client.16722 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a8381d90 0x55d4a83841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 4 client.16722 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:29:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.530+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d4a826d790 con 0x55d4a8381d90 2024-04-22T13:29:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.531+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a8381d90 0x55d4a83841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.531+0000 7f8bdaffd640 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a8381d90 0x55d4a83841b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54fd265e1c01cb8 server_cookie=1c182f89ff51ba5c in_seq=0 out_seq=0 2024-04-22T13:29:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.531+0000 7f8bbaffd640 10 client.16722 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8bbaffd640 1 -- 172.21.15.5:0/3765325051 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d4a826d790 con 0x55d4a8381d90 2024-04-22T13:29:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8bbaffd640 10 client.16722 handle_command_reply: tid=0 2024-04-22T13:29:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 client.16722 shutdown 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 msgr2=0x55d4a830ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 0x55d4a830ee90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc06a2d0 tx=0x7f8bc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 shutdown_connections 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55d4a8381d90 0x55d4a83841b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4a830ae10 0x55d4a830ee90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4a8302290 0x55d4a830e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 --2- 172.21.15.5:0/3765325051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4a83018e0 0x55d4a83135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 >> 172.21.15.5:0/3765325051 conn(0x55d4a82fbfb0 msgr2=0x55d4a82fb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.532+0000 7f8bbaffd640 0 client.16722 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 shutdown_connections 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be4566740 1 -- 172.21.15.5:0/3765325051 wait complete. 2024-04-22T13:29:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be4566740 20 client.16722 trim_cache size 0 max 16384 2024-04-22T13:29:27.536 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:27.536 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:29:27.536 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "cd177138-a340-4e6f-981f-9320b867b42b", 2024-04-22T13:29:27.536 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:29:27.536 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bc80a3dd0 msgr2=0x7f8bc80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bc80a3dd0 0x7f8bc80a6290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40992e0 tx=0x7f8bd4005890 comp rx=0 tx=0).stop 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 msgr2=0x7f8bdc1a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 0x7f8bdc1a6a40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400eaa0 tx=0x7f8bd400ef60 comp rx=0 tx=0).stop 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 shutdown_connections 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc101890 0x7f8bdc1a6a40 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bc80a3dd0 0x7f8bc80a6290 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bdc10c6d0 0x7f8bdc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 --2- 172.21.15.5:0/1861355335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bdc1031c0 0x7f8bdc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 >> 172.21.15.5:0/1861355335 conn(0x7f8bdc077f70 msgr2=0x7f8bdc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 shutdown_connections 2024-04-22T13:29:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.533+0000 7f8be1893640 1 -- 172.21.15.5:0/1861355335 wait complete. 2024-04-22T13:29:27.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:27.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 -- 172.21.15.5:0/540981543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0102b20 msgr2=0x7fedf0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/540981543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0102b20 0x7fedf0106b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feddc009900 tx=0x7feddc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 -- 172.21.15.5:0/540981543 shutdown_connections 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/540981543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0102b20 0x7fedf0106b60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/540981543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf0107240 0x7fedf010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/540981543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0102170 0x7fedf0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 -- 172.21.15.5:0/540981543 >> 172.21.15.5:0/540981543 conn(0x7fedf00fd9b0 msgr2=0x7fedf00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 -- 172.21.15.5:0/540981543 shutdown_connections 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.654+0000 7fedf6c9a640 1 -- 172.21.15.5:0/540981543 wait complete. 2024-04-22T13:29:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 Processor -- start 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 -- start start 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf0102170 0x7fedf0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0107240 0x7fedf0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 0x7fedf01a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01122b0 con 0x7fedf0074cd0 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0112130 con 0x7fedf0102170 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf6c9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0112430 con 0x7fedf0107240 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 0x7fedf01a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 0x7fedf01a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39402/0 (socket says 172.21.15.5:39402) 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 -- 172.21.15.5:0/2887240025 learned_addr learned my addr 172.21.15.5:0/2887240025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedeffff640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0107240 0x7fedf0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf0102170 0x7fedf0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 -- 172.21.15.5:0/2887240025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0107240 msgr2=0x7fedf0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0107240 0x7fedf0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 -- 172.21.15.5:0/2887240025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf0102170 msgr2=0x7fedf0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf0102170 0x7fedf0074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 -- 172.21.15.5:0/2887240025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf01a71b0 con 0x7fedf0074cd0 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedeffff640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0107240 0x7fedf0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fedf5237640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 0x7fedf01a6ab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fede800b970 tx=0x7fede800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.655+0000 7fededffb640 1 -- 172.21.15.5:0/2887240025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8019070 con 0x7fedf0074cd0 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.656+0000 7fededffb640 1 -- 172.21.15.5:0/2887240025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede8014440 con 0x7fedf0074cd0 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.656+0000 7fededffb640 1 -- 172.21.15.5:0/2887240025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80075d0 con 0x7fedf0074cd0 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.656+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf01a74a0 con 0x7fedf0074cd0 2024-04-22T13:29:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.656+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01a79c0 con 0x7fedf0074cd0 2024-04-22T13:29:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.658+0000 7fededffb640 1 -- 172.21.15.5:0/2887240025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fede801c070 con 0x7fedf0074cd0 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.658+0000 7fedf996e740 1 Processor -- start 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.658+0000 7fedf996e740 1 -- start start 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fededffb640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fede00a3dd0 0x7fede00a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf996e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2130960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf996e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2139540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf996e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c213be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf996e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1c20951e0 con 0x55a1c2139a80 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf996e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2032390 con 0x55a1c2130ea0 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf996e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a1c20911a0 con 0x55a1c2130580 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedeffff640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fede00a3dd0 0x7fede00a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c213be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c213be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39414/0 (socket says 172.21.15.5:39414) 2024-04-22T13:29:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 -- 172.21.15.5:0/2588369170 learned_addr learned my addr 172.21.15.5:0/2588369170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fededffb640 1 -- 172.21.15.5:0/2887240025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fede80cfc30 con 0x7fedf0074cd0 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2130960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedeffff640 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2139540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 -- 172.21.15.5:0/2588369170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 msgr2=0x55a1c2130960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2130960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 -- 172.21.15.5:0/2588369170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 msgr2=0x55a1c2139540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2139540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 -- 172.21.15.5:0/2588369170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c1e717d0 con 0x55a1c2139a80 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2130960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedf5237640 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c213be70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fede80a4450 tx=0x7fede80979b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cbc5e5472d15e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7fedeffff640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fede00a3dd0 0x7fede00a6290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feddc02c240 tx=0x7feddc004590 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.659+0000 7feddaffd640 1 -- 172.21.15.5:0/2588369170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8023040 con 0x55a1c2139a80 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7feddaffd640 1 -- 172.21.15.5:0/2588369170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede809c410 con 0x55a1c2139a80 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7feddaffd640 1 -- 172.21.15.5:0/2588369170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede809a0a0 con 0x55a1c2139a80 2024-04-22T13:29:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 -- 172.21.15.5:0/2588369170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 msgr2=0x55a1c213be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c213be70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fede80a4450 tx=0x7fede80979b0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 -- 172.21.15.5:0/2588369170 shutdown_connections 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c213be70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2139540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 --2- 172.21.15.5:0/2588369170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2130960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 -- 172.21.15.5:0/2588369170 >> 172.21.15.5:0/2588369170 conn(0x55a1c212acb0 msgr2=0x55a1c212b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 -- 172.21.15.5:0/2588369170 shutdown_connections 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.660+0000 7fedf996e740 1 -- 172.21.15.5:0/2588369170 wait complete. 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 -- start start 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 Processor -- start 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2137010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c2137550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1c213c870 con 0x55a1c2139a80 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a1c204e2f0 con 0x55a1c2130ea0 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a1c213c9f0 con 0x55a1c2130580 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf996e740 10 client.0 _ll_register_callbacks cb 0x55a1c205b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c2137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c2137550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39430/0 (socket says 172.21.15.5:39430) 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 -- 172.21.15.5:0/217942778 learned_addr learned my addr 172.21.15.5:0/217942778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf4a36640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedeffff640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2137010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 -- 172.21.15.5:0/217942778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 msgr2=0x55a1c2136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 -- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 msgr2=0x55a1c2137010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2137010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedf5237640 1 -- 172.21.15.5:0/217942778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c1da73f0 con 0x55a1c2139a80 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.661+0000 7fedeffff640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2137010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf5237640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c2137550 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fede809ba10 tx=0x7fede80a00c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9afa2cc0d174538 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedcffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedcffff640 1 -- 172.21.15.5:0/217942778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80b4070 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedcffff640 1 -- 172.21.15.5:0/217942778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fede8098380 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede809c410 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedcffff640 1 -- 172.21.15.5:0/217942778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fede80b5270 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 10 client.16734 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1c1d670f0 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedcffff640 1 -- 172.21.15.5:0/217942778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede809c410 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedcffff640 1 -- 172.21.15.5:0/217942778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fede808e160 con 0x55a1c2139a80 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 10 client.16734 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 20 client.16734 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c2149920 0x55a1c214bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 4 client.16734 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.662+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1c20a2c00 con 0x55a1c2149920 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.663+0000 7fedf4a36640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c2149920 0x55a1c214bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.663+0000 7fedf4a36640 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c2149920 0x55a1c214bd10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b10b07b5d3610f server_cookie=51ad4e8fcb4dc001 in_seq=0 out_seq=0 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.663+0000 7fedcffff640 10 client.16734 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedcffff640 1 -- 172.21.15.5:0/217942778 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1c20a2c00 con 0x55a1c2149920 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedcffff640 10 client.16734 handle_command_reply: tid=0 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 client.16734 shutdown 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 msgr2=0x55a1c2137550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c2137550 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fede809ba10 tx=0x7fede80a00c0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 shutdown_connections 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c2149920 0x55a1c214bd10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2139a80 0x55a1c2137550 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2130ea0 0x55a1c2137010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedcffff640 0 client.16734 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 --2- 172.21.15.5:0/217942778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130580 0x55a1c2136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 >> 172.21.15.5:0/217942778 conn(0x55a1c212acb0 msgr2=0x55a1c212ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 shutdown_connections 2024-04-22T13:29:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 1 -- 172.21.15.5:0/217942778 wait complete. 2024-04-22T13:29:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.664+0000 7fedf996e740 20 client.16734 trim_cache size 0 max 16384 2024-04-22T13:29:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.681+0000 7fedf996e740 1 Processor -- start 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.681+0000 7fedf996e740 1 -- start start 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf996e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c2139200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf996e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c2137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf996e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c2129bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf996e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01afe60 con 0x55a1c2130580 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf996e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01b19a0 con 0x55a1c2139a80 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf996e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01b0530 con 0x55a1c2130ea0 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c2139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c2139200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39444/0 (socket says 172.21.15.5:39444) 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 -- 172.21.15.5:0/2556198546 learned_addr learned my addr 172.21.15.5:0/2556198546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 -- 172.21.15.5:0/2556198546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 msgr2=0x55a1c2137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c2137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 -- 172.21.15.5:0/2556198546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 msgr2=0x55a1c2129bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c2129bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 -- 172.21.15.5:0/2556198546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c2130cd0 con 0x55a1c2130580 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c2139200 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fede400c940 tx=0x7fede4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d2d0c05bc49832 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedda7fc640 1 -- 172.21.15.5:0/2556198546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede400c300 con 0x55a1c2130580 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedda7fc640 1 -- 172.21.15.5:0/2556198546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede401f030 con 0x55a1c2130580 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.682+0000 7fedda7fc640 1 -- 172.21.15.5:0/2556198546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4014960 con 0x55a1c2130580 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 -- 172.21.15.5:0/2556198546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 msgr2=0x55a1c2139200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c2139200 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fede400c940 tx=0x7fede4008040 comp rx=0 tx=0).stop 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 -- 172.21.15.5:0/2556198546 shutdown_connections 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c2129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c2137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 --2- 172.21.15.5:0/2556198546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c2139200 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 -- 172.21.15.5:0/2556198546 >> 172.21.15.5:0/2556198546 conn(0x55a1c212acb0 msgr2=0x55a1c21294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 -- 172.21.15.5:0/2556198546 shutdown_connections 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 -- 172.21.15.5:0/2556198546 wait complete. 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 -- start start 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.683+0000 7fedf996e740 1 Processor -- start 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c214ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c214c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c214c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01b12d0 con 0x55a1c2130580 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01b0c00 con 0x55a1c2139a80 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01ade40 con 0x55a1c2130ea0 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf996e740 10 client.0 _ll_register_callbacks cb 0x7fede809bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c214ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c214ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39446/0 (socket says 172.21.15.5:39446) 2024-04-22T13:29:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 -- 172.21.15.5:0/2934173381 learned_addr learned my addr 172.21.15.5:0/2934173381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf5237640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c214c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedeffff640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c214c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 -- 172.21.15.5:0/2934173381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 msgr2=0x55a1c214c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c214c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 -- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 msgr2=0x55a1c214c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c214c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 -- 172.21.15.5:0/2934173381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c214cdc0 con 0x55a1c2130580 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf5237640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c214c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedf4a36640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c214ee40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fede4004810 tx=0x7fede4014680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146777dbd83e902d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedcffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.684+0000 7fedcffff640 1 -- 172.21.15.5:0/2934173381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4024420 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedcffff640 1 -- 172.21.15.5:0/2934173381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fede40245c0 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede401f030 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedcffff640 1 -- 172.21.15.5:0/2934173381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fede40247c0 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 10 client.16740 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1c21af850 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedcffff640 1 -- 172.21.15.5:0/2934173381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede401f030 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedcffff640 1 -- 172.21.15.5:0/2934173381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fede401b880 con 0x55a1c2130580 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 10 client.16740 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 20 client.16740 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c21afeb0 0x55a1c21b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 4 client.16740 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1c209ca70 con 0x55a1c21afeb0 2024-04-22T13:29:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.685+0000 7fedeffff640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c21afeb0 0x55a1c21b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedeffff640 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c21afeb0 0x55a1c21b22a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=618da1e12cc89a59 server_cookie=14c6e0c0c68011c1 in_seq=0 out_seq=0 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedcffff640 10 client.16740 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedcffff640 1 -- 172.21.15.5:0/2934173381 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a1c209ca70 con 0x55a1c21afeb0 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedcffff640 10 client.16740 handle_command_reply: tid=0 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 client.16740 shutdown 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 msgr2=0x55a1c214ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c214ee40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fede4004810 tx=0x7fede4014680 comp rx=0 tx=0).stop 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 shutdown_connections 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a1c21afeb0 0x55a1c21b22a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a1c2139a80 0x55a1c214c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedcffff640 0 client.16740 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a1c2130ea0 0x55a1c214c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 --2- 172.21.15.5:0/2934173381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1c2130580 0x55a1c214ee40 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 >> 172.21.15.5:0/2934173381 conn(0x55a1c212acb0 msgr2=0x55a1c212c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 shutdown_connections 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 1 -- 172.21.15.5:0/2934173381 wait complete. 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.686+0000 7fedf996e740 20 client.16740 trim_cache size 0 max 16384 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: "cd177138-a340-4e6f-981f-9320b867b42b": { 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cd177138-a340-4e6f-981f-9320b867b42b", 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:29:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:27.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:27.691 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fede00a3dd0 msgr2=0x7fede00a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fede00a3dd0 0x7fede00a6290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feddc02c240 tx=0x7feddc004590 comp rx=0 tx=0).stop 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 msgr2=0x7fedf01a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 0x7fedf01a6ab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fede800b970 tx=0x7fede800be30 comp rx=0 tx=0).stop 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 shutdown_connections 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedf0074cd0 0x7fedf01a6ab0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fede00a3dd0 0x7fede00a6290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf0107240 0x7fedf0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 --2- 172.21.15.5:0/2887240025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fedf0102170 0x7fedf0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 >> 172.21.15.5:0/2887240025 conn(0x7fedf00fd9b0 msgr2=0x7fedf00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 shutdown_connections 2024-04-22T13:29:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:27.687+0000 7fedf6c9a640 1 -- 172.21.15.5:0/2887240025 wait complete. 2024-04-22T13:29:27.694 INFO:tasks.cephfs.filesystem:scrub status for tag:cd177138-a340-4e6f-981f-9320b867b42b - {'path': '/', 'tag': 'cd177138-a340-4e6f-981f-9320b867b42b', 'options': 'recursive,force'} 2024-04-22T13:29:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:28 smithi049 ceph-mon[31379]: pgmap v73: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 MiB/s wr, 15 op/s 2024-04-22T13:29:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:28 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:29:29.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:28 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:29.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:28 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:29:29.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:28 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:28 smithi107 ceph-mon[31859]: pgmap v73: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 MiB/s wr, 15 op/s 2024-04-22T13:29:29.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:28 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:29:29.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:28 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:29.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:28 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:29:29.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:28 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:28 smithi005 ceph-mon[28006]: pgmap v73: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 MiB/s wr, 15 op/s 2024-04-22T13:29:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:28 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:29:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:28 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:28 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:29:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:28 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:31.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:30 smithi049 ceph-mon[31379]: pgmap v74: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-22T13:29:31.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:30 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:29:31.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:30 smithi107 ceph-mon[31859]: pgmap v74: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-22T13:29:31.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:30 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:29:31.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:30 smithi005 ceph-mon[28006]: pgmap v74: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-22T13:29:31.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:30 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:29:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 -- 172.21.15.5:0/102265822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 msgr2=0x7f048c110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 --2- 172.21.15.5:0/102265822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c110ce0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0484009ca0 tx=0x7f048402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 -- 172.21.15.5:0/102265822 shutdown_connections 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 --2- 172.21.15.5:0/102265822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c110ce0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 --2- 172.21.15.5:0/102265822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048c101660 0x7f048c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 --2- 172.21.15.5:0/102265822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f048c100d40 0x7f048c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 -- 172.21.15.5:0/102265822 >> 172.21.15.5:0/102265822 conn(0x7f048c077f70 msgr2=0x7f048c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 -- 172.21.15.5:0/102265822 shutdown_connections 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.824+0000 7f049234b640 1 -- 172.21.15.5:0/102265822 wait complete. 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 Processor -- start 2024-04-22T13:29:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 -- start start 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f048c100d40 0x7f048c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048c101660 0x7f048c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1143d0 con 0x7f048c10e8f0 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f048c114250 con 0x7f048c101660 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f049234b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f048c114550 con 0x7f048c100d40 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37930/0 (socket says 172.21.15.5:37930) 2024-04-22T13:29:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 -- 172.21.15.5:0/302692691 learned_addr learned my addr 172.21.15.5:0/302692691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f048bfff640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f048c100d40 0x7f048c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f048b7fe640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048c101660 0x7f048c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 -- 172.21.15.5:0/302692691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f048c100d40 msgr2=0x7f048c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f048c100d40 0x7f048c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 -- 172.21.15.5:0/302692691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048c101660 msgr2=0x7f048c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.825+0000 7f04908e8640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048c101660 0x7f048c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f04908e8640 1 -- 172.21.15.5:0/302692691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c19aff0 con 0x7f048c10e8f0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f04908e8640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c19a730 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0484004af0 tx=0x7f0484034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f04897fa640 1 -- 172.21.15.5:0/302692691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484039040 con 0x7f048c10e8f0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f04897fa640 1 -- 172.21.15.5:0/302692691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0484044070 con 0x7f048c10e8f0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c19b280 con 0x7f048c10e8f0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f04897fa640 1 -- 172.21.15.5:0/302692691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484003e60 con 0x7f048c10e8f0 2024-04-22T13:29:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.826+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a7410 con 0x7f048c10e8f0 2024-04-22T13:29:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f04897fa640 1 -- 172.21.15.5:0/302692691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f04840324c0 con 0x7f048c10e8f0 2024-04-22T13:29:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 Processor -- start 2024-04-22T13:29:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 -- start start 2024-04-22T13:29:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d928d0 0x55c365d91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d91fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d9b0e0 0x55c365d9d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c365cf68d0 con 0x55c365d9b0e0 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c365c93e80 con 0x55c365d91b60 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.829+0000 7f0495014740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c365cf2890 con 0x55c365d928d0 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d9b0e0 0x55c365d9d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d91fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d91fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37378/0 (socket says 172.21.15.5:37378) 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 -- 172.21.15.5:0/2922076256 learned_addr learned my addr 172.21.15.5:0/2922076256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d928d0 0x55c365d91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04897fa640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f04740a3d10 0x7f04740a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 -- 172.21.15.5:0/2922076256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d928d0 msgr2=0x55c365d91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d928d0 0x55c365d91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 -- 172.21.15.5:0/2922076256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d9b0e0 msgr2=0x55c365d9d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d9b0e0 0x55c365d9d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 -- 172.21.15.5:0/2922076256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c365cb5320 con 0x55c365d91b60 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04908e8640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f04740a3d10 0x7f04740a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04897fa640 1 -- 172.21.15.5:0/302692691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f04840f2ad0 con 0x7f048c10e8f0 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f048bfff640 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d91fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f047c00bcf0 tx=0x7f047c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=935c83a6f01ee5f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04727fc640 1 -- 172.21.15.5:0/2922076256 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c015070 con 0x55c365d91b60 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04908e8640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f04740a3d10 0x7f04740a61d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f04840beae0 tx=0x7f04840a9680 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04727fc640 1 -- 172.21.15.5:0/2922076256 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c01a910 con 0x55c365d91b60 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.830+0000 7f04727fc640 1 -- 172.21.15.5:0/2922076256 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c011900 con 0x55c365d91b60 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 -- 172.21.15.5:0/2922076256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 msgr2=0x55c365d91fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d91fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f047c00bcf0 tx=0x7f047c00f730 comp rx=0 tx=0).stop 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 -- 172.21.15.5:0/2922076256 shutdown_connections 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d9b0e0 0x55c365d9d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d91fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 --2- 172.21.15.5:0/2922076256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d928d0 0x55c365d91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 -- 172.21.15.5:0/2922076256 >> 172.21.15.5:0/2922076256 conn(0x55c365d58c90 msgr2=0x55c365d8c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 -- 172.21.15.5:0/2922076256 shutdown_connections 2024-04-22T13:29:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.831+0000 7f0495014740 1 -- 172.21.15.5:0/2922076256 wait complete. 2024-04-22T13:29:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 -- start start 2024-04-22T13:29:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 Processor -- start 2024-04-22T13:29:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d97e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c365d9de10 con 0x55c365d928d0 2024-04-22T13:29:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c365c7eeb0 con 0x55c365d91b60 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c365d9df90 con 0x55c365d9b0e0 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f0495014740 10 client.0 _ll_register_callbacks cb 0x55c365cabb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f048bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d97e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f048bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d97e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37946/0 (socket says 172.21.15.5:37946) 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f048bfff640 1 -- 172.21.15.5:0/2919943099 learned_addr learned my addr 172.21.15.5:0/2919943099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.832+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048bfff640 1 -- 172.21.15.5:0/2919943099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 msgr2=0x55c365d983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048bfff640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048bfff640 1 -- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 msgr2=0x55c365d9aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048bfff640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d9aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048bfff640 1 -- 172.21.15.5:0/2919943099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c365a09210 con 0x55c365d928d0 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f048bfff640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d97e20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f047c0115e0 tx=0x7f047c01a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b468926daae164 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f046b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f046b7fe640 1 -- 172.21.15.5:0/2919943099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c028060 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c01a910 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f046b7fe640 1 -- 172.21.15.5:0/2919943099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f047c01aac0 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f046b7fe640 1 -- 172.21.15.5:0/2919943099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f047c015070 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f0495014740 10 client.16752 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c365ad5620 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.833+0000 7f046b7fe640 1 -- 172.21.15.5:0/2919943099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c01a910 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f046b7fe640 1 -- 172.21.15.5:0/2919943099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f047c0118f0 con 0x55c365d928d0 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f0495014740 10 client.16752 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f0495014740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f0495014740 20 client.16752 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f0495014740 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365dab060 0x55c365dad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f0495014740 4 client.16752 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c365d0a880 con 0x55c365dab060 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365dab060 0x55c365dad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f046b7fe640 10 client.16752 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.834+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365dab060 0x55c365dad450 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3aad5465aac2b2 server_cookie=859bfef5f291e9b4 in_seq=0 out_seq=0 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f046b7fe640 1 -- 172.21.15.5:0/2919943099 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c365d0a880 con 0x55c365dab060 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f046b7fe640 10 client.16752 handle_command_reply: tid=0 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 client.16752 shutdown 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 msgr2=0x55c365d97e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d97e20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f047c0115e0 tx=0x7f047c01a460 comp rx=0 tx=0).stop 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 shutdown_connections 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365dab060 0x55c365dad450 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d97e20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 --2- 172.21.15.5:0/2919943099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d91b60 0x55c365d9aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 >> 172.21.15.5:0/2919943099 conn(0x55c365d58c90 msgr2=0x55c365d8e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f046b7fe640 0 client.16752 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 shutdown_connections 2024-04-22T13:29:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.835+0000 7f0495014740 1 -- 172.21.15.5:0/2919943099 wait complete. 2024-04-22T13:29:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.836+0000 7f0495014740 20 client.16752 trim_cache size 0 max 16384 2024-04-22T13:29:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 Processor -- start 2024-04-22T13:29:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 -- start start 2024-04-22T13:29:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 0x55c365da6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d928d0 0x55c365d9a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1afd60 con 0x55c365d91b60 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b18a0 con 0x55c365d928d0 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f0495014740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b0430 con 0x55c365d9b0e0 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 0x55c365da6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 0x55c365da6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37950/0 (socket says 172.21.15.5:37950) 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 -- 172.21.15.5:0/190347628 learned_addr learned my addr 172.21.15.5:0/190347628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048bfff640 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d928d0 0x55c365d9a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 -- 172.21.15.5:0/190347628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 msgr2=0x55c365d9aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 -- 172.21.15.5:0/190347628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d928d0 msgr2=0x55c365d9a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d928d0 0x55c365d9a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 -- 172.21.15.5:0/190347628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c365d98730 con 0x55c365d91b60 2024-04-22T13:29:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.851+0000 7f048b7fe640 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 0x55c365da6d40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f048000e7e0 tx=0x7f04800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8182b9c15714dd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0471ffb640 1 -- 172.21.15.5:0/190347628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480012d90 con 0x55c365d91b60 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0471ffb640 1 -- 172.21.15.5:0/190347628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f048001f920 con 0x55c365d91b60 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0471ffb640 1 -- 172.21.15.5:0/190347628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480010a80 con 0x55c365d91b60 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 -- 172.21.15.5:0/190347628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 msgr2=0x55c365da6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 0x55c365da6d40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f048000e7e0 tx=0x7f04800046c0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 -- 172.21.15.5:0/190347628 shutdown_connections 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d9b0e0 0x55c365d9aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d928d0 0x55c365d9a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 --2- 172.21.15.5:0/190347628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d91b60 0x55c365da6d40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 -- 172.21.15.5:0/190347628 >> 172.21.15.5:0/190347628 conn(0x55c365d58c90 msgr2=0x55c365d966f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 -- 172.21.15.5:0/190347628 shutdown_connections 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.852+0000 7f0495014740 1 -- 172.21.15.5:0/190347628 wait complete. 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 -- start start 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 Processor -- start 2024-04-22T13:29:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d91b60 0x55c365da36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d9b0e0 0x55c365d9efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b11d0 con 0x55c365d928d0 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1b0b00 con 0x55c365d9b0e0 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1add40 con 0x55c365d91b60 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f0495014740 10 client.0 _ll_register_callbacks cb 0x7f048000e520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f048bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f048bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d9ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37954/0 (socket says 172.21.15.5:37954) 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f048bfff640 1 -- 172.21.15.5:0/2588846994 learned_addr learned my addr 172.21.15.5:0/2588846994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d91b60 0x55c365da36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.853+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d9b0e0 0x55c365d9efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048bfff640 1 -- 172.21.15.5:0/2588846994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d91b60 msgr2=0x55c365da36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048bfff640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d91b60 0x55c365da36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048bfff640 1 -- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d9b0e0 msgr2=0x55c365d9efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048bfff640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d9b0e0 0x55c365d9efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048bfff640 1 -- 172.21.15.5:0/2588846994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c365d9f810 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d91b60 0x55c365da36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f048bfff640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d9ea40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f047c00f770 tx=0x7f047c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e945c76e233cb136 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f046b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f046b7fe640 1 -- 172.21.15.5:0/2588846994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c03a060 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f046b7fe640 1 -- 172.21.15.5:0/2588846994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f047c01aa70 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c365c963d0 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f046b7fe640 1 -- 172.21.15.5:0/2588846994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f047c028040 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f0495014740 10 client.16758 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c365e113a0 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.854+0000 7f046b7fe640 1 -- 172.21.15.5:0/2588846994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c365c963d0 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f046b7fe640 1 -- 172.21.15.5:0/2588846994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f047c01ac70 con 0x55c365d928d0 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f0495014740 10 client.16758 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f0495014740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f0495014740 20 client.16758 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f0495014740 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365e11ac0 0x55c365e13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f0495014740 4 client.16758 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:29:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c365d042f0 con 0x55c365e11ac0 2024-04-22T13:29:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365e11ac0 0x55c365e13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f046b7fe640 10 client.16758 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.855+0000 7f048b7fe640 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365e11ac0 0x55c365e13ee0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa28dd3099027d40 server_cookie=c9d9fc25650d72e1 in_seq=0 out_seq=0 2024-04-22T13:29:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f046b7fe640 1 -- 172.21.15.5:0/2588846994 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c365d042f0 con 0x55c365e11ac0 2024-04-22T13:29:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f046b7fe640 10 client.16758 handle_command_reply: tid=0 2024-04-22T13:29:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 client.16758 shutdown 2024-04-22T13:29:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 msgr2=0x55c365d9ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d9ea40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f047c00f770 tx=0x7f047c005c50 comp rx=0 tx=0).stop 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 shutdown_connections 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c365e11ac0 0x55c365e13ee0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c365d928d0 0x55c365d9ea40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c365d9b0e0 0x55c365d9efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f046b7fe640 0 client.16758 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 --2- 172.21.15.5:0/2588846994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c365d91b60 0x55c365da36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 >> 172.21.15.5:0/2588846994 conn(0x55c365d58c90 msgr2=0x55c365d8b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 shutdown_connections 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 1 -- 172.21.15.5:0/2588846994 wait complete. 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.859+0000 7f0495014740 20 client.16758 trim_cache size 0 max 16384 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:29:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:29:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:29:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f04740a3d10 msgr2=0x7f04740a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f04740a3d10 0x7f04740a61d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f04840beae0 tx=0x7f04840a9680 comp rx=0 tx=0).stop 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 msgr2=0x7f048c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c19a730 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0484004af0 tx=0x7f0484034760 comp rx=0 tx=0).stop 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 shutdown_connections 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f048c10e8f0 0x7f048c19a730 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f04740a3d10 0x7f04740a61d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f048c101660 0x7f048c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 --2- 172.21.15.5:0/302692691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f048c100d40 0x7f048c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 >> 172.21.15.5:0/302692691 conn(0x7f048c077f70 msgr2=0x7f048c10fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 shutdown_connections 2024-04-22T13:29:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:31.860+0000 7f049234b640 1 -- 172.21.15.5:0/302692691 wait complete. 2024-04-22T13:29:33.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:32 smithi049 ceph-mon[31379]: pgmap v75: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 MiB/s wr, 18 op/s 2024-04-22T13:29:33.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:32 smithi107 ceph-mon[31859]: pgmap v75: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 MiB/s wr, 18 op/s 2024-04-22T13:29:33.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:32 smithi005 ceph-mon[28006]: pgmap v75: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 MiB/s wr, 18 op/s 2024-04-22T13:29:34.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 -- 172.21.15.5:0/2706946565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0105dd0 msgr2=0x7f78a01061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2706946565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0105dd0 0x7f78a01061b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7890009ca0 tx=0x7f789002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 -- 172.21.15.5:0/2706946565 shutdown_connections 2024-04-22T13:29:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2706946565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 0x7f78a010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2706946565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a01010a0 0x7f78a0101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2706946565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0105dd0 0x7f78a01061b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 -- 172.21.15.5:0/2706946565 >> 172.21.15.5:0/2706946565 conn(0x7f78a0077c40 msgr2=0x7f78a0078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 -- 172.21.15.5:0/2706946565 shutdown_connections 2024-04-22T13:29:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.762+0000 7f78a54ba640 1 -- 172.21.15.5:0/2706946565 wait complete. 2024-04-22T13:29:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 Processor -- start 2024-04-22T13:29:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 -- start start 2024-04-22T13:29:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 0x7f78a019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 0x7f78a019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0105dd0 0x7f78a01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0111150 con 0x7f78a01010a0 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0110fd0 con 0x7f78a0105dd0 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f78a54ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78a01112d0 con 0x7f78a0101a40 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f789effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 0x7f78a019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f789effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 0x7f78a019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37964/0 (socket says 172.21.15.5:37964) 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f789effd640 1 -- 172.21.15.5:0/2410177163 learned_addr learned my addr 172.21.15.5:0/2410177163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.763+0000 7f789e7fc640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 0x7f78a019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789f7fe640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0105dd0 0x7f78a01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789effd640 1 -- 172.21.15.5:0/2410177163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 msgr2=0x7f78a019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789effd640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 0x7f78a019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789effd640 1 -- 172.21.15.5:0/2410177163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0105dd0 msgr2=0x7f78a01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789effd640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0105dd0 0x7f78a01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789effd640 1 -- 172.21.15.5:0/2410177163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a0198d30 con 0x7f78a01010a0 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789e7fc640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 0x7f78a019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f789effd640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 0x7f78a019e400 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7890005e30 tx=0x7f78900346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890039040 con 0x7f78a01010a0 2024-04-22T13:29:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7890044070 con 0x7f78a01010a0 2024-04-22T13:29:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a0198fc0 con 0x7f78a01010a0 2024-04-22T13:29:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900042a0 con 0x7f78a01010a0 2024-04-22T13:29:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.764+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a01a52c0 con 0x7f78a01010a0 2024-04-22T13:29:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.765+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a0069250 con 0x7f78a01010a0 2024-04-22T13:29:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.766+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7890004440 con 0x7f78a01010a0 2024-04-22T13:29:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.767+0000 7f7887fff640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f78880a3d20 0x7f78880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.767+0000 7f789e7fc640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f78880a3d20 0x7f78880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.767+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f78900f2a40 con 0x7f78a01010a0 2024-04-22T13:29:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.768+0000 7f789e7fc640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f78880a3d20 0x7f78880a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f78a0105770 tx=0x7f7894005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:34.772+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f789003e3a0 con 0x7f78a01010a0 2024-04-22T13:29:34.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.002+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78a010a330 con 0x7f78a01010a0 2024-04-22T13:29:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.003+0000 7f7887fff640 1 -- 172.21.15.5:0/2410177163 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f78900bf250 con 0x7f78a01010a0 2024-04-22T13:29:34.999 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:29:34.999 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:29:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f78880a3d20 msgr2=0x7f78880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f78880a3d20 0x7f78880a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f78a0105770 tx=0x7f7894005f90 comp rx=0 tx=0).stop 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 msgr2=0x7f78a019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 0x7f78a019e400 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7890005e30 tx=0x7f78900346f0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 shutdown_connections 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f78880a3d20 0x7f78880a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0105dd0 0x7f78a01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78a0101a40 0x7f78a019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 --2- 172.21.15.5:0/2410177163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a01010a0 0x7f78a019e400 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 >> 172.21.15.5:0/2410177163 conn(0x7f78a0077c40 msgr2=0x7f78a006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 shutdown_connections 2024-04-22T13:29:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.005+0000 7f78a54ba640 1 -- 172.21.15.5:0/2410177163 wait complete. 2024-04-22T13:29:35.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:34 smithi005 ceph-mon[28006]: pgmap v76: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-04-22T13:29:35.012 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db --set-id d7c67eb --release --await' 2024-04-22T13:29:35.012 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db --set-id d7c67eb --release --await 2024-04-22T13:29:35.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:34 smithi049 ceph-mon[31379]: pgmap v76: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-04-22T13:29:35.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:34 smithi107 ceph-mon[31859]: pgmap v76: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 -- 172.21.15.5:0/2288984313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134102a60 msgr2=0x7f4134102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2288984313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134102a60 0x7f4134102ec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4124009930 tx=0x7f412402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 -- 172.21.15.5:0/2288984313 shutdown_connections 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2288984313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134102a60 0x7f4134102ec0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2288984313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103400 0x7f413410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2288984313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4134101180 0x7f4134101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 -- 172.21.15.5:0/2288984313 >> 172.21.15.5:0/2288984313 conn(0x7f4134077f70 msgr2=0x7f4134078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 -- 172.21.15.5:0/2288984313 shutdown_connections 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.125+0000 7f413c4ba640 1 -- 172.21.15.5:0/2288984313 wait complete. 2024-04-22T13:29:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 Processor -- start 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 -- start start 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4134101180 0x7f4134199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134102a60 0x7f4134199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 0x7f41341a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111180 con 0x7f4134103400 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111000 con 0x7f4134101180 2024-04-22T13:29:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413c4ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111300 con 0x7f4134102a60 2024-04-22T13:29:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 0x7f41341a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 0x7f41341a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37984/0 (socket says 172.21.15.5:37984) 2024-04-22T13:29:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 -- 172.21.15.5:0/2221581568 learned_addr learned my addr 172.21.15.5:0/2221581568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413a256640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4134101180 0x7f4134199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f4139a55640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134102a60 0x7f4134199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 -- 172.21.15.5:0/2221581568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134102a60 msgr2=0x7f4134199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134102a60 0x7f4134199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 -- 172.21.15.5:0/2221581568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4134101180 msgr2=0x7f4134199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4134101180 0x7f4134199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.126+0000 7f413aa57640 1 -- 172.21.15.5:0/2221581568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341a4c40 con 0x7f4134103400 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.127+0000 7f413aa57640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 0x7f41341a4700 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.127+0000 7f412b7fe640 1 -- 172.21.15.5:0/2221581568 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c017070 con 0x7f4134103400 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.127+0000 7f412b7fe640 1 -- 172.21.15.5:0/2221581568 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412c003e20 con 0x7f4134103400 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.127+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41341a4e10 con 0x7f4134103400 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.127+0000 7f412b7fe640 1 -- 172.21.15.5:0/2221581568 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c010460 con 0x7f4134103400 2024-04-22T13:29:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.127+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a52e0 con 0x7f4134103400 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f412b7fe640 1 -- 172.21.15.5:0/2221581568 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f412c003970 con 0x7f4134103400 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 Processor -- start 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 -- start start 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf474b310 0x55dbf474b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf4754810 0x55dbf4756c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbf46b0500 con 0x55dbf474bc30 2024-04-22T13:29:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dbf46566a0 con 0x55dbf4754810 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.130+0000 7f413f185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dbf46ac4c0 con 0x55dbf474b310 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37998/0 (socket says 172.21.15.5:37998) 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 -- 172.21.15.5:0/2323048720 learned_addr learned my addr 172.21.15.5:0/2323048720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f412b7fe640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f41200a3d20 0x7f41200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f4139a55640 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf474b310 0x55dbf474b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413aa57640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f41200a3d20 0x7f41200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 -- 172.21.15.5:0/2323048720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf474b310 msgr2=0x55dbf474b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf474b310 0x55dbf474b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f4139a55640 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf4754810 0x55dbf4756c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 -- 172.21.15.5:0/2323048720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf4754810 msgr2=0x55dbf4756c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf4754810 0x55dbf4756c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 -- 172.21.15.5:0/2323048720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf46755e0 con 0x55dbf474bc30 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f412b7fe640 1 -- 172.21.15.5:0/2221581568 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f412c0d0490 con 0x7f4134103400 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f4139a55640 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf4754810 0x55dbf4756c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413a256640 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f413000bcd0 tx=0x7f41300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f1e8646bd9fa55 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f413aa57640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f41200a3d20 0x7f41200a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f412c0ce210 tx=0x7f412c004020 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f4128bfe640 1 -- 172.21.15.5:0/2323048720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130005220 con 0x55dbf474bc30 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f4128bfe640 1 -- 172.21.15.5:0/2323048720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4130004c50 con 0x55dbf474bc30 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.131+0000 7f4128bfe640 1 -- 172.21.15.5:0/2323048720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300116a0 con 0x55dbf474bc30 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 -- 172.21.15.5:0/2323048720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 msgr2=0x55dbf47542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f413000bcd0 tx=0x7f41300107a0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 -- 172.21.15.5:0/2323048720 shutdown_connections 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf4754810 0x55dbf4756c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 --2- 172.21.15.5:0/2323048720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf474b310 0x55dbf474b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 -- 172.21.15.5:0/2323048720 >> 172.21.15.5:0/2323048720 conn(0x55dbf4745a40 msgr2=0x55dbf4745e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 -- 172.21.15.5:0/2323048720 shutdown_connections 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.132+0000 7f413f185740 1 -- 172.21.15.5:0/2323048720 wait complete. 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 -- start start 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 Processor -- start 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf47542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4751660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf4751da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4757600 con 0x55dbf474bc30 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4669a60 con 0x55dbf474b310 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4757780 con 0x55dbf4754810 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413f185740 10 client.0 _ll_register_callbacks cb 0x55dbf43c2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4751660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4751660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38004/0 (socket says 172.21.15.5:38004) 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 -- 172.21.15.5:0/84071595 learned_addr learned my addr 172.21.15.5:0/84071595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f4139a55640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf4751da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 -- 172.21.15.5:0/84071595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 msgr2=0x55dbf4751da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf4751da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 -- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 msgr2=0x55dbf47542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf47542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 -- 172.21.15.5:0/84071595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf43c2920 con 0x55dbf474bc30 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f4139a55640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf4751da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f413a256640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4751660 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f413000bca0 tx=0x7f41300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e6cbeb111087f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.133+0000 7f411dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f411dffb640 1 -- 172.21.15.5:0/84071595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130005010 con 0x55dbf474bc30 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f411dffb640 1 -- 172.21.15.5:0/84071595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f41300051b0 con 0x55dbf474bc30 2024-04-22T13:29:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4130004c50 con 0x55dbf474bc30 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f411dffb640 1 -- 172.21.15.5:0/84071595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41300053b0 con 0x55dbf474bc30 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 10 client.16776 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbf448ed30 con 0x55dbf474bc30 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f411dffb640 1 -- 172.21.15.5:0/84071595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4130004c50 con 0x55dbf474bc30 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f411dffb640 1 -- 172.21.15.5:0/84071595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f413001b430 con 0x55dbf474bc30 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 10 client.16776 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 10 client.16776 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 20 client.16776 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47648b0 0x55dbf4766ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 4 client.16776 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf4683a10 con 0x55dbf47648b0 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.134+0000 7f4139a55640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47648b0 0x55dbf4766ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.135+0000 7f4139a55640 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47648b0 0x55dbf4766ca0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc044b628bd5069d server_cookie=8da123dce5f0d1b4 in_seq=0 out_seq=0 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.135+0000 7f411dffb640 10 client.16776 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f411dffb640 1 -- 172.21.15.5:0/84071595 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbf4683a10 con 0x55dbf47648b0 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f411dffb640 10 client.16776 handle_command_reply: tid=0 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 client.16776 shutdown 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 msgr2=0x55dbf4751660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4751660 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f413000bca0 tx=0x7f41300048f0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 shutdown_connections 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47648b0 0x55dbf4766ca0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4751660 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf4751da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f411dffb640 0 client.16776 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 --2- 172.21.15.5:0/84071595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf47542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 >> 172.21.15.5:0/84071595 conn(0x55dbf4745a40 msgr2=0x55dbf4747790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 shutdown_connections 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 1 -- 172.21.15.5:0/84071595 wait complete. 2024-04-22T13:29:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.136+0000 7f413f185740 20 client.16776 trim_cache size 0 max 16384 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 Processor -- start 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 -- start start 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 0x55dbf4753d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf476c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41341adaa0 con 0x55dbf474b310 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41341af5e0 con 0x55dbf474bc30 2024-04-22T13:29:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f413f185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ae170 con 0x55dbf4754810 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f4139a55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 0x55dbf4753d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f4139a55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 0x55dbf4753d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38010/0 (socket says 172.21.15.5:38010) 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.153+0000 7f4139a55640 1 -- 172.21.15.5:0/201419820 learned_addr learned my addr 172.21.15.5:0/201419820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f4139a55640 1 -- 172.21.15.5:0/201419820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 msgr2=0x55dbf476c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f4139a55640 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf476c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f4139a55640 1 -- 172.21.15.5:0/201419820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474bc30 msgr2=0x55dbf47542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f4139a55640 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f4139a55640 1 -- 172.21.15.5:0/201419820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf47574e0 con 0x55dbf474b310 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f4139a55640 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 0x55dbf4753d90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4124005ec0 tx=0x7f412402c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72fe8aed7ed469f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f411ffff640 1 -- 172.21.15.5:0/201419820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124004260 con 0x55dbf474b310 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f411ffff640 1 -- 172.21.15.5:0/201419820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412402cc90 con 0x55dbf474b310 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f411ffff640 1 -- 172.21.15.5:0/201419820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124004920 con 0x55dbf474b310 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 -- 172.21.15.5:0/201419820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 msgr2=0x55dbf4753d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 0x55dbf4753d90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4124005ec0 tx=0x7f412402c360 comp rx=0 tx=0).stop 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 -- 172.21.15.5:0/201419820 shutdown_connections 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf476c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474bc30 0x55dbf47542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 --2- 172.21.15.5:0/201419820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474b310 0x55dbf4753d90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.154+0000 7f413f185740 1 -- 172.21.15.5:0/201419820 >> 172.21.15.5:0/201419820 conn(0x55dbf4745a40 msgr2=0x55dbf4744b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 -- 172.21.15.5:0/201419820 shutdown_connections 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 -- 172.21.15.5:0/201419820 wait complete. 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 -- start start 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 Processor -- start 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf475caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4758e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf47593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41341aef10 con 0x55dbf474bc30 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ae840 con 0x55dbf474b310 2024-04-22T13:29:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41341aba80 con 0x55dbf4754810 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413f185740 10 client.0 _ll_register_callbacks cb 0x7f413000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4758e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4758e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38012/0 (socket says 172.21.15.5:38012) 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 -- 172.21.15.5:0/3695281704 learned_addr learned my addr 172.21.15.5:0/3695281704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f4139a55640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf47593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f4139a55640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf475caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 -- 172.21.15.5:0/3695281704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 msgr2=0x55dbf47593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf47593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 -- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 msgr2=0x55dbf475caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf475caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f413a256640 1 -- 172.21.15.5:0/3695281704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf47598f0 con 0x55dbf474bc30 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.155+0000 7f4139a55640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf475caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413a256640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4758e70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f413001b8d0 tx=0x7f413000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a3cdc0e7a3f929 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f411dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f411dffb640 1 -- 172.21.15.5:0/3695281704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130033070 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f411dffb640 1 -- 172.21.15.5:0/3695281704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f4130005010 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412402cc90 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f411dffb640 1 -- 172.21.15.5:0/3695281704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f413003f050 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 10 client.16782 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbf47cb9f0 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f411dffb640 1 -- 172.21.15.5:0/3695281704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f412402cc90 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f411dffb640 1 -- 172.21.15.5:0/3695281704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f413001b4c0 con 0x55dbf474bc30 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 10 client.16782 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 10 client.16782 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 20 client.16782 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47cc140 0x55dbf47ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 4 client.16782 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d7c67eb", "release": true, "await": true}] 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.156+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d7c67eb", "release": true, "await": true}) -- 0x55dbf46bdf20 con 0x55dbf47cc140 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.157+0000 7f4139a55640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47cc140 0x55dbf47ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.157+0000 7f411dffb640 10 client.16782 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.157+0000 7f4139a55640 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47cc140 0x55dbf47ce560 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de18416a9c8645d server_cookie=d231d5d0c2d52983 in_seq=0 out_seq=0 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f411dffb640 1 -- 172.21.15.5:0/3695281704 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55dbf46bdf20 con 0x55dbf47cc140 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f411dffb640 10 client.16782 handle_command_reply: tid=0 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 client.16782 shutdown 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 msgr2=0x55dbf4758e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4758e70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f413001b8d0 tx=0x7f413000b8e0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 shutdown_connections 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbf47cc140 0x55dbf47ce560 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbf474bc30 0x55dbf4758e70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f411dffb640 0 client.16782 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbf4754810 0x55dbf47593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 --2- 172.21.15.5:0/3695281704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbf474b310 0x55dbf475caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 >> 172.21.15.5:0/3695281704 conn(0x55dbf4745a40 msgr2=0x55dbf4746a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 shutdown_connections 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 1 -- 172.21.15.5:0/3695281704 wait complete. 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.160+0000 7f413f185740 20 client.16782 trim_cache size 0 max 16384 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 16, 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "d7c67eb": { 2024-04-22T13:29:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 16, 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 60.0, 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 86.5, 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:29:35.162 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f41200a3d20 msgr2=0x7f41200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f41200a3d20 0x7f41200a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f412c0ce210 tx=0x7f412c004020 comp rx=0 tx=0).stop 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 msgr2=0x7f41341a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 0x7f41341a4700 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c00b260 comp rx=0 tx=0).stop 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 shutdown_connections 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4134103400 0x7f41341a4700 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f41200a3d20 0x7f41200a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134102a60 0x7f4134199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 --2- 172.21.15.5:0/2221581568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4134101180 0x7f4134199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 >> 172.21.15.5:0/2221581568 conn(0x7f4134077f70 msgr2=0x7f4134104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 shutdown_connections 2024-04-22T13:29:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:35.161+0000 7f413c4ba640 1 -- 172.21.15.5:0/2221581568 wait complete. 2024-04-22T13:29:35.174 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: d7c67eb, seconds elapsed: 27.6 2024-04-22T13:29:36.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:35 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2410177163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:36.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:35 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2410177163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:36.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:35 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2410177163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:37.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:36 smithi049 ceph-mon[31379]: pgmap v77: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 86 KiB/s wr, 14 op/s 2024-04-22T13:29:37.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:36 smithi107 ceph-mon[31859]: pgmap v77: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 86 KiB/s wr, 14 op/s 2024-04-22T13:29:37.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:36 smithi005 ceph-mon[28006]: pgmap v77: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 86 KiB/s wr, 14 op/s 2024-04-22T13:29:39.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:38 smithi049 ceph-mon[31379]: pgmap v78: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 KiB/s wr, 10 op/s 2024-04-22T13:29:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:38 smithi107 ceph-mon[31859]: pgmap v78: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 KiB/s wr, 10 op/s 2024-04-22T13:29:39.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:38 smithi005 ceph-mon[28006]: pgmap v78: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 KiB/s wr, 10 op/s 2024-04-22T13:29:41.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:40 smithi049 ceph-mon[31379]: pgmap v79: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 KiB/s wr, 10 op/s 2024-04-22T13:29:41.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:40 smithi107 ceph-mon[31859]: pgmap v79: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 KiB/s wr, 10 op/s 2024-04-22T13:29:41.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:40 smithi005 ceph-mon[28006]: pgmap v79: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 86 KiB/s wr, 10 op/s 2024-04-22T13:29:43.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:42 smithi049 ceph-mon[31379]: pgmap v80: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 517 KiB/s wr, 15 op/s 2024-04-22T13:29:43.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:42 smithi107 ceph-mon[31859]: pgmap v80: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 517 KiB/s wr, 15 op/s 2024-04-22T13:29:43.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:42 smithi005 ceph-mon[28006]: pgmap v80: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 517 KiB/s wr, 15 op/s 2024-04-22T13:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 -- 172.21.15.5:0/221418277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 msgr2=0x7efee41035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/221418277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee41035a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efee0009900 tx=0x7efee002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 -- 172.21.15.5:0/221418277 shutdown_connections 2024-04-22T13:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/221418277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee410c590 0x7efee410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/221418277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efee4103ae0 0x7efee410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/221418277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee41035a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 -- 172.21.15.5:0/221418277 >> 172.21.15.5:0/221418277 conn(0x7efee4077fa0 msgr2=0x7efee40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 -- 172.21.15.5:0/221418277 shutdown_connections 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.953+0000 7efeeb6f2640 1 -- 172.21.15.5:0/221418277 wait complete. 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 Processor -- start 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 -- start start 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efee4103ae0 0x7efee419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee410c590 0x7efee41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee41121c0 con 0x7efee41031c0 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112040 con 0x7efee410c590 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efeeb6f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112340 con 0x7efee4103ae0 2024-04-22T13:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efee948e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efee948e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45126/0 (socket says 172.21.15.5:45126) 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efee948e640 1 -- 172.21.15.5:0/1620120730 learned_addr learned my addr 172.21.15.5:0/1620120730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.954+0000 7efee9c8f640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee410c590 0x7efee41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee8c8d640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efee4103ae0 0x7efee419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee948e640 1 -- 172.21.15.5:0/1620120730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efee4103ae0 msgr2=0x7efee419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee948e640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efee4103ae0 0x7efee419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee948e640 1 -- 172.21.15.5:0/1620120730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee410c590 msgr2=0x7efee41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee948e640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee410c590 0x7efee41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee948e640 1 -- 172.21.15.5:0/1620120730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee4198d40 con 0x7efee41031c0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efee948e640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee419e410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efee0005940 tx=0x7efee002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efeda7fc640 1 -- 172.21.15.5:0/1620120730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0043020 con 0x7efee41031c0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efeda7fc640 1 -- 172.21.15.5:0/1620120730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee002dc90 con 0x7efee41031c0 2024-04-22T13:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee4198fd0 con 0x7efee41031c0 2024-04-22T13:29:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efeda7fc640 1 -- 172.21.15.5:0/1620120730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0003ab0 con 0x7efee41031c0 2024-04-22T13:29:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.955+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a52d0 con 0x7efee41031c0 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.958+0000 7efeda7fc640 1 -- 172.21.15.5:0/1620120730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efee0046020 con 0x7efee41031c0 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.958+0000 7efeee3c3740 1 Processor -- start 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.958+0000 7efeee3c3740 1 -- start start 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeee3c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2393388d0 con 0x55a2393d48d0 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeee3c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a2392d5e80 con 0x55a2393d3b60 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeee3c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a239334890 con 0x55a2393dd0e0 2024-04-22T13:29:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeda7fc640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efed00a3d10 0x7efed00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee8c8d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393d3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45128/0 (socket says 172.21.15.5:45128) 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 -- 172.21.15.5:0/3393217467 learned_addr learned my addr 172.21.15.5:0/3393217467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efed00a3d10 0x7efed00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efeda7fc640 1 -- 172.21.15.5:0/1620120730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efee00f2890 con 0x7efee41031c0 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee948e640 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 -- 172.21.15.5:0/3393217467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 msgr2=0x55a2393df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 -- 172.21.15.5:0/3393217467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 msgr2=0x55a2393d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393d3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.959+0000 7efee9c8f640 1 -- 172.21.15.5:0/3393217467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2392f7320 con 0x55a2393d48d0 2024-04-22T13:29:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.960+0000 7efee9c8f640 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393d3620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7efedc005cc0 tx=0x7efedc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00b362fbffb4db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.960+0000 7efee9c8f640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efed00a3d10 0x7efed00a61d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efedc011cb0 tx=0x7efedc016000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.960+0000 7efecf7fe640 1 -- 172.21.15.5:0/3393217467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00cd70 con 0x55a2393d48d0 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.960+0000 7efecf7fe640 1 -- 172.21.15.5:0/3393217467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efedc00ac90 con 0x55a2393d48d0 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.960+0000 7efecf7fe640 1 -- 172.21.15.5:0/3393217467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc020660 con 0x55a2393d48d0 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 -- 172.21.15.5:0/3393217467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 msgr2=0x55a2393d3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393d3620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7efedc005cc0 tx=0x7efedc0117b0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 -- 172.21.15.5:0/3393217467 shutdown_connections 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393d3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3393217467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393d3620 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 -- 172.21.15.5:0/3393217467 >> 172.21.15.5:0/3393217467 conn(0x55a23939ac90 msgr2=0x55a2393ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 -- 172.21.15.5:0/3393217467 shutdown_connections 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 -- 172.21.15.5:0/3393217467 wait complete. 2024-04-22T13:29:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 -- start start 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.961+0000 7efeee3c3740 1 Processor -- start 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 0x55a2393dca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d48d0 0x55a2393d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2393dfdb0 con 0x55a2393d3b60 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a2392c0eb0 con 0x55a2393d48d0 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2393dff30 con 0x55a2393dd0e0 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 10 client.0 _ll_register_callbacks cb 0x55a2392edb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 0x55a2393dca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 0x55a2393dca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45136/0 (socket says 172.21.15.5:45136) 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 -- 172.21.15.5:0/351185657 learned_addr learned my addr 172.21.15.5:0/351185657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee9c8f640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee8c8d640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d48d0 0x55a2393d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 -- 172.21.15.5:0/351185657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 msgr2=0x55a2393da3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393da3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 -- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d48d0 msgr2=0x55a2393d9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d48d0 0x55a2393d9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 -- 172.21.15.5:0/351185657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23904b210 con 0x55a2393d3b60 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efee948e640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 0x55a2393dca90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7efee00f01d0 tx=0x7efee002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e72e408b821f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeccff9640 1 -- 172.21.15.5:0/351185657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00c75e0 con 0x55a2393d3b60 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeccff9640 1 -- 172.21.15.5:0/351185657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efee0003c50 con 0x55a2393d3b60 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.962+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efedc00ac90 con 0x55a2393d3b60 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeccff9640 1 -- 172.21.15.5:0/351185657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efee0003e50 con 0x55a2393d3b60 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 10 client.16794 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a239117620 con 0x55a2393d3b60 2024-04-22T13:29:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeccff9640 1 -- 172.21.15.5:0/351185657 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efedc00ac90 con 0x55a2393d3b60 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeccff9640 1 -- 172.21.15.5:0/351185657 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efee0004f90 con 0x55a2393d3b60 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 10 client.16794 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 20 client.16794 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a2393ed0e0 0x55a2393ef510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 4 client.16794 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a23934c880 con 0x55a2393ed0e0 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.963+0000 7efee8c8d640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a2393ed0e0 0x55a2393ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.964+0000 7efee8c8d640 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a2393ed0e0 0x55a2393ef510 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb82f2a356072f01 server_cookie=e2b709d7b2f5d97b in_seq=0 out_seq=0 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.964+0000 7efeccff9640 10 client.16794 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.964+0000 7efeccff9640 1 -- 172.21.15.5:0/351185657 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a23934c880 con 0x55a2393ed0e0 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.964+0000 7efeccff9640 10 client.16794 handle_command_reply: tid=0 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.964+0000 7efeee3c3740 1 client.16794 shutdown 2024-04-22T13:29:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 msgr2=0x55a2393dca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 0x55a2393dca90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7efee00f01d0 tx=0x7efee002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 shutdown_connections 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a2393ed0e0 0x55a2393ef510 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393dd0e0 0x55a2393da3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d48d0 0x55a2393d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeccff9640 0 client.16794 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 --2- 172.21.15.5:0/351185657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d3b60 0x55a2393dca90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 >> 172.21.15.5:0/351185657 conn(0x55a23939ac90 msgr2=0x55a2393d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 shutdown_connections 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 1 -- 172.21.15.5:0/351185657 wait complete. 2024-04-22T13:29:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.965+0000 7efeee3c3740 20 client.16794 trim_cache size 0 max 16384 2024-04-22T13:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 Processor -- start 2024-04-22T13:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 -- start start 2024-04-22T13:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d48d0 0x55a2393dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 0x55a2393dca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee41adbc0 con 0x55a2393dd0e0 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efee41af700 con 0x55a2393d3b60 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efeee3c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efee41ae290 con 0x55a2393d48d0 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efee9c8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 0x55a2393dca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efee9c8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 0x55a2393dca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45148/0 (socket says 172.21.15.5:45148) 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.980+0000 7efee9c8f640 1 -- 172.21.15.5:0/4191257165 learned_addr learned my addr 172.21.15.5:0/4191257165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee948e640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee8c8d640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d48d0 0x55a2393dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee9c8f640 1 -- 172.21.15.5:0/4191257165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d48d0 msgr2=0x55a2393dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee9c8f640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d48d0 0x55a2393dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee9c8f640 1 -- 172.21.15.5:0/4191257165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 msgr2=0x55a2393e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee9c8f640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee9c8f640 1 -- 172.21.15.5:0/4191257165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2393d4220 con 0x55a2393dd0e0 2024-04-22T13:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee8c8d640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d48d0 0x55a2393dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee948e640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efee9c8f640 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 0x55a2393dca90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7efedc00cb30 tx=0x7efedc02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2cb7d7dd02629e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efeceffd640 1 -- 172.21.15.5:0/4191257165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00ac90 con 0x55a2393dd0e0 2024-04-22T13:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efeceffd640 1 -- 172.21.15.5:0/4191257165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efedc00fd30 con 0x55a2393dd0e0 2024-04-22T13:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.981+0000 7efeceffd640 1 -- 172.21.15.5:0/4191257165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc020660 con 0x55a2393dd0e0 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 -- 172.21.15.5:0/4191257165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 msgr2=0x55a2393dca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 0x55a2393dca90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7efedc00cb30 tx=0x7efedc02a040 comp rx=0 tx=0).stop 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 -- 172.21.15.5:0/4191257165 shutdown_connections 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393dd0e0 0x55a2393dca90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d48d0 0x55a2393dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 --2- 172.21.15.5:0/4191257165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393d3b60 0x55a2393e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 -- 172.21.15.5:0/4191257165 >> 172.21.15.5:0/4191257165 conn(0x55a23939ac90 msgr2=0x55a2393ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 -- 172.21.15.5:0/4191257165 shutdown_connections 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 -- 172.21.15.5:0/4191257165 wait complete. 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.982+0000 7efeee3c3740 1 -- start start 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 Processor -- start 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d3b60 0x55a2393e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393dd0e0 0x55a239452630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee41af030 con 0x55a2393d48d0 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efee41ae960 con 0x55a2393dd0e0 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efee41abba0 con 0x55a2393d3b60 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efeee3c3740 10 client.0 _ll_register_callbacks cb 0x7efee00df0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee8c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee8c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45162/0 (socket says 172.21.15.5:45162) 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee8c8d640 1 -- 172.21.15.5:0/3258213660 learned_addr learned my addr 172.21.15.5:0/3258213660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee948e640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d3b60 0x55a2393e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee9c8f640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393dd0e0 0x55a239452630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee8c8d640 1 -- 172.21.15.5:0/3258213660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d3b60 msgr2=0x55a2393e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee8c8d640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d3b60 0x55a2393e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.983+0000 7efee8c8d640 1 -- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393dd0e0 msgr2=0x55a239452630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efee8c8d640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393dd0e0 0x55a239452630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efee8c8d640 1 -- 172.21.15.5:0/3258213660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a239452b70 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efee9c8f640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393dd0e0 0x55a239452630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efee8c8d640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393e1530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7efee41008e0 tx=0x7efed400de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edb2e0e53ed7f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeccff9640 1 -- 172.21.15.5:0/3258213660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed401d380 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeccff9640 1 -- 172.21.15.5:0/3258213660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efed401d520 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efedc00fd30 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeccff9640 1 -- 172.21.15.5:0/3258213660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efed401d720 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeee3c3740 10 client.16800 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a239453db0 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeccff9640 1 -- 172.21.15.5:0/3258213660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efedc00fd30 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.984+0000 7efeccff9640 1 -- 172.21.15.5:0/3258213660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efed4011440 con 0x55a2393d48d0 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeee3c3740 10 client.16800 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeee3c3740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeee3c3740 20 client.16800 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a239454510 0x55a239456930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeee3c3740 4 client.16800 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2392d83d0 con 0x55a239454510 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efee948e640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a239454510 0x55a239456930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efeccff9640 10 client.16800 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.985+0000 7efee948e640 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a239454510 0x55a239456930 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554e4c4539aafab1 server_cookie=40117390732f5ff0 in_seq=0 out_seq=0 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeccff9640 1 -- 172.21.15.5:0/3258213660 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2392d83d0 con 0x55a239454510 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeccff9640 10 client.16800 handle_command_reply: tid=0 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 client.16800 shutdown 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 msgr2=0x55a2393e1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393e1530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7efee41008e0 tx=0x7efed400de30 comp rx=0 tx=0).stop 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 shutdown_connections 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a239454510 0x55a239456930 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2393d48d0 0x55a2393e1530 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a2393dd0e0 0x55a239452630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeccff9640 0 client.16800 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 --2- 172.21.15.5:0/3258213660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2393d3b60 0x55a2393e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 >> 172.21.15.5:0/3258213660 conn(0x55a23939ac90 msgr2=0x55a2393cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 shutdown_connections 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 1 -- 172.21.15.5:0/3258213660 wait complete. 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.994+0000 7efeee3c3740 20 client.16800 trim_cache size 0 max 16384 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:29:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efed00a3d10 msgr2=0x7efed00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efed00a3d10 0x7efed00a61d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efedc011cb0 tx=0x7efedc016000 comp rx=0 tx=0).stop 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 msgr2=0x7efee419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee419e410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efee0005940 tx=0x7efee002d890 comp rx=0 tx=0).stop 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 shutdown_connections 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efed00a3d10 0x7efed00a61d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efee410c590 0x7efee41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efee4103ae0 0x7efee419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 --2- 172.21.15.5:0/1620120730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee41031c0 0x7efee419e410 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 >> 172.21.15.5:0/1620120730 conn(0x7efee4077fa0 msgr2=0x7efee40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 shutdown_connections 2024-04-22T13:29:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:44.995+0000 7efeeb6f2640 1 -- 172.21.15.5:0/1620120730 wait complete. 2024-04-22T13:29:45.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:44 smithi049 ceph-mon[31379]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 517 KiB/s wr, 10 op/s 2024-04-22T13:29:45.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:44 smithi107 ceph-mon[31859]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 517 KiB/s wr, 10 op/s 2024-04-22T13:29:45.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:44 smithi005 ceph-mon[28006]: pgmap v81: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 517 KiB/s wr, 10 op/s 2024-04-22T13:29:47.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:46 smithi049 ceph-mon[31379]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 949 KiB/s wr, 17 op/s 2024-04-22T13:29:47.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:46 smithi107 ceph-mon[31859]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 949 KiB/s wr, 17 op/s 2024-04-22T13:29:47.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:46 smithi005 ceph-mon[28006]: pgmap v82: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 949 KiB/s wr, 17 op/s 2024-04-22T13:29:49.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:48 smithi049 ceph-mon[31379]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:49.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:48 smithi107 ceph-mon[31859]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:48 smithi005 ceph-mon[28006]: pgmap v83: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:51.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:50 smithi049 ceph-mon[31379]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:51.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:50 smithi107 ceph-mon[31859]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:51.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:50 smithi005 ceph-mon[28006]: pgmap v84: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:53.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:52 smithi049 ceph-mon[31379]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-04-22T13:29:53.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:52 smithi107 ceph-mon[31859]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-04-22T13:29:53.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:52 smithi005 ceph-mon[28006]: pgmap v85: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-04-22T13:29:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:54 smithi049 ceph-mon[31379]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:55.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:54 smithi107 ceph-mon[31859]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:55.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:54 smithi005 ceph-mon[28006]: pgmap v86: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 863 KiB/s wr, 13 op/s 2024-04-22T13:29:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:56 smithi049 ceph-mon[31379]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-04-22T13:29:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:56 smithi107 ceph-mon[31859]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-04-22T13:29:57.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:56 smithi005 ceph-mon[28006]: pgmap v87: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-04-22T13:29:57.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:57.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 -- 172.21.15.5:0/913531602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c069f20 msgr2=0x7fd46c106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 --2- 172.21.15.5:0/913531602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c069f20 0x7fd46c106b50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009ca0 tx=0x7fd45c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 -- 172.21.15.5:0/913531602 shutdown_connections 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 --2- 172.21.15.5:0/913531602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c069f20 0x7fd46c106b50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 --2- 172.21.15.5:0/913531602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd46c107230 0x7fd46c10ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 --2- 172.21.15.5:0/913531602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd46c069570 0x7fd46c069950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 -- 172.21.15.5:0/913531602 >> 172.21.15.5:0/913531602 conn(0x7fd46c077fa0 msgr2=0x7fd46c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 -- 172.21.15.5:0/913531602 shutdown_connections 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 -- 172.21.15.5:0/913531602 wait complete. 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.810+0000 7fd472f8f640 1 Processor -- start 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 -- start start 2024-04-22T13:29:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd46c069570 0x7fd46c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 0x7fd46c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd46c19cff0 0x7fd46c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c112150 con 0x7fd46c107230 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c111fd0 con 0x7fd46c069570 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd472f8f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1122d0 con 0x7fd46c19cff0 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 0x7fd46c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 0x7fd46c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33570/0 (socket says 172.21.15.5:33570) 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 -- 172.21.15.5:0/1378009493 learned_addr learned my addr 172.21.15.5:0/1378009493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd47152c640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd46c19cff0 0x7fd46c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd470d2b640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd46c069570 0x7fd46c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 -- 172.21.15.5:0/1378009493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd46c19cff0 msgr2=0x7fd46c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd46c19cff0 0x7fd46c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 -- 172.21.15.5:0/1378009493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd46c069570 msgr2=0x7fd46c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd46c069570 0x7fd46c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 -- 172.21.15.5:0/1378009493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c19dae0 con 0x7fd46c107230 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.811+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 0x7fd46c198c70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd45c004af0 tx=0x7fd45c0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.812+0000 7fd469ffb640 1 -- 172.21.15.5:0/1378009493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c039040 con 0x7fd46c107230 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.812+0000 7fd469ffb640 1 -- 172.21.15.5:0/1378009493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c044070 con 0x7fd46c107230 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.812+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c19dd70 con 0x7fd46c107230 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.812+0000 7fd469ffb640 1 -- 172.21.15.5:0/1378009493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0042a0 con 0x7fd46c107230 2024-04-22T13:29:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.812+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c1a5530 con 0x7fd46c107230 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.814+0000 7fd469ffb640 1 -- 172.21.15.5:0/1378009493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd45c004440 con 0x7fd46c107230 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.814+0000 7fd475b7c740 1 Processor -- start 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 -- start start 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 0x561c10370960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10379540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c102d51e0 con 0x561c10370ea0 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c10272390 con 0x561c10379a80 2024-04-22T13:29:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd475b7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c102d11a0 con 0x561c10370580 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd470d2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10379540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 0x561c10370960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd470d2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10379540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33572/0 (socket says 172.21.15.5:33572) 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd470d2b640 1 -- 172.21.15.5:0/4260966779 learned_addr learned my addr 172.21.15.5:0/4260966779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 0x561c10370960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:35594/0 (socket says 172.21.15.5:35594) 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd46bfff640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd469ffb640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4580a3d20 0x7fd4580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 -- 172.21.15.5:0/4260966779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 msgr2=0x561c1037be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 -- 172.21.15.5:0/4260966779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 msgr2=0x561c10379540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10379540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd47152c640 1 -- 172.21.15.5:0/4260966779 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c100b17d0 con 0x561c10370580 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd470d2b640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10379540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.815+0000 7fd46bfff640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd469ffb640 1 -- 172.21.15.5:0/1378009493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fd45c0f28d0 con 0x7fd46c107230 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd47152c640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4580a3d20 0x7fd4580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd47152c640 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 0x561c10370960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd4640057b0 tx=0x7fd46400bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff43f38d0f49a761 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd47152c640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4580a3d20 0x7fd4580a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd464021000 tx=0x7fd46401e000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd462ffd640 1 -- 172.21.15.5:0/4260966779 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464024070 con 0x561c10370580 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd462ffd640 1 -- 172.21.15.5:0/4260966779 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd46401e930 con 0x561c10370580 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.816+0000 7fd462ffd640 1 -- 172.21.15.5:0/4260966779 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464011730 con 0x561c10370580 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 -- 172.21.15.5:0/4260966779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 msgr2=0x561c10370960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 0x561c10370960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd4640057b0 tx=0x7fd46400bb10 comp rx=0 tx=0).stop 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 -- 172.21.15.5:0/4260966779 shutdown_connections 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10379540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 --2- 172.21.15.5:0/4260966779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370580 0x561c10370960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 -- 172.21.15.5:0/4260966779 >> 172.21.15.5:0/4260966779 conn(0x561c1036acb0 msgr2=0x561c1036b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 -- 172.21.15.5:0/4260966779 shutdown_connections 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 -- 172.21.15.5:0/4260966779 wait complete. 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 -- start start 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.817+0000 7fd475b7c740 1 Processor -- start 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10376b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c10377050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 0x561c10377590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c1037c870 con 0x561c10379a80 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561c1028e2f0 con 0x561c10370580 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c1037c9f0 con 0x561c10370ea0 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd475b7c740 10 client.0 _ll_register_callbacks cb 0x561c1029b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd47152c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 0x561c10377590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c10377050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd47152c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 0x561c10377590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33584/0 (socket says 172.21.15.5:33584) 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd47152c640 1 -- 172.21.15.5:0/3477615027 learned_addr learned my addr 172.21.15.5:0/3477615027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd470d2b640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10376b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd46bfff640 1 -- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 msgr2=0x561c10376b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd46bfff640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10376b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd46bfff640 1 -- 172.21.15.5:0/3477615027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 msgr2=0x561c10377590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd46bfff640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 0x561c10377590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd46bfff640 1 -- 172.21.15.5:0/3477615027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0ffe73f0 con 0x561c10370ea0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.818+0000 7fd47152c640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 0x561c10377590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd46bfff640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c10377050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0f0160 tx=0x7fd45c0099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231c32e094303db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd44bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd44bfff640 1 -- 172.21.15.5:0/3477615027 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0be580 con 0x561c10370ea0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd44bfff640 1 -- 172.21.15.5:0/3477615027 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fd45c0bb0b0 con 0x561c10370ea0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46401e930 con 0x561c10370ea0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd44bfff640 1 -- 172.21.15.5:0/3477615027 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd45c0be3a0 con 0x561c10370ea0 2024-04-22T13:29:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 10 client.26293 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c0ffa70f0 con 0x561c10370ea0 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd44bfff640 1 -- 172.21.15.5:0/3477615027 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd46401e930 con 0x561c10370ea0 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd44bfff640 1 -- 172.21.15.5:0/3477615027 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd45c03c030 con 0x561c10370ea0 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 10 client.26293 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 10 client.26293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 20 client.26293 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c10389b10 0x561c1038bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 4 client.26293 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.819+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c102e2c00 con 0x561c10389b10 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.820+0000 7fd470d2b640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c10389b10 0x561c1038bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.820+0000 7fd470d2b640 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c10389b10 0x561c1038bf50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89474bd32a92fbe server_cookie=163814e69ee85692 in_seq=0 out_seq=0 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.820+0000 7fd44bfff640 10 client.26293 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd44bfff640 1 -- 172.21.15.5:0/3477615027 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c102e2c00 con 0x561c10389b10 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd44bfff640 10 client.26293 handle_command_reply: tid=0 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 client.26293 shutdown 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 msgr2=0x561c10377050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c10377050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0f0160 tx=0x7fd45c0099c0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 shutdown_connections 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c10389b10 0x561c1038bf50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10379a80 0x561c10377590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c10377050 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 --2- 172.21.15.5:0/3477615027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10376b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd44bfff640 0 client.26293 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 >> 172.21.15.5:0/3477615027 conn(0x561c1036acb0 msgr2=0x561c1036ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 shutdown_connections 2024-04-22T13:29:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 1 -- 172.21.15.5:0/3477615027 wait complete. 2024-04-22T13:29:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.821+0000 7fd475b7c740 20 client.26293 trim_cache size 0 max 16384 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 Processor -- start 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 -- start start 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10379240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10385730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10379a80 0x561c10372260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1ada80 con 0x561c10370ea0 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1af5c0 con 0x561c10370580 2024-04-22T13:29:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd475b7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1ae150 con 0x561c10379a80 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.839+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10385730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10385730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33596/0 (socket says 172.21.15.5:33596) 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 -- 172.21.15.5:0/790595391 learned_addr learned my addr 172.21.15.5:0/790595391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd47152c640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10379a80 0x561c10372260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd470d2b640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10379240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 -- 172.21.15.5:0/790595391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10379a80 msgr2=0x561c10372260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10379a80 0x561c10372260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 -- 172.21.15.5:0/790595391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 msgr2=0x561c10379240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10379240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 -- 172.21.15.5:0/790595391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1038c5d0 con 0x561c10370ea0 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd47152c640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10379a80 0x561c10372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd470d2b640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10379240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd46bfff640 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10385730 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0baea0 tx=0x7fd45c0bcf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d454d39bc8d16962 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd4627fc640 1 -- 172.21.15.5:0/790595391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0be580 con 0x561c10370ea0 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd4627fc640 1 -- 172.21.15.5:0/790595391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c035e50 con 0x561c10370ea0 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.840+0000 7fd4627fc640 1 -- 172.21.15.5:0/790595391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0bea40 con 0x561c10370ea0 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 -- 172.21.15.5:0/790595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 msgr2=0x561c10385730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10385730 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0baea0 tx=0x7fd45c0bcf50 comp rx=0 tx=0).stop 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 -- 172.21.15.5:0/790595391 shutdown_connections 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370ea0 0x561c10385730 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10379a80 0x561c10372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 --2- 172.21.15.5:0/790595391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10370580 0x561c10379240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 -- 172.21.15.5:0/790595391 >> 172.21.15.5:0/790595391 conn(0x561c1036acb0 msgr2=0x561c103694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 -- 172.21.15.5:0/790595391 shutdown_connections 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 -- 172.21.15.5:0/790595391 wait complete. 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.841+0000 7fd475b7c740 1 -- start start 2024-04-22T13:29:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 Processor -- start 2024-04-22T13:29:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 0x561c10382150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c1037d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1aeef0 con 0x561c10370580 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1ae820 con 0x561c10379a80 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1aba80 con 0x561c10370ea0 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd475b7c740 10 client.0 _ll_register_callbacks cb 0x7fd45c003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 0x561c10382150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 0x561c10382150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33606/0 (socket says 172.21.15.5:33606) 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 -- 172.21.15.5:0/1307455359 learned_addr learned my addr 172.21.15.5:0/1307455359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c1037d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd47152c640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 -- 172.21.15.5:0/1307455359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 msgr2=0x561c1037d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c1037d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 -- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 msgr2=0x561c1037dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd470d2b640 1 -- 172.21.15.5:0/1307455359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1037be70 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c1037d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.842+0000 7fd47152c640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd470d2b640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 0x561c10382150 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x561c1038c7a0 tx=0x7fd4540039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd7116564156adb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd44bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd44bfff640 1 -- 172.21.15.5:0/1307455359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454020160 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd44bfff640 1 -- 172.21.15.5:0/1307455359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fd454020300 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45c035e50 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd44bfff640 1 -- 172.21.15.5:0/1307455359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd45402f050 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 10 client.16818 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c103f00b0 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd44bfff640 1 -- 172.21.15.5:0/1307455359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c035e50 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd44bfff640 1 -- 172.21.15.5:0/1307455359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd454011bc0 con 0x561c10370580 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 10 client.16818 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 20 client.16818 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c103f0880 0x561c103f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 4 client.16818 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.843+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c102dca70 con 0x561c103f0880 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c103f0880 0x561c103f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd44bfff640 10 client.16818 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd46bfff640 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c103f0880 0x561c103f2ca0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec05cf2d32eac38 server_cookie=257fddaa4564bae7 in_seq=0 out_seq=0 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd44bfff640 1 -- 172.21.15.5:0/1307455359 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561c102dca70 con 0x561c103f0880 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd44bfff640 10 client.16818 handle_command_reply: tid=0 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 client.16818 shutdown 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 msgr2=0x561c10382150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 0x561c10382150 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x561c1038c7a0 tx=0x7fd4540039c0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 shutdown_connections 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561c103f0880 0x561c103f2ca0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561c10379a80 0x561c1037dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c10370ea0 0x561c1037d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd44bfff640 0 client.16818 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 --2- 172.21.15.5:0/1307455359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c10370580 0x561c10382150 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 >> 172.21.15.5:0/1307455359 conn(0x561c1036acb0 msgr2=0x561c10377360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.844+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 shutdown_connections 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd475b7c740 1 -- 172.21.15.5:0/1307455359 wait complete. 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd475b7c740 20 client.16818 trim_cache size 0 max 16384 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:29:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:29:57.848 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:29:57.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4580a3d20 msgr2=0x7fd4580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4580a3d20 0x7fd4580a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd464021000 tx=0x7fd46401e000 comp rx=0 tx=0).stop 2024-04-22T13:29:57.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 msgr2=0x7fd46c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 0x7fd46c198c70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd45c004af0 tx=0x7fd45c0346f0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 shutdown_connections 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd46c107230 0x7fd46c198c70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4580a3d20 0x7fd4580a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd46c19cff0 0x7fd46c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 --2- 172.21.15.5:0/1378009493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd46c069570 0x7fd46c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 >> 172.21.15.5:0/1378009493 conn(0x7fd46c077fa0 msgr2=0x7fd46c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 shutdown_connections 2024-04-22T13:29:57.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.845+0000 7fd472f8f640 1 -- 172.21.15.5:0/1378009493 wait complete. 2024-04-22T13:29:57.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:29:57.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 -- 172.21.15.5:0/3061573637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8101a40 msgr2=0x7fdcf810dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 --2- 172.21.15.5:0/3061573637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8101a40 0x7fdcf810dca0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009ca0 tx=0x7fdcf002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 -- 172.21.15.5:0/3061573637 shutdown_connections 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 --2- 172.21.15.5:0/3061573637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8101a40 0x7fdcf810dca0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 --2- 172.21.15.5:0/3061573637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf81010a0 0x7fdcf8101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 --2- 172.21.15.5:0/3061573637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8105dd0 0x7fdcf81061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 -- 172.21.15.5:0/3061573637 >> 172.21.15.5:0/3061573637 conn(0x7fdcf8077c40 msgr2=0x7fdcf8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 -- 172.21.15.5:0/3061573637 shutdown_connections 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.965+0000 7fdcffab2640 1 -- 172.21.15.5:0/3061573637 wait complete. 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 Processor -- start 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 -- start start 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 0x7fdcf8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf8101a40 0x7fdcf8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8105dd0 0x7fdcf819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf8111180 con 0x7fdcf81010a0 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf8111000 con 0x7fdcf8105dd0 2024-04-22T13:29:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcffab2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf8111300 con 0x7fdcf8101a40 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcfd84e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 0x7fdcf8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcfd84e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 0x7fdcf8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33620/0 (socket says 172.21.15.5:33620) 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcfd84e640 1 -- 172.21.15.5:0/161329434 learned_addr learned my addr 172.21.15.5:0/161329434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.966+0000 7fdcfe04f640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8105dd0 0x7fdcf819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd04d640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf8101a40 0x7fdcf8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd84e640 1 -- 172.21.15.5:0/161329434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf8101a40 msgr2=0x7fdcf8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd84e640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf8101a40 0x7fdcf8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd84e640 1 -- 172.21.15.5:0/161329434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8105dd0 msgr2=0x7fdcf819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd84e640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8105dd0 0x7fdcf819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd84e640 1 -- 172.21.15.5:0/161329434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf819d6f0 con 0x7fdcf81010a0 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd04d640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf8101a40 0x7fdcf8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcfd84e640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 0x7fdcf8198750 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d950 tx=0x7fdce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8017070 con 0x7fdcf81010a0 2024-04-22T13:29:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce8009c30 con 0x7fdcf81010a0 2024-04-22T13:29:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf819d9e0 con 0x7fdcf81010a0 2024-04-22T13:29:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce800f620 con 0x7fdcf81010a0 2024-04-22T13:29:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.967+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf81a5270 con 0x7fdcf81010a0 2024-04-22T13:29:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.968+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf8102540 con 0x7fdcf81010a0 2024-04-22T13:29:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.969+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fdce8010460 con 0x7fdcf81010a0 2024-04-22T13:29:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.970+0000 7fdceeffd640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdce40a3d20 0x7fdce40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.970+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fdce80d0430 con 0x7fdcf81010a0 2024-04-22T13:29:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.970+0000 7fdcfd04d640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdce40a3d20 0x7fdce40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.971+0000 7fdcfd04d640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdce40a3d20 0x7fdce40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8104ff0 tx=0x7fdcf4005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:57.975+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdce809cb00 con 0x7fdcf81010a0 2024-04-22T13:29:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 -- 172.21.15.5:0/4277751885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 msgr2=0x7f1b04101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 --2- 172.21.15.5:0/4277751885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b04101390 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009930 tx=0x7f1af402bd10 comp rx=0 tx=0).stop 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 -- 172.21.15.5:0/4277751885 shutdown_connections 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 --2- 172.21.15.5:0/4277751885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b041063b0 0x7f1b04106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 --2- 172.21.15.5:0/4277751885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041018d0 0x7f1b04105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 --2- 172.21.15.5:0/4277751885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b04101390 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.088+0000 7f1b0909a640 1 -- 172.21.15.5:0/4277751885 >> 172.21.15.5:0/4277751885 conn(0x7f1b04077fa0 msgr2=0x7f1b040783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.089+0000 7f1b0909a640 1 -- 172.21.15.5:0/4277751885 shutdown_connections 2024-04-22T13:29:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.089+0000 7f1b0909a640 1 -- 172.21.15.5:0/4277751885 wait complete. 2024-04-22T13:29:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.089+0000 7f1b0909a640 1 Processor -- start 2024-04-22T13:29:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 -- start start 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b041a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b041018d0 0x7f1b041a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041063b0 0x7f1b0419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b04118830 con 0x7f1b04100fb0 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041186b0 con 0x7f1b041018d0 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0909a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041189b0 con 0x7f1b041063b0 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b02d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b041a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b02d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b041a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33636/0 (socket says 172.21.15.5:33636) 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041063b0 0x7f1b0419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b02d9d640 1 -- 172.21.15.5:0/2724237337 learned_addr learned my addr 172.21.15.5:0/2724237337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0259c640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b041018d0 0x7f1b041a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0359e640 1 -- 172.21.15.5:0/2724237337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b041018d0 msgr2=0x7f1b041a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0359e640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b041018d0 0x7f1b041a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0359e640 1 -- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 msgr2=0x7f1b041a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0359e640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b041a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b0359e640 1 -- 172.21.15.5:0/2724237337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0419f2d0 con 0x7f1b041063b0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.090+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b041a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.091+0000 7f1b0359e640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041063b0 0x7f1b0419eb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00cab0 tx=0x7f1afc00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.091+0000 7f1aebfff640 1 -- 172.21.15.5:0/2724237337 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00ae20 con 0x7f1b041063b0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.091+0000 7f1aebfff640 1 -- 172.21.15.5:0/2724237337 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1afc00f900 con 0x7f1b041063b0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.091+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0419f5c0 con 0x7f1b041063b0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.091+0000 7f1aebfff640 1 -- 172.21.15.5:0/2724237337 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0189f0 con 0x7f1b041063b0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.091+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b041ab810 con 0x7f1b041063b0 2024-04-22T13:29:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.093+0000 7f1b0bd6a740 1 Processor -- start 2024-04-22T13:29:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.094+0000 7f1aebfff640 1 -- 172.21.15.5:0/2724237337 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1afc00f450 con 0x7f1b041063b0 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 -- start start 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640386988d0 0x564038697620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564038697b60 0x564038697fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640385fc8d0 con 0x5640386988d0 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564038599e80 con 0x5640386a10e0 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5640385f8890 con 0x564038697b60 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640386988d0 0x564038697620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b02d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564038697b60 0x564038697fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b02d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564038697b60 0x564038697fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:35652/0 (socket says 172.21.15.5:35652) 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b02d9d640 1 -- 172.21.15.5:0/260481165 learned_addr learned my addr 172.21.15.5:0/260481165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1aebfff640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1aec0a3d10 0x7f1aec0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0259c640 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0359e640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1aec0a3d10 0x7f1aec0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0259c640 1 -- 172.21.15.5:0/260481165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564038697b60 msgr2=0x564038697fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0259c640 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564038697b60 0x564038697fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0259c640 1 -- 172.21.15.5:0/260481165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 msgr2=0x5640386a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.095+0000 7f1b0259c640 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1b0259c640 1 -- 172.21.15.5:0/260481165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640385bb320 con 0x5640386988d0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1aebfff640 1 -- 172.21.15.5:0/2724237337 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1afc0d0ea0 con 0x7f1b041063b0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1b0259c640 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640386988d0 0x564038697620 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1b04078680 tx=0x7f1af80041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5639b6d7f540c17b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1ae97fa640 1 -- 172.21.15.5:0/260481165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8003d80 con 0x5640386988d0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1ae97fa640 1 -- 172.21.15.5:0/260481165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af8012930 con 0x5640386988d0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1ae97fa640 1 -- 172.21.15.5:0/260481165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af80108a0 con 0x5640386988d0 2024-04-22T13:29:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.096+0000 7f1b0359e640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1aec0a3d10 0x7f1aec0a61d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1afc09cb10 tx=0x7f1afc0922a0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/260481165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640386988d0 msgr2=0x564038697620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640386988d0 0x564038697620 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1b04078680 tx=0x7f1af80041f0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/260481165 shutdown_connections 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564038697b60 0x564038697fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/260481165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640386988d0 0x564038697620 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/260481165 >> 172.21.15.5:0/260481165 conn(0x56403865ec90 msgr2=0x564038692850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/260481165 shutdown_connections 2024-04-22T13:29:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.097+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/260481165 wait complete. 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 -- start start 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 Processor -- start 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x56403869e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640386a3db0 con 0x564038697b60 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564038584eb0 con 0x5640386988d0 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5640386a3f30 con 0x5640386a10e0 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0bd6a740 10 client.0 _ll_register_callbacks cb 0x5640385b1b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33654/0 (socket says 172.21.15.5:33654) 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b0259c640 1 -- 172.21.15.5:0/202619309 learned_addr learned my addr 172.21.15.5:0/202619309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.098+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x56403869e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 -- 172.21.15.5:0/202619309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 msgr2=0x56403869e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x56403869e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 -- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 msgr2=0x56403869dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 -- 172.21.15.5:0/202619309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403830f210 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0259c640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386a0970 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1af8003b80 tx=0x7f1af800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c9bc6011b85c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1ae6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/202619309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8012ce0 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/202619309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1af8005020 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af8012930 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/202619309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1af8005220 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0bd6a740 10 client.16836 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640383db620 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.099+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/202619309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af8012930 con 0x564038697b60 2024-04-22T13:29:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/202619309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1af801d070 con 0x564038697b60 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b0bd6a740 10 client.16836 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b0bd6a740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b0bd6a740 20 client.16836 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5640386b0f50 0x5640386b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b0bd6a740 4 client.16836 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564038610880 con 0x5640386b0f50 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5640386b0f50 0x5640386b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5640386b0f50 0x5640386b3340 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c50d2958b20c501 server_cookie=23e47fa4cd5ebb2a in_seq=0 out_seq=0 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.100+0000 7f1ae6ff5640 10 client.16836 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/202619309 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564038610880 con 0x5640386b0f50 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1ae6ff5640 10 client.16836 handle_command_reply: tid=0 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 client.16836 shutdown 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 msgr2=0x5640386a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386a0970 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1af8003b80 tx=0x7f1af800b710 comp rx=0 tx=0).stop 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 shutdown_connections 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5640386b0f50 0x5640386b3340 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x56403869e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/202619309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386a0970 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.101+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 >> 172.21.15.5:0/202619309 conn(0x56403865ec90 msgr2=0x5640386945a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.102+0000 7f1ae6ff5640 0 client.16836 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.102+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 shutdown_connections 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.102+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/202619309 wait complete. 2024-04-22T13:29:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.102+0000 7f1b0bd6a740 20 client.16836 trim_cache size 0 max 16384 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 Processor -- start 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 -- start start 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386acce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x564038694360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041b40d0 con 0x564038697b60 2024-04-22T13:29:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041b5c10 con 0x5640386988d0 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041b47a0 con 0x5640386a10e0 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386acce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33668/0 (socket says 172.21.15.5:33668) 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0259c640 1 -- 172.21.15.5:0/2082040039 learned_addr learned my addr 172.21.15.5:0/2082040039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.117+0000 7f1b0259c640 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x564038694360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b0259c640 1 -- 172.21.15.5:0/2082040039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 msgr2=0x564038694360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b0259c640 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x564038694360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b0259c640 1 -- 172.21.15.5:0/2082040039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 msgr2=0x56403869d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b0259c640 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b0259c640 1 -- 172.21.15.5:0/2082040039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640386a3f50 con 0x564038697b60 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1b0259c640 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386acce0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1af8006010 tx=0x7f1af80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a66413774ee9d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1ae8ff9640 1 -- 172.21.15.5:0/2082040039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af800b390 con 0x564038697b60 2024-04-22T13:29:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1ae8ff9640 1 -- 172.21.15.5:0/2082040039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af8005000 con 0x564038697b60 2024-04-22T13:29:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.118+0000 7f1ae8ff9640 1 -- 172.21.15.5:0/2082040039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8012930 con 0x564038697b60 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/2082040039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 msgr2=0x5640386acce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386acce0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1af8006010 tx=0x7f1af80039c0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/2082040039 shutdown_connections 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386a10e0 0x564038694360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386988d0 0x56403869d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/2082040039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x5640386acce0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/2082040039 >> 172.21.15.5:0/2082040039 conn(0x56403865ec90 msgr2=0x564038693c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/2082040039 shutdown_connections 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/2082040039 wait complete. 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 -- start start 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.119+0000 7f1b0bd6a740 1 Processor -- start 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x56403870f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386988d0 0x56403870fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041b5540 con 0x564038697b60 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041b4e70 con 0x5640386a10e0 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041b20b0 con 0x5640386988d0 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0bd6a740 10 client.0 _ll_register_callbacks cb 0x7f1af800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x56403870f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x56403870f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33678/0 (socket says 172.21.15.5:33678) 2024-04-22T13:29:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 -- 172.21.15.5:0/1649367306 learned_addr learned my addr 172.21.15.5:0/1649367306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386988d0 0x56403870fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 -- 172.21.15.5:0/1649367306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386988d0 msgr2=0x56403870fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386988d0 0x56403870fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 -- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 msgr2=0x5640386a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 -- 172.21.15.5:0/1649367306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564038696e30 con 0x564038697b60 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386988d0 0x56403870fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1b0259c640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x56403870f5d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b800 tx=0x7f1af800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0846be7a9825fda server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1ae6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.120+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/1649367306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af80053c0 con 0x564038697b60 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/1649367306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1af8010b20 con 0x564038697b60 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af8005000 con 0x564038697b60 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/1649367306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1af8012930 con 0x564038697b60 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 10 client.16842 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564038716b50 con 0x564038697b60 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/1649367306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af8005000 con 0x564038697b60 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/1649367306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1af801d070 con 0x564038697b60 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 10 client.16842 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 20 client.16842 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564038716fd0 0x5640387193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 4 client.16842 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56403859c3d0 con 0x564038716fd0 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.121+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564038716fd0 0x5640387193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.122+0000 7f1ae6ff5640 10 client.16842 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.122+0000 7f1b02d9d640 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564038716fd0 0x5640387193c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0de2926af52863 server_cookie=9ce0c6275e61ea61 in_seq=0 out_seq=0 2024-04-22T13:29:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.130+0000 7f1ae6ff5640 1 -- 172.21.15.5:0/1649367306 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56403859c3d0 con 0x564038716fd0 2024-04-22T13:29:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.130+0000 7f1ae6ff5640 10 client.16842 handle_command_reply: tid=0 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 client.16842 shutdown 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 msgr2=0x56403870f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x56403870f5d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b800 tx=0x7f1af800b830 comp rx=0 tx=0).stop 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 shutdown_connections 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564038716fd0 0x5640387193c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1ae6ff5640 0 client.16842 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640386a10e0 0x5640386a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640386988d0 0x56403870fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 --2- 172.21.15.5:0/1649367306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564038697b60 0x56403870f5d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 >> 172.21.15.5:0/1649367306 conn(0x56403865ec90 msgr2=0x5640386918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 shutdown_connections 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 1 -- 172.21.15.5:0/1649367306 wait complete. 2024-04-22T13:29:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.131+0000 7f1b0bd6a740 20 client.16842 trim_cache size 0 max 16384 2024-04-22T13:29:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-04-22T13:29:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-04-22T13:29:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-04-22T13:29:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1aec0a3d10 msgr2=0x7f1aec0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1aec0a3d10 0x7f1aec0a61d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1afc09cb10 tx=0x7f1afc0922a0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041063b0 msgr2=0x7f1b0419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041063b0 0x7f1b0419eb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00cab0 tx=0x7f1afc00cf70 comp rx=0 tx=0).stop 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 shutdown_connections 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1aec0a3d10 0x7f1aec0a61d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b041063b0 0x7f1b0419eb50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b041018d0 0x7f1b041a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 --2- 172.21.15.5:0/2724237337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b04100fb0 0x7f1b041a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 >> 172.21.15.5:0/2724237337 conn(0x7f1b04077fa0 msgr2=0x7f1b04103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 shutdown_connections 2024-04-22T13:29:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-04-22T13:29:58.132+0000 7f1b0909a640 1 -- 172.21.15.5:0/2724237337 wait complete. 2024-04-22T13:29:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.213+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcf810a370 con 0x7fdcf81010a0 2024-04-22T13:29:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.214+0000 7fdceeffd640 1 -- 172.21.15.5:0/161329434 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7fdce8009de0 con 0x7fdcf81010a0 2024-04-22T13:29:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:29:58.211 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:29:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:29:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdce40a3d20 msgr2=0x7fdce40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdce40a3d20 0x7fdce40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8104ff0 tx=0x7fdcf4005f90 comp rx=0 tx=0).stop 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 msgr2=0x7fdcf8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 0x7fdcf8198750 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d950 tx=0x7fdce800de10 comp rx=0 tx=0).stop 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 shutdown_connections 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdce40a3d20 0x7fdce40a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8105dd0 0x7fdcf819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcf8101a40 0x7fdcf8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 --2- 172.21.15.5:0/161329434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81010a0 0x7fdcf8198750 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 >> 172.21.15.5:0/161329434 conn(0x7fdcf8077c40 msgr2=0x7fdcf806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 shutdown_connections 2024-04-22T13:29:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.217+0000 7fdcffab2640 1 -- 172.21.15.5:0/161329434 wait complete. 2024-04-22T13:29:58.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:29:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.339+0000 7ff8b5385640 1 -- 172.21.15.5:0/4000717311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101a40 msgr2=0x7ff8b0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.339+0000 7ff8b5385640 1 --2- 172.21.15.5:0/4000717311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101a40 0x7ff8b0105a80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 -- 172.21.15.5:0/4000717311 shutdown_connections 2024-04-22T13:29:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 --2- 172.21.15.5:0/4000717311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101a40 0x7ff8b0105a80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 --2- 172.21.15.5:0/4000717311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b0106160 0x7ff8b010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 --2- 172.21.15.5:0/4000717311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8b0101090 0x7ff8b0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 -- 172.21.15.5:0/4000717311 >> 172.21.15.5:0/4000717311 conn(0x7ff8b0078000 msgr2=0x7ff8b0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 -- 172.21.15.5:0/4000717311 shutdown_connections 2024-04-22T13:29:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 -- 172.21.15.5:0/4000717311 wait complete. 2024-04-22T13:29:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 Processor -- start 2024-04-22T13:29:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.340+0000 7ff8b5385640 1 -- start start 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8b5385640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 0x7ff8b01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8b5385640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8b0106160 0x7ff8b01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8b5385640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b019a7b0 0x7ff8b019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8b5385640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01110d0 con 0x7ff8b0101090 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8b5385640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0110f50 con 0x7ff8b0106160 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8b5385640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0111250 con 0x7ff8b019a7b0 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 0x7ff8b01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 0x7ff8b01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33690/0 (socket says 172.21.15.5:33690) 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3269728517 learned_addr learned my addr 172.21.15.5:0/3269728517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b019a7b0 0x7ff8b019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8b0106160 0x7ff8b01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3269728517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b019a7b0 msgr2=0x7ff8b019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b019a7b0 0x7ff8b019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3269728517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8b0106160 msgr2=0x7ff8b01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8b0106160 0x7ff8b01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3269728517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b019b340 con 0x7ff8b0101090 2024-04-22T13:29:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 0x7ff8b01a0640 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000d950 tx=0x7ff8a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.341+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3269728517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0017030 con 0x7ff8b0101090 2024-04-22T13:29:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.342+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3269728517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a0004020 con 0x7ff8b0101090 2024-04-22T13:29:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.342+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b019b630 con 0x7ff8b0101090 2024-04-22T13:29:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.342+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3269728517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00047a0 con 0x7ff8b0101090 2024-04-22T13:29:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.342+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b01a77e0 con 0x7ff8b0101090 2024-04-22T13:29:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.344+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3269728517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff8a0009a30 con 0x7ff8b0101090 2024-04-22T13:29:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.344+0000 7ff8a7fff640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.345+0000 7ff8a7fff640 1 -- 172.21.15.5:0/3269728517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff8a00d0200 con 0x7ff8b0101090 2024-04-22T13:29:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.345+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.345+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff89802c240 tx=0x7ff898005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.345+0000 7ff8b7f72740 1 Processor -- start 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 -- start start 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16ead8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eade80 0x555c16eb6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 0x555c16eb8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c16e121f0 con 0x555c16eb6a00 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555c16daf4b0 con 0x555c16eade80 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555c16e0e1b0 con 0x555c16ead4d0 2024-04-22T13:29:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 0x555c16eb8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 0x555c16eb8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33702/0 (socket says 172.21.15.5:33702) 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8af7fe640 1 -- 172.21.15.5:0/321816210 learned_addr learned my addr 172.21.15.5:0/321816210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16ead8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.346+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eade80 0x555c16eb6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8af7fe640 1 -- 172.21.15.5:0/321816210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 msgr2=0x555c16ead8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16ead8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8af7fe640 1 -- 172.21.15.5:0/321816210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eade80 msgr2=0x555c16eb6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eade80 0x555c16eb6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8af7fe640 1 -- 172.21.15.5:0/321816210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c16da11b0 con 0x555c16eb6a00 2024-04-22T13:29:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 0x555c16eb8e50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009f80 tx=0x7ff8a8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605e355210860acc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8a57fa640 1 -- 172.21.15.5:0/321816210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8011700 con 0x555c16eb6a00 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8a57fa640 1 -- 172.21.15.5:0/321816210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a8011d50 con 0x555c16eb6a00 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.347+0000 7ff8a57fa640 1 -- 172.21.15.5:0/321816210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8019d50 con 0x555c16eb6a00 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 -- 172.21.15.5:0/321816210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 msgr2=0x555c16eb8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 0x555c16eb8e50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009f80 tx=0x7ff8a8010860 comp rx=0 tx=0).stop 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 -- 172.21.15.5:0/321816210 shutdown_connections 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eb6a00 0x555c16eb8e50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eade80 0x555c16eb6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/321816210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16ead8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 -- 172.21.15.5:0/321816210 >> 172.21.15.5:0/321816210 conn(0x555c16ea7ba0 msgr2=0x555c16ea7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 -- 172.21.15.5:0/321816210 shutdown_connections 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 -- 172.21.15.5:0/321816210 wait complete. 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 -- start start 2024-04-22T13:29:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.348+0000 7ff8b7f72740 1 Processor -- start 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16eb62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 0x555c16eb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c16eb9970 con 0x555c16eade80 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555c16dcb500 con 0x555c16eb6a00 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555c16eb9af0 con 0x555c16ead4d0 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8b7f72740 10 client.0 _ll_register_callbacks cb 0x555c16d99b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 0x555c16eb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 0x555c16eb3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33716/0 (socket says 172.21.15.5:33716) 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 -- 172.21.15.5:0/3293002613 learned_addr learned my addr 172.21.15.5:0/3293002613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16eb62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 -- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 msgr2=0x555c16eb62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16eb62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 -- 172.21.15.5:0/3293002613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 msgr2=0x555c16eb3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 -- 172.21.15.5:0/3293002613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c16b243f0 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 0x555c16eb3630 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff8b006b670 tx=0x7ff898009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1480a041bf306fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.349+0000 7ff8967fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8967fc640 1 -- 172.21.15.5:0/3293002613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898050030 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8967fc640 1 -- 172.21.15.5:0/3293002613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff89803a040 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a8011d50 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8967fc640 1 -- 172.21.15.5:0/3293002613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8980046c0 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 10 client.16854 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c16bee7d0 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8967fc640 1 -- 172.21.15.5:0/3293002613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a8011d50 con 0x555c16eade80 2024-04-22T13:29:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8967fc640 1 -- 172.21.15.5:0/3293002613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff898003a30 con 0x555c16eade80 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 10 client.16854 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 10 client.16854 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 20 client.16854 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16ec8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 4 client.16854 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.350+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c16db1930 con 0x555c16ec6b00 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.351+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16ec8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.351+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16ec8f70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89f2b87e9494691c server_cookie=58722f127f19742b in_seq=0 out_seq=0 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.351+0000 7ff8967fc640 10 client.16854 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8967fc640 1 -- 172.21.15.5:0/3293002613 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c16db1930 con 0x555c16ec6b00 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8967fc640 10 client.16854 handle_command_reply: tid=0 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 client.16854 shutdown 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 msgr2=0x555c16eb3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 0x555c16eb3630 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff8b006b670 tx=0x7ff898009260 comp rx=0 tx=0).stop 2024-04-22T13:29:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 shutdown_connections 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16ec8f70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16eade80 0x555c16eb3630 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8967fc640 0 client.16854 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3293002613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16ead4d0 0x555c16eb62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 >> 172.21.15.5:0/3293002613 conn(0x555c16ea7ba0 msgr2=0x555c16ea98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 shutdown_connections 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3293002613 wait complete. 2024-04-22T13:29:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.352+0000 7ff8b7f72740 20 client.16854 trim_cache size 0 max 16384 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 Processor -- start 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 -- start start 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16eb3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16ec2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01afde0 con 0x555c16ead4d0 2024-04-22T13:29:58.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01b1920 con 0x555c16eb6a00 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01b04b0 con 0x555c16eade80 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16eb3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16eb3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33728/0 (socket says 172.21.15.5:33728) 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.366+0000 7ff8aeffd640 1 -- 172.21.15.5:0/2256729320 learned_addr learned my addr 172.21.15.5:0/2256729320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16ec2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8aeffd640 1 -- 172.21.15.5:0/2256729320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 msgr2=0x555c16ec2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16ec2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8aeffd640 1 -- 172.21.15.5:0/2256729320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 msgr2=0x555c16eb5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8aeffd640 1 -- 172.21.15.5:0/2256729320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c16ea8520 con 0x555c16ead4d0 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16eb3b50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0098580 tx=0x7ff8a009bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfff550f6858fb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8a4ff9640 1 -- 172.21.15.5:0/2256729320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a009a470 con 0x555c16ead4d0 2024-04-22T13:29:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8a4ff9640 1 -- 172.21.15.5:0/2256729320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a0009d20 con 0x555c16ead4d0 2024-04-22T13:29:58.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.367+0000 7ff8a4ff9640 1 -- 172.21.15.5:0/2256729320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0098950 con 0x555c16ead4d0 2024-04-22T13:29:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 -- 172.21.15.5:0/2256729320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 msgr2=0x555c16eb3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16eb3b50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0098580 tx=0x7ff8a009bfc0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 -- 172.21.15.5:0/2256729320 shutdown_connections 2024-04-22T13:29:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eb5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16ec2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/2256729320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16eb3b50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 -- 172.21.15.5:0/2256729320 >> 172.21.15.5:0/2256729320 conn(0x555c16ea7ba0 msgr2=0x555c16ea6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 -- 172.21.15.5:0/2256729320 shutdown_connections 2024-04-22T13:29:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.368+0000 7ff8b7f72740 1 -- 172.21.15.5:0/2256729320 wait complete. 2024-04-22T13:29:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 -- start start 2024-04-22T13:29:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 Processor -- start 2024-04-22T13:29:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16ebefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16eba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01b1250 con 0x555c16ead4d0 2024-04-22T13:29:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01b0b80 con 0x555c16eb6a00 2024-04-22T13:29:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01addc0 con 0x555c16eade80 2024-04-22T13:29:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8b7f72740 10 client.0 _ll_register_callbacks cb 0x7ff898004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16ebefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16ebefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33732/0 (socket says 172.21.15.5:33732) 2024-04-22T13:29:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.369+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3576180934 learned_addr learned my addr 172.21.15.5:0/3576180934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8af7fe640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16eba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 msgr2=0x555c16eba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16eba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3576180934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 msgr2=0x555c16eba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8aeffd640 1 -- 172.21.15.5:0/3576180934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c16ebae30 con 0x555c16ead4d0 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8aeffd640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16ebefb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00a48b0 tx=0x7ff8a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbb004a3d354bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8967fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8967fc640 1 -- 172.21.15.5:0/3576180934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a007e020 con 0x555c16ead4d0 2024-04-22T13:29:58.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8967fc640 1 -- 172.21.15.5:0/3576180934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff8a00cdba0 con 0x555c16ead4d0 2024-04-22T13:29:58.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a0009d20 con 0x555c16ead4d0 2024-04-22T13:29:58.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8967fc640 1 -- 172.21.15.5:0/3576180934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8a009a530 con 0x555c16ead4d0 2024-04-22T13:29:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8b7f72740 10 client.16860 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c16f2cf20 con 0x555c16ead4d0 2024-04-22T13:29:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.370+0000 7ff8967fc640 1 -- 172.21.15.5:0/3576180934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a0009d20 con 0x555c16ead4d0 2024-04-22T13:29:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8967fc640 1 -- 172.21.15.5:0/3576180934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8a0096ce0 con 0x555c16ead4d0 2024-04-22T13:29:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8b7f72740 10 client.16860 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8b7f72740 10 client.16860 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8b7f72740 20 client.16860 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16f2f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8b7f72740 4 client.16860 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c16e1fc10 con 0x555c16ec6b00 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.371+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16f2f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8ae7fc640 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16f2f610 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5da8f3dd87a23d32 server_cookie=4ae7bc5cbc611bab in_seq=0 out_seq=0 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8967fc640 10 client.16860 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:58.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8967fc640 1 -- 172.21.15.5:0/3576180934 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c16e1fc10 con 0x555c16ec6b00 2024-04-22T13:29:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8967fc640 10 client.16860 handle_command_reply: tid=0 2024-04-22T13:29:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 client.16860 shutdown 2024-04-22T13:29:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 msgr2=0x555c16ebefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16ebefb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00a48b0 tx=0x7ff8a00046c0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 shutdown_connections 2024-04-22T13:29:58.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c16eb6a00 0x555c16eba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555c16eade80 0x555c16eba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x555c16ec6b00 0x555c16f2f610 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 --2- 172.21.15.5:0/3576180934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c16ead4d0 0x555c16ebefb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 >> 172.21.15.5:0/3576180934 conn(0x555c16ea7ba0 msgr2=0x555c16ea6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8967fc640 0 client.16860 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:29:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 shutdown_connections 2024-04-22T13:29:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 1 -- 172.21.15.5:0/3576180934 wait complete. 2024-04-22T13:29:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.372+0000 7ff8b7f72740 20 client.16860 trim_cache size 0 max 16384 2024-04-22T13:29:58.383 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff89c0a3d20 msgr2=0x7ff89c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff89802c240 tx=0x7ff898005a20 comp rx=0 tx=0).stop 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 msgr2=0x7ff8b01a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 0x7ff8b01a0640 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000d950 tx=0x7ff8a000de10 comp rx=0 tx=0).stop 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 shutdown_connections 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b019a7b0 0x7ff8b019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8b0106160 0x7ff8b01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 --2- 172.21.15.5:0/3269728517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8b0101090 0x7ff8b01a0640 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 >> 172.21.15.5:0/3269728517 conn(0x7ff8b0078000 msgr2=0x7ff8b0069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 shutdown_connections 2024-04-22T13:29:58.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.373+0000 7ff8b5385640 1 -- 172.21.15.5:0/3269728517 wait complete. 2024-04-22T13:29:58.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:29:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 -- 172.21.15.5:0/2691663673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 msgr2=0x7f07c8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 --2- 172.21.15.5:0/2691663673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c8101470 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009900 tx=0x7f07bc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 -- 172.21.15.5:0/2691663673 shutdown_connections 2024-04-22T13:29:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 --2- 172.21.15.5:0/2691663673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 0x7f07c810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 --2- 172.21.15.5:0/2691663673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c8101a40 0x7f07c8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 --2- 172.21.15.5:0/2691663673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c8101470 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 -- 172.21.15.5:0/2691663673 >> 172.21.15.5:0/2691663673 conn(0x7f07c8078000 msgr2=0x7f07c8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 -- 172.21.15.5:0/2691663673 shutdown_connections 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.500+0000 7f07cef04640 1 -- 172.21.15.5:0/2691663673 wait complete. 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 Processor -- start 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 -- start start 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 0x7f07c819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c81985e0 0x7f07c8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81110c0 con 0x7f07c8101090 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8110f40 con 0x7f07c8106160 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cef04640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8111240 con 0x7f07c81985e0 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07ccca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07ccca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33754/0 (socket says 172.21.15.5:33754) 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07ccca0640 1 -- 172.21.15.5:0/817482290 learned_addr learned my addr 172.21.15.5:0/817482290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c81985e0 0x7f07c8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.501+0000 7f07c7fff640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 0x7f07c819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07ccca0640 1 -- 172.21.15.5:0/817482290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c81985e0 msgr2=0x7f07c8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07ccca0640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c81985e0 0x7f07c8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07ccca0640 1 -- 172.21.15.5:0/817482290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 msgr2=0x7f07c819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07ccca0640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 0x7f07c819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07ccca0640 1 -- 172.21.15.5:0/817482290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c8199100 con 0x7f07c8101090 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c81985e0 0x7f07c8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07c7fff640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 0x7f07c819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07ccca0640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c819e430 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f07bc005940 tx=0x7f07bc02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07c5ffb640 1 -- 172.21.15.5:0/817482290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc043020 con 0x7f07c8101090 2024-04-22T13:29:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c8199390 con 0x7f07c8101090 2024-04-22T13:29:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07c5ffb640 1 -- 172.21.15.5:0/817482290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07bc02ca70 con 0x7f07c8101090 2024-04-22T13:29:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07c5ffb640 1 -- 172.21.15.5:0/817482290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0039f0 con 0x7f07c8101090 2024-04-22T13:29:58.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.502+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81a5630 con 0x7f07c8101090 2024-04-22T13:29:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07c5ffb640 1 -- 172.21.15.5:0/817482290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f07bc046020 con 0x7f07c8101090 2024-04-22T13:29:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 Processor -- start 2024-04-22T13:29:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 -- start start 2024-04-22T13:29:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 0x55c31b8c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c3d60 0x55c31b8cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8cce60 0x55c31b8cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c31b8281f0 con 0x55c31b8c33b0 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c31b7c54b0 con 0x55c31b8c3d60 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c31b8241b0 con 0x55c31b8cce60 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07cd4a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 0x55c31b8c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07cd4a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 0x55c31b8c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33758/0 (socket says 172.21.15.5:33758) 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07cd4a1640 1 -- 172.21.15.5:0/2833494436 learned_addr learned my addr 172.21.15.5:0/2833494436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07c7fff640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8cce60 0x55c31b8cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07c5ffb640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f07b80a3d20 0x7f07b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.505+0000 7f07ccca0640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c3d60 0x55c31b8cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 -- 172.21.15.5:0/2833494436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8cce60 msgr2=0x55c31b8cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8cce60 0x55c31b8cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 -- 172.21.15.5:0/2833494436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c3d60 msgr2=0x55c31b8cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c3d60 0x55c31b8cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 -- 172.21.15.5:0/2833494436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31b7b71b0 con 0x55c31b8c33b0 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07ccca0640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c3d60 0x55c31b8cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07c7fff640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8cce60 0x55c31b8cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f07b80a3d20 0x7f07b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07c5ffb640 1 -- 172.21.15.5:0/817482290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f07bc0f2820 con 0x7f07c8101090 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 0x55c31b8c3790 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f07c00118b0 tx=0x7f07c0011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aaa281e07d9088 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07b2ffd640 1 -- 172.21.15.5:0/2833494436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0017050 con 0x55c31b8c33b0 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07b2ffd640 1 -- 172.21.15.5:0/2833494436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c001bb90 con 0x55c31b8c33b0 2024-04-22T13:29:58.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.506+0000 7f07b2ffd640 1 -- 172.21.15.5:0/2833494436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0026db0 con 0x55c31b8c33b0 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 -- 172.21.15.5:0/2833494436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 msgr2=0x55c31b8c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f07b80a3d20 0x7f07b80a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07c0023000 tx=0x7f07c0024000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 0x55c31b8c3790 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f07c00118b0 tx=0x7f07c0011d70 comp rx=0 tx=0).stop 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 -- 172.21.15.5:0/2833494436 shutdown_connections 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8cce60 0x55c31b8cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c3d60 0x55c31b8cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/2833494436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c33b0 0x55c31b8c3790 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 -- 172.21.15.5:0/2833494436 >> 172.21.15.5:0/2833494436 conn(0x55c31b8bdba0 msgr2=0x55c31b8bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 -- 172.21.15.5:0/2833494436 shutdown_connections 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 -- 172.21.15.5:0/2833494436 wait complete. 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 -- start start 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.507+0000 7f07d1bd4740 1 Processor -- start 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c31b8d3da0 con 0x55c31b8c3d60 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c31b7e1500 con 0x55c31b8cce60 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c31b8d3f20 con 0x55c31b8c33b0 2024-04-22T13:29:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07d1bd4740 10 client.0 _ll_register_callbacks cb 0x55c31b7afb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07ccca0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33768/0 (socket says 172.21.15.5:33768) 2024-04-22T13:29:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 -- 172.21.15.5:0/444038160 learned_addr learned my addr 172.21.15.5:0/444038160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 -- 172.21.15.5:0/444038160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 msgr2=0x55c31b8d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 -- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 msgr2=0x55c31b8d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.508+0000 7f07c7fff640 1 -- 172.21.15.5:0/444038160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31b53a3f0 con 0x55c31b8c3d60 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07c7fff640 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d0640 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f07b4013790 tx=0x7f07b4013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fb2c8798669a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07a7fff640 1 -- 172.21.15.5:0/444038160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4011dd0 con 0x55c31b8c3d60 2024-04-22T13:29:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07a7fff640 1 -- 172.21.15.5:0/444038160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f07b4005020 con 0x55c31b8c3d60 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c001bb90 con 0x55c31b8c3d60 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07a7fff640 1 -- 172.21.15.5:0/444038160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f07b4005220 con 0x55c31b8c3d60 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 10 client.16872 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c31b6047d0 con 0x55c31b8c3d60 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07a7fff640 1 -- 172.21.15.5:0/444038160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c001bb90 con 0x55c31b8c3d60 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07a7fff640 1 -- 172.21.15.5:0/444038160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f07b401a030 con 0x55c31b8c3d60 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 10 client.16872 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 10 client.16872 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 20 client.16872 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b8e1060 0x55c31b8e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 4 client.16872 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.509+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c31b7c7930 con 0x55c31b8e1060 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.510+0000 7f07ccca0640 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b8e1060 0x55c31b8e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.510+0000 7f07ccca0640 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b8e1060 0x55c31b8e3480 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49bc43a02707be7b server_cookie=346bd949d75e6676 in_seq=0 out_seq=0 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.510+0000 7f07a7fff640 10 client.16872 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07a7fff640 1 -- 172.21.15.5:0/444038160 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c31b7c7930 con 0x55c31b8e1060 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07a7fff640 10 client.16872 handle_command_reply: tid=0 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 client.16872 shutdown 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 msgr2=0x55c31b8d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d0640 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f07b4013790 tx=0x7f07b4013c50 comp rx=0 tx=0).stop 2024-04-22T13:29:58.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 shutdown_connections 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b8e1060 0x55c31b8e3480 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d0640 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/444038160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 >> 172.21.15.5:0/444038160 conn(0x55c31b8bdba0 msgr2=0x55c31b8bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07a7fff640 0 client.16872 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 shutdown_connections 2024-04-22T13:29:58.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 1 -- 172.21.15.5:0/444038160 wait complete. 2024-04-22T13:29:58.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.511+0000 7f07d1bd4740 20 client.16872 trim_cache size 0 max 16384 2024-04-22T13:29:58.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 Processor -- start 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 -- start start 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c33b0 0x55c31b8ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c3d60 0x55c31b8dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8cce60 0x55c31b8cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81ab9e0 con 0x55c31b8cce60 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81ad520 con 0x55c31b8c33b0 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81ac0b0 con 0x55c31b8c3d60 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07cd4a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8cce60 0x55c31b8cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c3d60 0x55c31b8dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07cd4a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8cce60 0x55c31b8cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33778/0 (socket says 172.21.15.5:33778) 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07cd4a1640 1 -- 172.21.15.5:0/902067229 learned_addr learned my addr 172.21.15.5:0/902067229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.525+0000 7f07ccca0640 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c33b0 0x55c31b8ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07c7fff640 1 -- 172.21.15.5:0/902067229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c33b0 msgr2=0x55c31b8ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07c7fff640 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c33b0 0x55c31b8ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07c7fff640 1 -- 172.21.15.5:0/902067229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8cce60 msgr2=0x55c31b8cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07c7fff640 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8cce60 0x55c31b8cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07c7fff640 1 -- 172.21.15.5:0/902067229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31b8cf610 con 0x55c31b8c3d60 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07c7fff640 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c3d60 0x55c31b8dcca0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f07b400f580 tx=0x7f07b40118b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5416bd09f184701 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07b27fc640 1 -- 172.21.15.5:0/902067229 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4011bd0 con 0x55c31b8c3d60 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07b27fc640 1 -- 172.21.15.5:0/902067229 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b4011d70 con 0x55c31b8c3d60 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.526+0000 7f07b27fc640 1 -- 172.21.15.5:0/902067229 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4004b30 con 0x55c31b8c3d60 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 -- 172.21.15.5:0/902067229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c3d60 msgr2=0x55c31b8dcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c3d60 0x55c31b8dcca0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f07b400f580 tx=0x7f07b40118b0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 -- 172.21.15.5:0/902067229 shutdown_connections 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8cce60 0x55c31b8cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c3d60 0x55c31b8dcca0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/902067229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8c33b0 0x55c31b8ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 -- 172.21.15.5:0/902067229 >> 172.21.15.5:0/902067229 conn(0x55c31b8bdba0 msgr2=0x55c31b8bf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 -- 172.21.15.5:0/902067229 shutdown_connections 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 -- 172.21.15.5:0/902067229 wait complete. 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 -- start start 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.527+0000 7f07d1bd4740 1 Processor -- start 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81ace50 con 0x55c31b8c3d60 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81ac780 con 0x55c31b8cce60 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a99c0 con 0x55c31b8c33b0 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07d1bd4740 10 client.0 _ll_register_callbacks cb 0x7f07b4009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33784/0 (socket says 172.21.15.5:33784) 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 -- 172.21.15.5:0/1490226969 learned_addr learned my addr 172.21.15.5:0/1490226969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07ccca0640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07cd4a1640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 -- 172.21.15.5:0/1490226969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 msgr2=0x55c31b8d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 -- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 msgr2=0x55c31b8d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.528+0000 7f07c7fff640 1 -- 172.21.15.5:0/1490226969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31b9465d0 con 0x55c31b8c3d60 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07c7fff640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d2770 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f07b4013c90 tx=0x7f07b4005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a2c8efbbe355b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07a7fff640 1 -- 172.21.15.5:0/1490226969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b403d060 con 0x55c31b8c3d60 2024-04-22T13:29:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07a7fff640 1 -- 172.21.15.5:0/1490226969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f07b400fd30 con 0x55c31b8c3d60 2024-04-22T13:29:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b4011d70 con 0x55c31b8c3d60 2024-04-22T13:29:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07a7fff640 1 -- 172.21.15.5:0/1490226969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f07b4011bd0 con 0x55c31b8c3d60 2024-04-22T13:29:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 10 client.16878 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c31b9478d0 con 0x55c31b8c3d60 2024-04-22T13:29:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07a7fff640 1 -- 172.21.15.5:0/1490226969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b4011d70 con 0x55c31b8c3d60 2024-04-22T13:29:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07a7fff640 1 -- 172.21.15.5:0/1490226969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f07b401a040 con 0x55c31b8c3d60 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 10 client.16878 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 10 client.16878 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 20 client.16878 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.529+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b947f80 0x55c31b94a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07d1bd4740 4 client.16878 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c31b835c10 con 0x55c31b947f80 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07ccca0640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b947f80 0x55c31b94a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07ccca0640 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b947f80 0x55c31b94a3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca9dec2fe1863633 server_cookie=81a2b081b95b9a73 in_seq=0 out_seq=0 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07a7fff640 10 client.16878 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07a7fff640 1 -- 172.21.15.5:0/1490226969 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c31b835c10 con 0x55c31b947f80 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07a7fff640 10 client.16878 handle_command_reply: tid=0 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.530+0000 7f07d1bd4740 1 client.16878 shutdown 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 msgr2=0x55c31b8d2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d2770 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f07b4013c90 tx=0x7f07b4005670 comp rx=0 tx=0).stop 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 shutdown_connections 2024-04-22T13:29:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55c31b947f80 0x55c31b94a3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31b8c3d60 0x55c31b8d2770 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c31b8cce60 0x55c31b8d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 --2- 172.21.15.5:0/1490226969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c31b8c33b0 0x55c31b8d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07a7fff640 0 client.16878 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 >> 172.21.15.5:0/1490226969 conn(0x55c31b8bdba0 msgr2=0x55c31b8cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 shutdown_connections 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 1 -- 172.21.15.5:0/1490226969 wait complete. 2024-04-22T13:29:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07d1bd4740 20 client.16878 trim_cache size 0 max 16384 2024-04-22T13:29:58.538 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f07b80a3d20 msgr2=0x7f07b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f07b80a3d20 0x7f07b80a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07c0023000 tx=0x7f07c0024000 comp rx=0 tx=0).stop 2024-04-22T13:29:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 msgr2=0x7f07c819e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c819e430 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f07bc005940 tx=0x7f07bc02c5f0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 shutdown_connections 2024-04-22T13:29:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f07b80a3d20 0x7f07b80a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.531+0000 7f07cef04640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07c81985e0 0x7f07c8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.532+0000 7f07cef04640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c8106160 0x7f07c819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.532+0000 7f07cef04640 1 --2- 172.21.15.5:0/817482290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c8101090 0x7f07c819e430 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.532+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 >> 172.21.15.5:0/817482290 conn(0x7f07c8078000 msgr2=0x7f07c80696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.532+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 shutdown_connections 2024-04-22T13:29:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.532+0000 7f07cef04640 1 -- 172.21.15.5:0/817482290 wait complete. 2024-04-22T13:29:58.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:29:58.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 -- 172.21.15.5:0/1482665351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35810c500 msgr2=0x7fb35810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 --2- 172.21.15.5:0/1482665351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35810c500 0x7fb35810e9a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb350009ca0 tx=0x7fb35002c180 comp rx=0 tx=0).stop 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 -- 172.21.15.5:0/1482665351 shutdown_connections 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 --2- 172.21.15.5:0/1482665351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35810c500 0x7fb35810e9a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 --2- 172.21.15.5:0/1482665351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb358103ae0 0x7fb35810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 --2- 172.21.15.5:0/1482665351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3581031c0 0x7fb3581035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 -- 172.21.15.5:0/1482665351 >> 172.21.15.5:0/1482665351 conn(0x7fb358078000 msgr2=0x7fb358078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 -- 172.21.15.5:0/1482665351 shutdown_connections 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 -- 172.21.15.5:0/1482665351 wait complete. 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 Processor -- start 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.656+0000 7fb3605ce640 1 -- start start 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb3605ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581031c0 0x7fb358198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb3605ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 0x7fb358198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb3605ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35810c500 0x7fb35819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb3605ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb358112190 con 0x7fb358103ae0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb3605ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb358112010 con 0x7fb3581031c0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb3605ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb358112310 con 0x7fb35810c500 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 0x7fb358198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 0x7fb358198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33790/0 (socket says 172.21.15.5:33790) 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 -- 172.21.15.5:0/3589073370 learned_addr learned my addr 172.21.15.5:0/3589073370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35810c500 0x7fb35819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35e36a640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581031c0 0x7fb358198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 -- 172.21.15.5:0/3589073370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35810c500 msgr2=0x7fb35819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35810c500 0x7fb35819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 -- 172.21.15.5:0/3589073370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581031c0 msgr2=0x7fb358198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581031c0 0x7fb358198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 -- 172.21.15.5:0/3589073370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35819d6b0 con 0x7fb358103ae0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb35db69640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 0x7fb358198c50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb34800e9f0 tx=0x7fb34800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.657+0000 7fb34f7fe640 1 -- 172.21.15.5:0/3589073370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34800cd80 con 0x7fb358103ae0 2024-04-22T13:29:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.658+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35819d9a0 con 0x7fb358103ae0 2024-04-22T13:29:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.658+0000 7fb34f7fe640 1 -- 172.21.15.5:0/3589073370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3480108e0 con 0x7fb358103ae0 2024-04-22T13:29:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.658+0000 7fb34f7fe640 1 -- 172.21.15.5:0/3589073370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348018980 con 0x7fb358103ae0 2024-04-22T13:29:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.658+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a5230 con 0x7fb358103ae0 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.660+0000 7fb34f7fe640 1 -- 172.21.15.5:0/3589073370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb348018b20 con 0x7fb358103ae0 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.660+0000 7fb3631c3740 1 Processor -- start 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.660+0000 7fb3631c3740 1 -- start start 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x56487117d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871186430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871188e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb3631c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5648710e21f0 con 0x56487117d4d0 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb3631c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56487107f4b0 con 0x564871186a00 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb3631c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648710de1b0 con 0x56487117de80 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x56487117d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x56487117d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33792/0 (socket says 172.21.15.5:33792) 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 -- 172.21.15.5:0/2452710086 learned_addr learned my addr 172.21.15.5:0/2452710086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35db69640 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871188e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35e36a640 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871186430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 -- 172.21.15.5:0/2452710086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 msgr2=0x564871186430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871186430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb34f7fe640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3440a3d20 0x7fb3440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 -- 172.21.15.5:0/2452710086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 msgr2=0x564871188e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871188e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35eb6b640 1 -- 172.21.15.5:0/2452710086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648710711b0 con 0x56487117d4d0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb35db69640 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871188e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.661+0000 7fb34f7fe640 1 -- 172.21.15.5:0/3589073370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb3480d0ae0 con 0x7fb358103ae0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.662+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3440a3d20 0x7fb3440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.662+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x56487117d8b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb350009ca0 tx=0x7fb350009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5219aa9f5dbc4b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.662+0000 7fb34cbfe640 1 -- 172.21.15.5:0/2452710086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350039040 con 0x56487117d4d0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.662+0000 7fb34cbfe640 1 -- 172.21.15.5:0/2452710086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb350043050 con 0x56487117d4d0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.662+0000 7fb34cbfe640 1 -- 172.21.15.5:0/2452710086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350051070 con 0x56487117d4d0 2024-04-22T13:29:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 -- 172.21.15.5:0/2452710086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 msgr2=0x56487117d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x56487117d8b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb350009ca0 tx=0x7fb350009900 comp rx=0 tx=0).stop 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3440a3d20 0x7fb3440a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb35004a000 tx=0x7fb35004b000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 -- 172.21.15.5:0/2452710086 shutdown_connections 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871188e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871186430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2452710086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x56487117d8b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 -- 172.21.15.5:0/2452710086 >> 172.21.15.5:0/2452710086 conn(0x564871177ba0 msgr2=0x564871177f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 -- 172.21.15.5:0/2452710086 shutdown_connections 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 -- 172.21.15.5:0/2452710086 wait complete. 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 -- start start 2024-04-22T13:29:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 Processor -- start 2024-04-22T13:29:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871186300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56487117de80 0x564871183690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.663+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564871186a00 0x564871183c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb3631c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564871189970 con 0x56487117d4d0 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb3631c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56487109b500 con 0x56487117de80 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb3631c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564871189af0 con 0x564871186a00 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb3631c3740 10 client.0 _ll_register_callbacks cb 0x564871069b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871186300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871186300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33800/0 (socket says 172.21.15.5:33800) 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 -- 172.21.15.5:0/2634929021 learned_addr learned my addr 172.21.15.5:0/2634929021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564871186a00 0x564871183c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35db69640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56487117de80 0x564871183690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 -- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564871186a00 msgr2=0x564871183c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564871186a00 0x564871183c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 -- 172.21.15.5:0/2634929021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56487117de80 msgr2=0x564871183690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56487117de80 0x564871183690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 -- 172.21.15.5:0/2634929021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564870df43f0 con 0x56487117d4d0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564871186a00 0x564871183c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35db69640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56487117de80 0x564871183690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb35e36a640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871186300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb354013710 tx=0x7fb354013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7135a569ad28895d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb341ffb640 1 -- 172.21.15.5:0/2634929021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354004460 con 0x56487117d4d0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb341ffb640 1 -- 172.21.15.5:0/2634929021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb3540119c0 con 0x56487117d4d0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb350043050 con 0x56487117d4d0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.664+0000 7fb341ffb640 1 -- 172.21.15.5:0/2634929021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb354011c00 con 0x56487117d4d0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 10 client.16890 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564870ebe7d0 con 0x56487117d4d0 2024-04-22T13:29:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb341ffb640 1 -- 172.21.15.5:0/2634929021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb350043050 con 0x56487117d4d0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb341ffb640 1 -- 172.21.15.5:0/2634929021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb35401a070 con 0x56487117d4d0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 10 client.16890 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 10 client.16890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 20 client.16890 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x564871196cd0 0x564871199150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 4 client.16890 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564871081930 con 0x564871196cd0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.665+0000 7fb35db69640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x564871196cd0 0x564871199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.666+0000 7fb341ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.666+0000 7fb35db69640 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x564871196cd0 0x564871199150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cba93dc1f9898179 server_cookie=93e5f1f2eb49ac7e in_seq=0 out_seq=0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.666+0000 7fb341ffb640 1 -- 172.21.15.5:0/2634929021 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564871081930 con 0x564871196cd0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.666+0000 7fb341ffb640 10 client.16890 handle_command_reply: tid=0 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.666+0000 7fb3631c3740 1 client.16890 shutdown 2024-04-22T13:29:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 msgr2=0x564871186300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871186300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb354013710 tx=0x7fb354013bd0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 shutdown_connections 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x564871196cd0 0x564871199150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564871186a00 0x564871183c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56487117de80 0x564871183690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb341ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 --2- 172.21.15.5:0/2634929021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871186300 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 >> 172.21.15.5:0/2634929021 conn(0x564871177ba0 msgr2=0x5648711798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 shutdown_connections 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 1 -- 172.21.15.5:0/2634929021 wait complete. 2024-04-22T13:29:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.667+0000 7fb3631c3740 20 client.16890 trim_cache size 0 max 16384 2024-04-22T13:29:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 Processor -- start 2024-04-22T13:29:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 -- start start 2024-04-22T13:29:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871192860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871176a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871185dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581adb20 con 0x56487117d4d0 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581af660 con 0x564871186a00 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb3631c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581ae1f0 con 0x56487117de80 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb35e36a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871192860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb35e36a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871192860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33808/0 (socket says 172.21.15.5:33808) 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb35e36a640 1 -- 172.21.15.5:0/1158929588 learned_addr learned my addr 172.21.15.5:0/1158929588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871185dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.681+0000 7fb35db69640 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871176a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb35e36a640 1 -- 172.21.15.5:0/1158929588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 msgr2=0x564871176a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb35e36a640 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871176a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb35e36a640 1 -- 172.21.15.5:0/1158929588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 msgr2=0x564871185dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb35e36a640 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871185dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb35e36a640 1 -- 172.21.15.5:0/1158929588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564871183f80 con 0x56487117d4d0 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb35e36a640 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871192860 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb3540136e0 tx=0x7fb35401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea19008c7685d060 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb343fff640 1 -- 172.21.15.5:0/1158929588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354029070 con 0x56487117d4d0 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb343fff640 1 -- 172.21.15.5:0/1158929588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb35402b070 con 0x56487117d4d0 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.682+0000 7fb343fff640 1 -- 172.21.15.5:0/1158929588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354003a70 con 0x56487117d4d0 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 -- 172.21.15.5:0/1158929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 msgr2=0x564871192860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871192860 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb3540136e0 tx=0x7fb35401c440 comp rx=0 tx=0).stop 2024-04-22T13:29:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 -- 172.21.15.5:0/1158929588 shutdown_connections 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x564871185dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117de80 0x564871176a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 --2- 172.21.15.5:0/1158929588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117d4d0 0x564871192860 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 -- 172.21.15.5:0/1158929588 >> 172.21.15.5:0/1158929588 conn(0x564871177ba0 msgr2=0x564871176370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 -- 172.21.15.5:0/1158929588 shutdown_connections 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 -- 172.21.15.5:0/1158929588 wait complete. 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 -- start start 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.683+0000 7fb3631c3740 1 Processor -- start 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117d4d0 0x56487118f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 0x564871180130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x5648711806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581aef90 con 0x56487117de80 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581ae8c0 con 0x564871186a00 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581abb00 con 0x56487117d4d0 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 10 client.0 _ll_register_callbacks cb 0x7fb354009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 0x564871180130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 0x564871180130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33814/0 (socket says 172.21.15.5:33814) 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 -- 172.21.15.5:0/3492476476 learned_addr learned my addr 172.21.15.5:0/3492476476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35eb6b640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x5648711806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35e36a640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117d4d0 0x56487118f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 -- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117d4d0 msgr2=0x56487118f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117d4d0 0x56487118f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 -- 172.21.15.5:0/3492476476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 msgr2=0x5648711806b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x5648711806b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 -- 172.21.15.5:0/3492476476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564871180ee0 con 0x56487117de80 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35e36a640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117d4d0 0x56487118f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb35db69640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 0x564871180130 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb348099480 tx=0x7fb3480d2ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e0580ed3ec9fac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb341ffb640 1 -- 172.21.15.5:0/3492476476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480a0e60 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb341ffb640 1 -- 172.21.15.5:0/3492476476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb34809adb0 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.684+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb35402b070 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb341ffb640 1 -- 172.21.15.5:0/3492476476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3480a5720 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 10 client.16896 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648711fd380 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb341ffb640 1 -- 172.21.15.5:0/3492476476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb35402b070 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb341ffb640 1 -- 172.21.15.5:0/3492476476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb34809bf00 con 0x56487117de80 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 10 client.16896 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 10 client.16896 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 20 client.16896 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5648711fdab0 0x5648711ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 4 client.16896 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.685+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648710efc10 con 0x5648711fdab0 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb35e36a640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5648711fdab0 0x5648711ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb35e36a640 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5648711fdab0 0x5648711ffed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14d4400da221c36b server_cookie=7371e7e0a896a70e in_seq=0 out_seq=0 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb341ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb341ffb640 1 -- 172.21.15.5:0/3492476476 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648710efc10 con 0x5648711fdab0 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb341ffb640 10 client.16896 handle_command_reply: tid=0 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 client.16896 shutdown 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 msgr2=0x564871180130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 0x564871180130 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb348099480 tx=0x7fb3480d2ff0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 shutdown_connections 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5648711fdab0 0x5648711ffed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56487117de80 0x564871180130 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564871186a00 0x5648711806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb341ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 --2- 172.21.15.5:0/3492476476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56487117d4d0 0x56487118f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 >> 172.21.15.5:0/3492476476 conn(0x564871177ba0 msgr2=0x564871176c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 shutdown_connections 2024-04-22T13:29:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.686+0000 7fb3631c3740 1 -- 172.21.15.5:0/3492476476 wait complete. 2024-04-22T13:29:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3631c3740 20 client.16896 trim_cache size 0 max 16384 2024-04-22T13:29:58.693 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3440a3d20 msgr2=0x7fb3440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3440a3d20 0x7fb3440a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb35004a000 tx=0x7fb35004b000 comp rx=0 tx=0).stop 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 msgr2=0x7fb358198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 0x7fb358198c50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb34800e9f0 tx=0x7fb34800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 shutdown_connections 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb358103ae0 0x7fb358198c50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3440a3d20 0x7fb3440a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35810c500 0x7fb35819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 --2- 172.21.15.5:0/3589073370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3581031c0 0x7fb358198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 >> 172.21.15.5:0/3589073370 conn(0x7fb358078000 msgr2=0x7fb3580fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 shutdown_connections 2024-04-22T13:29:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.687+0000 7fb3605ce640 1 -- 172.21.15.5:0/3589073370 wait complete. 2024-04-22T13:29:58.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 -- 172.21.15.5:0/3858612057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 msgr2=0x7fbfb4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/3858612057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb4102570 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009900 tx=0x7fbfb002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 -- 172.21.15.5:0/3858612057 shutdown_connections 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/3858612057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb4107260 0x7fbfb410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/3858612057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb4102b40 0x7fbfb4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/3858612057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb4102570 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 -- 172.21.15.5:0/3858612057 >> 172.21.15.5:0/3858612057 conn(0x7fbfb40fda10 msgr2=0x7fbfb40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 -- 172.21.15.5:0/3858612057 shutdown_connections 2024-04-22T13:29:58.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.810+0000 7fbfbc25c640 1 -- 172.21.15.5:0/3858612057 wait complete. 2024-04-22T13:29:58.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 Processor -- start 2024-04-22T13:29:58.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 -- start start 2024-04-22T13:29:58.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb4107260 0x7fbfb419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb4198620 0x7fbfb4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41121a0 con 0x7fbfb4102190 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4112020 con 0x7fbfb4107260 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfbc25c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4112320 con 0x7fbfb4198620 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35354/0 (socket says 172.21.15.5:35354) 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 -- 172.21.15.5:0/2074033133 learned_addr learned my addr 172.21.15.5:0/2074033133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb4198620 0x7fbfb4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb4107260 0x7fbfb419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 -- 172.21.15.5:0/2074033133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb4198620 msgr2=0x7fbfb4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb4198620 0x7fbfb4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 -- 172.21.15.5:0/2074033133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb4107260 msgr2=0x7fbfb419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb4107260 0x7fbfb419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 -- 172.21.15.5:0/2074033133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb4199140 con 0x7fbfb4102190 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.811+0000 7fbfb9ff8640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb419e470 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0005940 tx=0x7fbfb002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.812+0000 7fbfa6ffd640 1 -- 172.21.15.5:0/2074033133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0043020 con 0x7fbfb4102190 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.812+0000 7fbfa6ffd640 1 -- 172.21.15.5:0/2074033133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb002cbe0 con 0x7fbfb4102190 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.812+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb41993d0 con 0x7fbfb4102190 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.812+0000 7fbfa6ffd640 1 -- 172.21.15.5:0/2074033133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0003bf0 con 0x7fbfb4102190 2024-04-22T13:29:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.812+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a5670 con 0x7fbfb4102190 2024-04-22T13:29:58.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:58 smithi005 ceph-mon[28006]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-04-22T13:29:58.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:29:58 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/161329434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.814+0000 7fbfa6ffd640 1 -- 172.21.15.5:0/2074033133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbfb0046020 con 0x7fbfb4102190 2024-04-22T13:29:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.814+0000 7fbfbee55740 1 Processor -- start 2024-04-22T13:29:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 -- start start 2024-04-22T13:29:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfa6ffd640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbfa00a3da0 0x7fbfa00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fe19161f0 con 0x559fe19b1e80 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559fe18b34b0 con 0x559fe19baa00 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfbee55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559fe19121b0 con 0x559fe19b14d0 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfb97f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfb97f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35364/0 (socket says 172.21.15.5:35364) 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfb97f7640 1 -- 172.21.15.5:0/2179609884 learned_addr learned my addr 172.21.15.5:0/2179609884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfb9ff8640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbfa00a3da0 0x7fbfa00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.815+0000 7fbfa6ffd640 1 -- 172.21.15.5:0/2074033133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbfb00f28a0 con 0x7fbfb4102190 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb97f7640 1 -- 172.21.15.5:0/2179609884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 msgr2=0x559fe19b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb97f7640 1 -- 172.21.15.5:0/2179609884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 msgr2=0x559fe19bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb97f7640 1 -- 172.21.15.5:0/2179609884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe18a51b0 con 0x559fe19b1e80 2024-04-22T13:29:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb9ff8640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbfa00a3da0 0x7fbfa00a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00f01e0 tx=0x7fbfb00036f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19ba430 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800bc10 tx=0x7fbfa800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e78232048b636fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbf9ffff640 1 -- 172.21.15.5:0/2179609884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8015070 con 0x559fe19b1e80 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbf9ffff640 1 -- 172.21.15.5:0/2179609884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa801a910 con 0x559fe19b1e80 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.816+0000 7fbf9ffff640 1 -- 172.21.15.5:0/2179609884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80117d0 con 0x559fe19b1e80 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 -- 172.21.15.5:0/2179609884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 msgr2=0x559fe19ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19ba430 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800bc10 tx=0x7fbfa800f730 comp rx=0 tx=0).stop 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 -- 172.21.15.5:0/2179609884 shutdown_connections 2024-04-22T13:29:58.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19ba430 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2179609884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 -- 172.21.15.5:0/2179609884 >> 172.21.15.5:0/2179609884 conn(0x559fe19abba0 msgr2=0x559fe19abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 -- 172.21.15.5:0/2179609884 shutdown_connections 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 -- 172.21.15.5:0/2179609884 wait complete. 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 -- start start 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.817+0000 7fbfbee55740 1 Processor -- start 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b1e80 0x559fe19b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fe19bd970 con 0x559fe19baa00 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559fe18cf500 con 0x559fe19b1e80 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559fe19bdaf0 con 0x559fe19b14d0 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfbee55740 10 client.0 _ll_register_callbacks cb 0x559fe189db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35372/0 (socket says 172.21.15.5:35372) 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 -- 172.21.15.5:0/698402844 learned_addr learned my addr 172.21.15.5:0/698402844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b1e80 0x559fe19b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 -- 172.21.15.5:0/698402844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 msgr2=0x559fe19ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 -- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b1e80 msgr2=0x559fe19b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b1e80 0x559fe19b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 -- 172.21.15.5:0/698402844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe16283f0 con 0x559fe19baa00 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19b7d20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbfac013e90 tx=0x7fbfac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da8d56e7806a41c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.818+0000 7fbf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/698402844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac013050 con 0x559fe19baa00 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/698402844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbfac0131f0 con 0x559fe19baa00 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa801a910 con 0x559fe19baa00 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/698402844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfac0133f0 con 0x559fe19baa00 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 10 client.16908 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559fe16f27d0 con 0x559fe19baa00 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/698402844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa801a910 con 0x559fe19baa00 2024-04-22T13:29:58.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/698402844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbfac00f040 con 0x559fe19baa00 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 10 client.16908 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 10 client.16908 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 20 client.16908 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe19cacf0 0x559fe19cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 4 client.16908 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.819+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe18b5930 con 0x559fe19cacf0 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.820+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe19cacf0 0x559fe19cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.820+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe19cacf0 0x559fe19cd120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e737689bb08f8ec1 server_cookie=a2137c7ed25c3db in_seq=0 out_seq=0 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.820+0000 7fbf9d7fa640 10 client.16908 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/698402844 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fe18b5930 con 0x559fe19cacf0 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbf9d7fa640 10 client.16908 handle_command_reply: tid=0 2024-04-22T13:29:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 client.16908 shutdown 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 msgr2=0x559fe19b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19b7d20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbfac013e90 tx=0x7fbfac004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 shutdown_connections 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe19cacf0 0x559fe19cd120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19b7d20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b1e80 0x559fe19b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbf9d7fa640 0 client.16908 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 --2- 172.21.15.5:0/698402844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 >> 172.21.15.5:0/698402844 conn(0x559fe19abba0 msgr2=0x559fe19ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 shutdown_connections 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 1 -- 172.21.15.5:0/698402844 wait complete. 2024-04-22T13:29:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.821+0000 7fbfbee55740 20 client.16908 trim_cache size 0 max 16384 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 Processor -- start 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 -- start start 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41adbb0 con 0x559fe19b1e80 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41af6f0 con 0x559fe19baa00 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfbee55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41ae280 con 0x559fe19b14d0 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfb97f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfb97f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19b9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35388/0 (socket says 172.21.15.5:35388) 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfb97f7640 1 -- 172.21.15.5:0/3068572888 learned_addr learned my addr 172.21.15.5:0/3068572888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.835+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfb97f7640 1 -- 172.21.15.5:0/3068572888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 msgr2=0x559fe19c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfb97f7640 1 -- 172.21.15.5:0/3068572888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 msgr2=0x559fe19ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfb97f7640 1 -- 172.21.15.5:0/3068572888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe19b1c20 con 0x559fe19b1e80 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19b9e40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800bc10 tx=0x7fbfa8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c958778a791004 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbf9f7fe640 1 -- 172.21.15.5:0/3068572888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa802c070 con 0x559fe19b1e80 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbf9f7fe640 1 -- 172.21.15.5:0/3068572888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa80054f0 con 0x559fe19b1e80 2024-04-22T13:29:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.836+0000 7fbf9f7fe640 1 -- 172.21.15.5:0/3068572888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8015040 con 0x559fe19b1e80 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 -- 172.21.15.5:0/3068572888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 msgr2=0x559fe19b9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19b9e40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800bc10 tx=0x7fbfa8000bc0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 -- 172.21.15.5:0/3068572888 shutdown_connections 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19b1e80 0x559fe19b9e40 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19baa00 0x559fe19ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 --2- 172.21.15.5:0/3068572888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b14d0 0x559fe19c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 -- 172.21.15.5:0/3068572888 >> 172.21.15.5:0/3068572888 conn(0x559fe19abba0 msgr2=0x559fe19b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 -- 172.21.15.5:0/3068572888 shutdown_connections 2024-04-22T13:29:58.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 -- 172.21.15.5:0/3068572888 wait complete. 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 -- start start 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.837+0000 7fbfbee55740 1 Processor -- start 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b14d0 0x559fe19c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b1e80 0x559fe19be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41af020 con 0x559fe19baa00 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41ae950 con 0x559fe19b14d0 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41abbb0 con 0x559fe19b1e80 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfbee55740 10 client.0 _ll_register_callbacks cb 0x7fbfac013b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35394/0 (socket says 172.21.15.5:35394) 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 -- 172.21.15.5:0/2443211698 learned_addr learned my addr 172.21.15.5:0/2443211698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b1e80 0x559fe19be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b14d0 0x559fe19c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 -- 172.21.15.5:0/2443211698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b1e80 msgr2=0x559fe19be400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b1e80 0x559fe19be400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 -- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b14d0 msgr2=0x559fe19c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b14d0 0x559fe19c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 -- 172.21.15.5:0/2443211698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe19bf140 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbfba7f9640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19be940 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbfac014fd0 tx=0x7fbfac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c86470c1544bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/2443211698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac02d040 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.838+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/2443211698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbfac004ab0 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa80054f0 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/2443211698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfac013200 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 10 client.16914 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559fe1a311a0 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/2443211698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa80054f0 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/2443211698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbfac033070 con 0x559fe19baa00 2024-04-22T13:29:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 10 client.16914 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 20 client.16914 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe1a31800 0x559fe1a33c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 4 client.16914 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fe1923c10 con 0x559fe1a31800 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.839+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe1a31800 0x559fe1a33c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfb97f7640 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe1a31800 0x559fe1a33c20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f39973f8f0b7bcfc server_cookie=9884bc2c02ab50b5 in_seq=0 out_seq=0 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbf9d7fa640 10 client.16914 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbf9d7fa640 1 -- 172.21.15.5:0/2443211698 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fe1923c10 con 0x559fe1a31800 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbf9d7fa640 10 client.16914 handle_command_reply: tid=0 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 client.16914 shutdown 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 msgr2=0x559fe19be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19be940 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbfac014fd0 tx=0x7fbfac004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:58.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 shutdown_connections 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x559fe1a31800 0x559fe1a33c20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fe19baa00 0x559fe19be940 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe19b1e80 0x559fe19be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbf9d7fa640 0 client.16914 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 --2- 172.21.15.5:0/2443211698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fe19b14d0 0x559fe19c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 >> 172.21.15.5:0/2443211698 conn(0x559fe19abba0 msgr2=0x559fe19b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 shutdown_connections 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 1 -- 172.21.15.5:0/2443211698 wait complete. 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.840+0000 7fbfbee55740 20 client.16914 trim_cache size 0 max 16384 2024-04-22T13:29:58.842 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:58.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbfa00a3da0 msgr2=0x7fbfa00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbfa00a3da0 0x7fbfa00a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00f01e0 tx=0x7fbfb00036f0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 msgr2=0x7fbfb419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb419e470 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0005940 tx=0x7fbfb002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 shutdown_connections 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbfa00a3da0 0x7fbfa00a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb4198620 0x7fbfb4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb4107260 0x7fbfb419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 --2- 172.21.15.5:0/2074033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4102190 0x7fbfb419e470 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 >> 172.21.15.5:0/2074033133 conn(0x7fbfb40fda10 msgr2=0x7fbfb40fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 shutdown_connections 2024-04-22T13:29:58.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.841+0000 7fbfbc25c640 1 -- 172.21.15.5:0/2074033133 wait complete. 2024-04-22T13:29:58.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:29:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 -- 172.21.15.5:0/3031019045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90102540 msgr2=0x7fad90105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 --2- 172.21.15.5:0/3031019045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90102540 0x7fad90105ae0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009900 tx=0x7fad7c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 -- 172.21.15.5:0/3031019045 shutdown_connections 2024-04-22T13:29:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 --2- 172.21.15.5:0/3031019045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90102540 0x7fad90105ae0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 --2- 172.21.15.5:0/3031019045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad90106210 0x7fad9010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 --2- 172.21.15.5:0/3031019045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad90101b90 0x7fad90101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 -- 172.21.15.5:0/3031019045 >> 172.21.15.5:0/3031019045 conn(0x7fad90076170 msgr2=0x7fad90076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 -- 172.21.15.5:0/3031019045 shutdown_connections 2024-04-22T13:29:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.964+0000 7fad96dc7640 1 -- 172.21.15.5:0/3031019045 wait complete. 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 Processor -- start 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 -- start start 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 0x7fad90198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad90106210 0x7fad90198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad9019cfe0 0x7fad9019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad90112200 con 0x7fad90101b90 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad90112080 con 0x7fad90106210 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.965+0000 7fad96dc7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad90112380 con 0x7fad9019cfe0 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 0x7fad90198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 0x7fad90198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35418/0 (socket says 172.21.15.5:35418) 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 -- 172.21.15.5:0/2765705130 learned_addr learned my addr 172.21.15.5:0/2765705130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad8ffff640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad90106210 0x7fad90198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad95364640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad9019cfe0 0x7fad9019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 -- 172.21.15.5:0/2765705130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad9019cfe0 msgr2=0x7fad9019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad9019cfe0 0x7fad9019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 -- 172.21.15.5:0/2765705130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad90106210 msgr2=0x7fad90198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad90106210 0x7fad90198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 -- 172.21.15.5:0/2765705130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9019dad0 con 0x7fad90101b90 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad94b63640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 0x7fad90198720 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9f0 tx=0x7fad8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad8dffb640 1 -- 172.21.15.5:0/2765705130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8400cd80 con 0x7fad90101b90 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad8dffb640 1 -- 172.21.15.5:0/2765705130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad840108e0 con 0x7fad90101b90 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9019ddc0 con 0x7fad90101b90 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.966+0000 7fad8dffb640 1 -- 172.21.15.5:0/2765705130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84018980 con 0x7fad90101b90 2024-04-22T13:29:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.967+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad901a5580 con 0x7fad90101b90 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad8dffb640 1 -- 172.21.15.5:0/2765705130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fad84010430 con 0x7fad90101b90 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 Processor -- start 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 -- start start 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc9734d0 0x558edc9738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc973e80 0x558edc97c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558edc8d81f0 con 0x558edc97ca00 2024-04-22T13:29:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558edc8754b0 con 0x558edc9734d0 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad999bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558edc8d41b0 con 0x558edc973e80 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35420/0 (socket says 172.21.15.5:35420) 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad8ffff640 1 -- 172.21.15.5:0/2634172426 learned_addr learned my addr 172.21.15.5:0/2634172426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad94b63640 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc973e80 0x558edc97c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.970+0000 7fad95364640 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc9734d0 0x558edc9738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8ffff640 1 -- 172.21.15.5:0/2634172426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc973e80 msgr2=0x558edc97c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8ffff640 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc973e80 0x558edc97c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8ffff640 1 -- 172.21.15.5:0/2634172426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc9734d0 msgr2=0x558edc9738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8ffff640 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc9734d0 0x558edc9738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8ffff640 1 -- 172.21.15.5:0/2634172426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edc8671b0 con 0x558edc97ca00 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8dffb640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fad800a3d20 0x7fad800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad95364640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fad800a3d20 0x7fad800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8ffff640 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97ee50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fad90077740 tx=0x7fad7c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71eecf35a3120313 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad8dffb640 1 -- 172.21.15.5:0/2765705130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fad840d0b60 con 0x7fad90101b90 2024-04-22T13:29:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad7affd640 1 -- 172.21.15.5:0/2634172426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c045020 con 0x558edc97ca00 2024-04-22T13:29:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad7affd640 1 -- 172.21.15.5:0/2634172426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad7c02c7e0 con 0x558edc97ca00 2024-04-22T13:29:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.971+0000 7fad7affd640 1 -- 172.21.15.5:0/2634172426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0054b0 con 0x558edc97ca00 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 -- 172.21.15.5:0/2634172426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 msgr2=0x558edc97ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad95364640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fad800a3d20 0x7fad800a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad88005fd0 tx=0x7fad88009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97ee50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fad90077740 tx=0x7fad7c0095f0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 -- 172.21.15.5:0/2634172426 shutdown_connections 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97ee50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc973e80 0x558edc97c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 --2- 172.21.15.5:0/2634172426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc9734d0 0x558edc9738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 -- 172.21.15.5:0/2634172426 >> 172.21.15.5:0/2634172426 conn(0x558edc96dba0 msgr2=0x558edc96df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 -- 172.21.15.5:0/2634172426 shutdown_connections 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 -- 172.21.15.5:0/2634172426 wait complete. 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 -- start start 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.972+0000 7fad999bc740 1 Processor -- start 2024-04-22T13:29:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc97c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 0x558edc979630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc97ca00 0x558edc979b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558edc97f910 con 0x558edc973e80 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558edc891500 con 0x558edc97ca00 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558edc97fa90 con 0x558edc9734d0 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad999bc740 10 client.0 _ll_register_callbacks cb 0x558edc85fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 0x558edc979630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 0x558edc979630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35436/0 (socket says 172.21.15.5:35436) 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 -- 172.21.15.5:0/1283633558 learned_addr learned my addr 172.21.15.5:0/1283633558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad94b63640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc97c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad95364640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc97ca00 0x558edc979b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 -- 172.21.15.5:0/1283633558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 msgr2=0x558edc97c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc97c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 -- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc97ca00 msgr2=0x558edc979b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc97ca00 0x558edc979b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.973+0000 7fad8ffff640 1 -- 172.21.15.5:0/1283633558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edc5ea3f0 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad8ffff640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 0x558edc979630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fad7c005ec0 tx=0x7fad7c02c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7349eaffa6a06adb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad6ffff640 1 -- 172.21.15.5:0/1283633558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c04e030 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7c02c7e0 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad6ffff640 1 -- 172.21.15.5:0/1283633558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fad7c004250 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad6ffff640 1 -- 172.21.15.5:0/1283633558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad7c058050 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad999bc740 10 client.16926 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558edc6b47d0 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.974+0000 7fad6ffff640 1 -- 172.21.15.5:0/1283633558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad7c02c7e0 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad6ffff640 1 -- 172.21.15.5:0/1283633558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fad7c0046c0 con 0x558edc973e80 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad999bc740 10 client.16926 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad999bc740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad999bc740 20 client.16926 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad999bc740 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc98c9e0 0x558edc98edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad999bc740 4 client.16926 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558edc877930 con 0x558edc98c9e0 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad94b63640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc98c9e0 0x558edc98edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad6ffff640 10 client.16926 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.975+0000 7fad94b63640 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc98c9e0 0x558edc98edd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=766cb302db080996 server_cookie=3ef39045d53c4a1b in_seq=0 out_seq=0 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad6ffff640 1 -- 172.21.15.5:0/1283633558 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558edc877930 con 0x558edc98c9e0 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad6ffff640 10 client.16926 handle_command_reply: tid=0 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 client.16926 shutdown 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 msgr2=0x558edc979630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 0x558edc979630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fad7c005ec0 tx=0x7fad7c02c3e0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 shutdown_connections 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc98c9e0 0x558edc98edd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc973e80 0x558edc979630 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc97ca00 0x558edc979b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad6ffff640 0 client.16926 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 --2- 172.21.15.5:0/1283633558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc97c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 >> 172.21.15.5:0/1283633558 conn(0x558edc96dba0 msgr2=0x558edc96f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 shutdown_connections 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 1 -- 172.21.15.5:0/1283633558 wait complete. 2024-04-22T13:29:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.976+0000 7fad999bc740 20 client.16926 trim_cache size 0 max 16384 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 Processor -- start 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 -- start start 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc9887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc97bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad901ada60 con 0x558edc97ca00 2024-04-22T13:29:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad901af5a0 con 0x558edc973e80 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad999bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad901ae130 con 0x558edc9734d0 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.990+0000 7fad95364640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35452/0 (socket says 172.21.15.5:35452) 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 -- 172.21.15.5:0/2252926609 learned_addr learned my addr 172.21.15.5:0/2252926609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad94b63640 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc9887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad8ffff640 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc97bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 -- 172.21.15.5:0/2252926609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 msgr2=0x558edc9887d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc9887d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 -- 172.21.15.5:0/2252926609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 msgr2=0x558edc97bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc97bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 -- 172.21.15.5:0/2252926609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edc977b80 con 0x558edc97ca00 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad94b63640 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc9887d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad95364640 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97c2a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fad88012820 tx=0x7fad88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311075e91b1905fa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad7a7fc640 1 -- 172.21.15.5:0/2252926609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88012e60 con 0x558edc97ca00 2024-04-22T13:29:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad7a7fc640 1 -- 172.21.15.5:0/2252926609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad88004d20 con 0x558edc97ca00 2024-04-22T13:29:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.991+0000 7fad7a7fc640 1 -- 172.21.15.5:0/2252926609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad880058c0 con 0x558edc97ca00 2024-04-22T13:29:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 -- 172.21.15.5:0/2252926609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 msgr2=0x558edc97c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97c2a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fad88012820 tx=0x7fad88014000 comp rx=0 tx=0).stop 2024-04-22T13:29:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 -- 172.21.15.5:0/2252926609 shutdown_connections 2024-04-22T13:29:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc97ca00 0x558edc97c2a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc97bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 --2- 172.21.15.5:0/2252926609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc9734d0 0x558edc9887d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 -- 172.21.15.5:0/2252926609 >> 172.21.15.5:0/2252926609 conn(0x558edc96dba0 msgr2=0x558edc96cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 -- 172.21.15.5:0/2252926609 shutdown_connections 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 -- 172.21.15.5:0/2252926609 wait complete. 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.992+0000 7fad999bc740 1 -- start start 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 Processor -- start 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 0x558edc980830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc980d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc97ca00 0x558edc9812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad901aeed0 con 0x558edc9734d0 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fad901ae800 con 0x558edc973e80 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad901aba60 con 0x558edc97ca00 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad999bc740 10 client.0 _ll_register_callbacks cb 0x7fad7c02ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 0x558edc980830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 0x558edc980830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35462/0 (socket says 172.21.15.5:35462) 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 -- 172.21.15.5:0/3691858165 learned_addr learned my addr 172.21.15.5:0/3691858165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad95364640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc97ca00 0x558edc9812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad8ffff640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc980d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 -- 172.21.15.5:0/3691858165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc97ca00 msgr2=0x558edc9812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc97ca00 0x558edc9812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 -- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 msgr2=0x558edc980d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc980d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad94b63640 1 -- 172.21.15.5:0/3691858165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edc9f1e90 con 0x558edc9734d0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad95364640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc97ca00 0x558edc9812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.993+0000 7fad8ffff640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc980d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad94b63640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 0x558edc980830 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fad8409c830 tx=0x7fad840d46b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb634a507beb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad6ffff640 1 -- 172.21.15.5:0/3691858165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8409de60 con 0x558edc9734d0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad6ffff640 1 -- 172.21.15.5:0/3691858165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fad84010a90 con 0x558edc9734d0 2024-04-22T13:29:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad88004d20 con 0x558edc9734d0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad6ffff640 1 -- 172.21.15.5:0/3691858165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad8409d230 con 0x558edc9734d0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 10 client.16932 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558edc9f31d0 con 0x558edc9734d0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad6ffff640 1 -- 172.21.15.5:0/3691858165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad88004d20 con 0x558edc9734d0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad6ffff640 1 -- 172.21.15.5:0/3691858165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fad840ce560 con 0x558edc9734d0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 10 client.16932 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 20 client.16932 populate_metadata read hostname 'smithi005' 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc9f38a0 0x558edc9f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 4 client.16932 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.994+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558edc8e5c10 con 0x558edc9f38a0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad8ffff640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc9f38a0 0x558edc9f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad6ffff640 10 client.16932 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad8ffff640 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc9f38a0 0x558edc9f5cc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9729a57901b92693 server_cookie=33b7019d98bac85f in_seq=0 out_seq=0 2024-04-22T13:29:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad6ffff640 1 -- 172.21.15.5:0/3691858165 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558edc8e5c10 con 0x558edc9f38a0 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad6ffff640 10 client.16932 handle_command_reply: tid=0 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad999bc740 1 client.16932 shutdown 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 msgr2=0x558edc980830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad999bc740 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 0x558edc980830 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fad8409c830 tx=0x7fad840d46b0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 shutdown_connections 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad999bc740 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558edc9f38a0 0x558edc9f5cc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.995+0000 7fad999bc740 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558edc97ca00 0x558edc9812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad6ffff640 0 client.16932 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad999bc740 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558edc973e80 0x558edc980d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad999bc740 1 --2- 172.21.15.5:0/3691858165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558edc9734d0 0x558edc980830 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 >> 172.21.15.5:0/3691858165 conn(0x558edc96dba0 msgr2=0x558edc96c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 shutdown_connections 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad999bc740 1 -- 172.21.15.5:0/3691858165 wait complete. 2024-04-22T13:29:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad999bc740 20 client.16932 trim_cache size 0 max 16384 2024-04-22T13:29:58.998 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:29:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fad800a3d20 msgr2=0x7fad800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fad800a3d20 0x7fad800a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad88005fd0 tx=0x7fad88009500 comp rx=0 tx=0).stop 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 msgr2=0x7fad90198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 0x7fad90198720 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9f0 tx=0x7fad8400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 shutdown_connections 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fad800a3d20 0x7fad800a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad9019cfe0 0x7fad9019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad90106210 0x7fad90198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 --2- 172.21.15.5:0/2765705130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad90101b90 0x7fad90198720 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.996+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 >> 172.21.15.5:0/2765705130 conn(0x7fad90076170 msgr2=0x7fad900feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.997+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 shutdown_connections 2024-04-22T13:29:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:29:58.997+0000 7fad96dc7640 1 -- 172.21.15.5:0/2765705130 wait complete. 2024-04-22T13:29:59.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:58 smithi049 ceph-mon[31379]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-04-22T13:29:59.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:29:58 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/161329434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:59.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:58 smithi107 ceph-mon[31859]: pgmap v88: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-04-22T13:29:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:29:58 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/161329434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:00.004 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:30:00.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:30:00.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 -- 172.21.15.5:0/1136017423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4103b00 msgr2=0x7f15d410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 --2- 172.21.15.5:0/1136017423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4103b00 0x7f15d410bfe0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009900 tx=0x7f15d002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 -- 172.21.15.5:0/1136017423 shutdown_connections 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 --2- 172.21.15.5:0/1136017423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d4103b00 0x7f15d410bfe0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 --2- 172.21.15.5:0/1136017423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d410c520 0x7f15d410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 --2- 172.21.15.5:0/1136017423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15d41031e0 0x7f15d41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 -- 172.21.15.5:0/1136017423 >> 172.21.15.5:0/1136017423 conn(0x7f15d4078000 msgr2=0x7f15d4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 -- 172.21.15.5:0/1136017423 shutdown_connections 2024-04-22T13:30:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 -- 172.21.15.5:0/1136017423 wait complete. 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.123+0000 7f15dc42e640 1 Processor -- start 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 -- start start 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15d41031e0 0x7f15d4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4103b00 0x7f15d4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 0x7f15d41018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4112180 con 0x7f15d410c520 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4112000 con 0x7f15d4103b00 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15dc42e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4112300 con 0x7f15d41031e0 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 0x7f15d41018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 0x7f15d41018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35488/0 (socket says 172.21.15.5:35488) 2024-04-22T13:30:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 -- 172.21.15.5:0/3814208615 learned_addr learned my addr 172.21.15.5:0/3814208615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da1ca640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15d41031e0 0x7f15d4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15d99c9640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4103b00 0x7f15d4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 -- 172.21.15.5:0/3814208615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15d41031e0 msgr2=0x7f15d4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15d41031e0 0x7f15d4100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 -- 172.21.15.5:0/3814208615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4103b00 msgr2=0x7f15d4101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4103b00 0x7f15d4101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.124+0000 7f15da9cb640 1 -- 172.21.15.5:0/3814208615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41a6c80 con 0x7f15d410c520 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.125+0000 7f15da9cb640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 0x7f15d41018b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00e960 tx=0x7f15cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.125+0000 7f15c77fe640 1 -- 172.21.15.5:0/3814208615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc004d20 con 0x7f15d410c520 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.125+0000 7f15c77fe640 1 -- 172.21.15.5:0/3814208615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc005020 con 0x7f15d410c520 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.125+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41a6f70 con 0x7f15d410c520 2024-04-22T13:30:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.125+0000 7f15c77fe640 1 -- 172.21.15.5:0/3814208615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0116d0 con 0x7f15d410c520 2024-04-22T13:30:00.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.125+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41a7440 con 0x7f15d410c520 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.127+0000 7f15c77fe640 1 -- 172.21.15.5:0/3814208615 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f15cc0051d0 con 0x7f15d410c520 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 Processor -- start 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 -- start start 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15c77fe640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f15c00a3d20 0x7f15c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db37ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db385840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db388260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9db2e0f00 con 0x55b9db37d290 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b9db27e590 con 0x55b9db385e10 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15df016740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9db2dcec0 con 0x55b9db37c8e0 2024-04-22T13:30:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15c77fe640 1 -- 172.21.15.5:0/3814208615 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f15cc0d0890 con 0x7f15d410c520 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15d99c9640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f15c00a3d20 0x7f15c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.128+0000 7f15da1ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db37ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db388260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db388260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:40746/0 (socket says 172.21.15.5:40746) 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 -- 172.21.15.5:0/1973425187 learned_addr learned my addr 172.21.15.5:0/1973425187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15d99c9640 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db385840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 -- 172.21.15.5:0/1973425187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 msgr2=0x55b9db37ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db37ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 -- 172.21.15.5:0/1973425187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 msgr2=0x55b9db385840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db385840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 -- 172.21.15.5:0/1973425187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9db0bb7e0 con 0x55b9db385e10 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da1ca640 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db37ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15d99c9640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f15c00a3d20 0x7f15c00a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f15d41028c0 tx=0x7f15d002c6f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15da9cb640 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db388260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15cc099230 tx=0x7f15cc09c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd36ba1953103adc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15c4bfe640 1 -- 172.21.15.5:0/1973425187 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc01f070 con 0x55b9db385e10 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15c4bfe640 1 -- 172.21.15.5:0/1973425187 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc09cf60 con 0x55b9db385e10 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.129+0000 7f15c4bfe640 1 -- 172.21.15.5:0/1973425187 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0a54d0 con 0x55b9db385e10 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 -- 172.21.15.5:0/1973425187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 msgr2=0x55b9db388260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db388260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15cc099230 tx=0x7f15cc09c740 comp rx=0 tx=0).stop 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 -- 172.21.15.5:0/1973425187 shutdown_connections 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db385840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db388260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 --2- 172.21.15.5:0/1973425187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db37ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 -- 172.21.15.5:0/1973425187 >> 172.21.15.5:0/1973425187 conn(0x55b9db376fb0 msgr2=0x55b9db377390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 -- 172.21.15.5:0/1973425187 shutdown_connections 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 -- 172.21.15.5:0/1973425187 wait complete. 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.130+0000 7f15df016740 1 -- start start 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 Processor -- start 2024-04-22T13:30:00.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db37c8e0 0x55b9db382dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db383300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 0x55b9db383840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9db388d80 con 0x55b9db385e10 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b9db29a580 con 0x55b9db37c8e0 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9db388f00 con 0x55b9db37d290 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15df016740 10 client.0 _ll_register_callbacks cb 0x55b9db2a0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 0x55b9db383840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 0x55b9db383840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35516/0 (socket says 172.21.15.5:35516) 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 -- 172.21.15.5:0/2418227391 learned_addr learned my addr 172.21.15.5:0/2418227391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da1ca640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db37c8e0 0x55b9db382dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15d99c9640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db383300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 -- 172.21.15.5:0/2418227391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 msgr2=0x55b9db383300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db383300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 -- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db37c8e0 msgr2=0x55b9db382dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db37c8e0 0x55b9db382dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 -- 172.21.15.5:0/2418227391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9db11bf70 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15da9cb640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 0x55b9db383840 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00efd0 tx=0x7f15cc0a0c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907035ba2bf01577 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.131+0000 7f15bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15bdffb640 1 -- 172.21.15.5:0/2418227391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc01f070 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15bdffb640 1 -- 172.21.15.5:0/2418227391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f15cc098ed0 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc09cf60 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15bdffb640 1 -- 172.21.15.5:0/2418227391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15cc0a54d0 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 10 client.16944 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9dafb3170 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15bdffb640 1 -- 172.21.15.5:0/2418227391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc09cf60 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15bdffb640 1 -- 172.21.15.5:0/2418227391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f15cc00f040 con 0x55b9db385e10 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 10 client.16944 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 20 client.16944 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3961c0 0x55b9db3985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 4 client.16944 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.132+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9db2ee920 con 0x55b9db3961c0 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.133+0000 7f15da1ca640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3961c0 0x55b9db3985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.133+0000 7f15da1ca640 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3961c0 0x55b9db3985b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f473c4b06720f967 server_cookie=ca97d405416538be in_seq=0 out_seq=0 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.133+0000 7f15bdffb640 10 client.16944 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15bdffb640 1 -- 172.21.15.5:0/2418227391 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9db2ee920 con 0x55b9db3961c0 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15bdffb640 10 client.16944 handle_command_reply: tid=0 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 client.16944 shutdown 2024-04-22T13:30:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 msgr2=0x55b9db383840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 0x55b9db383840 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00efd0 tx=0x7f15cc0a0c80 comp rx=0 tx=0).stop 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 shutdown_connections 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3961c0 0x55b9db3985b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db385e10 0x55b9db383840 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15bdffb640 0 client.16944 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db383300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 --2- 172.21.15.5:0/2418227391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db37c8e0 0x55b9db382dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 >> 172.21.15.5:0/2418227391 conn(0x55b9db376fb0 msgr2=0x55b9db378d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 shutdown_connections 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 1 -- 172.21.15.5:0/2418227391 wait complete. 2024-04-22T13:30:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.134+0000 7f15df016740 20 client.16944 trim_cache size 0 max 16384 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 Processor -- start 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 -- start start 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db3854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db381c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db39b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41afe10 con 0x55b9db37d290 2024-04-22T13:30:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b1950 con 0x55b9db385e10 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.152+0000 7f15df016740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b04e0 con 0x55b9db37c8e0 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db381c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db381c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35526/0 (socket says 172.21.15.5:35526) 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 -- 172.21.15.5:0/1397139189 learned_addr learned my addr 172.21.15.5:0/1397139189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15da9cb640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db39b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15da1ca640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db3854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 -- 172.21.15.5:0/1397139189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 msgr2=0x55b9db3854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db3854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 -- 172.21.15.5:0/1397139189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 msgr2=0x55b9db39b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db39b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 -- 172.21.15.5:0/1397139189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9db381310 con 0x55b9db37d290 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15da1ca640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db3854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15da9cb640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db39b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15d99c9640 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db381c70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005770 tx=0x7f15d0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d042cf8865e7cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15bffff640 1 -- 172.21.15.5:0/1397139189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00051c0 con 0x55b9db37d290 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.153+0000 7f15bffff640 1 -- 172.21.15.5:0/1397139189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d0053070 con 0x55b9db37d290 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15bffff640 1 -- 172.21.15.5:0/1397139189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00035f0 con 0x55b9db37d290 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 -- 172.21.15.5:0/1397139189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 msgr2=0x55b9db381c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db381c70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005770 tx=0x7f15d0046000 comp rx=0 tx=0).stop 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 -- 172.21.15.5:0/1397139189 shutdown_connections 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37d290 0x55b9db381c70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db39b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 --2- 172.21.15.5:0/1397139189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37c8e0 0x55b9db3854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 -- 172.21.15.5:0/1397139189 >> 172.21.15.5:0/1397139189 conn(0x55b9db376fb0 msgr2=0x55b9db381560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 -- 172.21.15.5:0/1397139189 shutdown_connections 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.154+0000 7f15df016740 1 -- 172.21.15.5:0/1397139189 wait complete. 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 -- start start 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 Processor -- start 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 0x55b9db399080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db3995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db399b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b1280 con 0x55b9db37c8e0 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b0bb0 con 0x55b9db385e10 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41addf0 con 0x55b9db37d290 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15df016740 10 client.0 _ll_register_callbacks cb 0x7f15d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 0x55b9db399080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 0x55b9db399080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35534/0 (socket says 172.21.15.5:35534) 2024-04-22T13:30:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 -- 172.21.15.5:0/499898190 learned_addr learned my addr 172.21.15.5:0/499898190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15d99c9640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db3995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da9cb640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db399b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 -- 172.21.15.5:0/499898190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 msgr2=0x55b9db3995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db3995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 -- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 msgr2=0x55b9db399b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db399b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da1ca640 1 -- 172.21.15.5:0/499898190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9db3fb3f0 con 0x55b9db37c8e0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.155+0000 7f15da9cb640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db399b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15da1ca640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 0x55b9db399080 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x55b9db388f80 tx=0x7f15c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bf147418b59e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15bdffb640 1 -- 172.21.15.5:0/499898190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c80204f0 con 0x55b9db37c8e0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15bdffb640 1 -- 172.21.15.5:0/499898190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f15c8020690 con 0x55b9db37c8e0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d0053070 con 0x55b9db37c8e0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15bdffb640 1 -- 172.21.15.5:0/499898190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15c802f050 con 0x55b9db37c8e0 2024-04-22T13:30:00.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 10 client.16950 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9db3fc790 con 0x55b9db37c8e0 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15bdffb640 1 -- 172.21.15.5:0/499898190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d0053070 con 0x55b9db37c8e0 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15bdffb640 1 -- 172.21.15.5:0/499898190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f15c8010430 con 0x55b9db37c8e0 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 10 client.16950 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 20 client.16950 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3fcf40 0x55b9db3ff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 4 client.16950 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:30:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.156+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b9db2e8790 con 0x55b9db3fcf40 2024-04-22T13:30:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.157+0000 7f15d99c9640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3fcf40 0x55b9db3ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.157+0000 7f15bdffb640 10 client.16950 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.157+0000 7f15d99c9640 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3fcf40 0x55b9db3ff330 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3bae1bff5b016e server_cookie=a9640a22281d0ec5 in_seq=0 out_seq=0 2024-04-22T13:30:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15bdffb640 1 -- 172.21.15.5:0/499898190 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b9db2e8790 con 0x55b9db3fcf40 2024-04-22T13:30:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15bdffb640 10 client.16950 handle_command_reply: tid=0 2024-04-22T13:30:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 client.16950 shutdown 2024-04-22T13:30:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 msgr2=0x55b9db399080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 0x55b9db399080 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x55b9db388f80 tx=0x7f15c80118d0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 shutdown_connections 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9db3fcf40 0x55b9db3ff330 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15bdffb640 0 client.16950 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9db385e10 0x55b9db399b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9db37d290 0x55b9db3995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 --2- 172.21.15.5:0/499898190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9db37c8e0 0x55b9db399080 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.158+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 >> 172.21.15.5:0/499898190 conn(0x55b9db376fb0 msgr2=0x55b9db3919e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 shutdown_connections 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15df016740 1 -- 172.21.15.5:0/499898190 wait complete. 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15df016740 20 client.16950 trim_cache size 0 max 16384 2024-04-22T13:30:00.161 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:30:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:30:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "19277220-82a0-430c-8695-a533993be220", 2024-04-22T13:30:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:30:00.162 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f15c00a3d20 msgr2=0x7f15c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f15c00a3d20 0x7f15c00a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f15d41028c0 tx=0x7f15d002c6f0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 msgr2=0x7f15d41018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 0x7f15d41018b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00e960 tx=0x7f15cc00ee20 comp rx=0 tx=0).stop 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 shutdown_connections 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d410c520 0x7f15d41018b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f15c00a3d20 0x7f15c00a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15d4103b00 0x7f15d4101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 --2- 172.21.15.5:0/3814208615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15d41031e0 0x7f15d4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.159+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 >> 172.21.15.5:0/3814208615 conn(0x7f15d4078000 msgr2=0x7f15d40ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.160+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 shutdown_connections 2024-04-22T13:30:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.160+0000 7f15dc42e640 1 -- 172.21.15.5:0/3814208615 wait complete. 2024-04-22T13:30:00.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:00.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 -- 172.21.15.5:0/2533918624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14101021a0 msgr2=0x7f1410102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 --2- 172.21.15.5:0/2533918624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14101021a0 0x7f1410102580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f140402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 -- 172.21.15.5:0/2533918624 shutdown_connections 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 --2- 172.21.15.5:0/2533918624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1410107270 0x7f141010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 --2- 172.21.15.5:0/2533918624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1410102b50 0x7f1410106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 --2- 172.21.15.5:0/2533918624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14101021a0 0x7f1410102580 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 -- 172.21.15.5:0/2533918624 >> 172.21.15.5:0/2533918624 conn(0x7f14100fd9c0 msgr2=0x7f14100ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 -- 172.21.15.5:0/2533918624 shutdown_connections 2024-04-22T13:30:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.281+0000 7f14170c0640 1 -- 172.21.15.5:0/2533918624 wait complete. 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 Processor -- start 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 -- start start 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14101021a0 0x7f1410198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1410107270 0x7f1410198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 0x7f141019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14101121e0 con 0x7f141019cfd0 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1410112060 con 0x7f1410107270 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f14170c0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1410112360 con 0x7f14101021a0 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 0x7f141019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 0x7f141019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35554/0 (socket says 172.21.15.5:35554) 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 -- 172.21.15.5:0/1605162498 learned_addr learned my addr 172.21.15.5:0/1605162498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14101021a0 0x7f1410198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f140ffff640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1410107270 0x7f1410198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 -- 172.21.15.5:0/1605162498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14101021a0 msgr2=0x7f1410198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14101021a0 0x7f1410198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 -- 172.21.15.5:0/1605162498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1410107270 msgr2=0x7f1410198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1410107270 0x7f1410198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.282+0000 7f141565d640 1 -- 172.21.15.5:0/1605162498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019dac0 con 0x7f141019cfd0 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.283+0000 7f141565d640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 0x7f141019d480 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f140800eb10 tx=0x7f140800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.283+0000 7f140dffb640 1 -- 172.21.15.5:0/1605162498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408004b20 con 0x7f141019cfd0 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.283+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019ddb0 con 0x7f141019cfd0 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.283+0000 7f140dffb640 1 -- 172.21.15.5:0/1605162498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1408004cc0 con 0x7f141019cfd0 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.283+0000 7f140dffb640 1 -- 172.21.15.5:0/1605162498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408010690 con 0x7f141019cfd0 2024-04-22T13:30:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.283+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101a55f0 con 0x7f141019cfd0 2024-04-22T13:30:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 Processor -- start 2024-04-22T13:30:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f140dffb640 1 -- 172.21.15.5:0/1605162498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1408005020 con 0x7f141019cfd0 2024-04-22T13:30:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 -- start start 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x559384469960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 0x559384472540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x559384474e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593843ce1e0 con 0x559384469ea0 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55938436b390 con 0x559384469580 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1419d82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5593843ca1a0 con 0x559384472a80 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1414e5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 0x559384472540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1414e5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 0x559384472540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35566/0 (socket says 172.21.15.5:35566) 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f1414e5c640 1 -- 172.21.15.5:0/4265960924 learned_addr learned my addr 172.21.15.5:0/4265960924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f140ffff640 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x559384469960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f140dffb640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f14000a3d20 0x7f14000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.286+0000 7f140ffff640 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x559384474e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f1414e5c640 1 -- 172.21.15.5:0/4265960924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 msgr2=0x559384474e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f1414e5c640 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x559384474e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f1414e5c640 1 -- 172.21.15.5:0/4265960924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 msgr2=0x559384469960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f1414e5c640 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x559384469960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f1414e5c640 1 -- 172.21.15.5:0/4265960924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593841aa7d0 con 0x559384469ea0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f140ffff640 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x559384469960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f141565d640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f14000a3d20 0x7f14000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f140dffb640 1 -- 172.21.15.5:0/1605162498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f14080d0ca0 con 0x7f141019cfd0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f1414e5c640 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 0x559384472540 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f140402c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5bcea9834e3d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f13faffd640 1 -- 172.21.15.5:0/4265960924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404045020 con 0x559384469ea0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f13faffd640 1 -- 172.21.15.5:0/4265960924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f140402c7e0 con 0x559384469ea0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.287+0000 7f13faffd640 1 -- 172.21.15.5:0/4265960924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14040046c0 con 0x559384469ea0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 -- 172.21.15.5:0/4265960924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 msgr2=0x559384472540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f141565d640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f14000a3d20 0x7f14000a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14080ce4d0 tx=0x7f1408067320 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 0x559384472540 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f140402c610 comp rx=0 tx=0).stop 2024-04-22T13:30:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 -- 172.21.15.5:0/4265960924 shutdown_connections 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469ea0 0x559384472540 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x559384474e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 --2- 172.21.15.5:0/4265960924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x559384469960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 -- 172.21.15.5:0/4265960924 >> 172.21.15.5:0/4265960924 conn(0x559384463cb0 msgr2=0x559384464090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 -- 172.21.15.5:0/4265960924 shutdown_connections 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 -- 172.21.15.5:0/4265960924 wait complete. 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 -- start start 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.288+0000 7f1419d82740 1 Processor -- start 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x55938446fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469ea0 0x55938446fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x559384470530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559384475810 con 0x559384472a80 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593843872f0 con 0x559384469580 2024-04-22T13:30:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559384475990 con 0x559384469ea0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1419d82740 10 client.0 _ll_register_callbacks cb 0x559384394580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x559384470530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x559384470530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35568/0 (socket says 172.21.15.5:35568) 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 -- 172.21.15.5:0/394333163 learned_addr learned my addr 172.21.15.5:0/394333163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f1414e5c640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469ea0 0x55938446fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x55938446fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 -- 172.21.15.5:0/394333163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469ea0 msgr2=0x55938446fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469ea0 0x55938446fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 -- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 msgr2=0x55938446fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x55938446fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 -- 172.21.15.5:0/394333163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593840e03f0 con 0x559384472a80 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f140ffff640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x559384470530 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0126a0 tx=0x7f13fc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c433ffe952ff8e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.289+0000 7f13effff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f13effff640 1 -- 172.21.15.5:0/394333163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc024060 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f13effff640 1 -- 172.21.15.5:0/394333163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f13fc004230 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140402c7e0 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f13effff640 1 -- 172.21.15.5:0/394333163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f13fc004430 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 10 client.16962 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593840a00f0 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f13effff640 1 -- 172.21.15.5:0/394333163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f140402c7e0 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f13effff640 1 -- 172.21.15.5:0/394333163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f13fc01e070 con 0x559384472a80 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 10 client.16962 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 20 client.16962 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559384482af0 0x559384484f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 4 client.16962 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.290+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593843dbc00 con 0x559384482af0 2024-04-22T13:30:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.291+0000 7f1414e5c640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559384482af0 0x559384484f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.291+0000 7f1414e5c640 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559384482af0 0x559384484f00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae413e3841b5641e server_cookie=4dc63b94ebcbce72 in_seq=0 out_seq=0 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.291+0000 7f13effff640 10 client.16962 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f13effff640 1 -- 172.21.15.5:0/394333163 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593843dbc00 con 0x559384482af0 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f13effff640 10 client.16962 handle_command_reply: tid=0 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 client.16962 shutdown 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 msgr2=0x559384470530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x559384470530 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0126a0 tx=0x7f13fc014040 comp rx=0 tx=0).stop 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 shutdown_connections 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559384482af0 0x559384484f00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x559384470530 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469ea0 0x55938446fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f13effff640 0 client.16962 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 --2- 172.21.15.5:0/394333163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469580 0x55938446fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 >> 172.21.15.5:0/394333163 conn(0x559384463cb0 msgr2=0x559384465a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 shutdown_connections 2024-04-22T13:30:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 1 -- 172.21.15.5:0/394333163 wait complete. 2024-04-22T13:30:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.292+0000 7f1419d82740 20 client.16962 trim_cache size 0 max 16384 2024-04-22T13:30:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 Processor -- start 2024-04-22T13:30:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 -- start start 2024-04-22T13:30:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469580 0x5593844721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x55938447e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x55938447aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ada90 con 0x559384472a80 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f14101af5d0 con 0x559384469ea0 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f1419d82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ae160 con 0x559384469580 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f140ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x55938447aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f140ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469580 0x5593844721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f140ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x55938447aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35572/0 (socket says 172.21.15.5:35572) 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.311+0000 7f140ffff640 1 -- 172.21.15.5:0/1730824909 learned_addr learned my addr 172.21.15.5:0/1730824909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x55938447e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f140ffff640 1 -- 172.21.15.5:0/1730824909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469580 msgr2=0x5593844721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f140ffff640 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469580 0x5593844721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f140ffff640 1 -- 172.21.15.5:0/1730824909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 msgr2=0x55938447e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f140ffff640 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x55938447e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f140ffff640 1 -- 172.21.15.5:0/1730824909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938446e000 con 0x559384472a80 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x55938447e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f140ffff640 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x55938447aff0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005900 tx=0x7f13fc0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdca60620bd565e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f13fa7fc640 1 -- 172.21.15.5:0/1730824909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0123a0 con 0x559384472a80 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f13fa7fc640 1 -- 172.21.15.5:0/1730824909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13fc010770 con 0x559384472a80 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.312+0000 7f13fa7fc640 1 -- 172.21.15.5:0/1730824909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc005240 con 0x559384472a80 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 -- 172.21.15.5:0/1730824909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 msgr2=0x55938447aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x55938447aff0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005900 tx=0x7f13fc0045d0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 -- 172.21.15.5:0/1730824909 shutdown_connections 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384472a80 0x55938447aff0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x55938447e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 --2- 172.21.15.5:0/1730824909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384469580 0x5593844721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 -- 172.21.15.5:0/1730824909 >> 172.21.15.5:0/1730824909 conn(0x559384463cb0 msgr2=0x5593844624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 -- 172.21.15.5:0/1730824909 shutdown_connections 2024-04-22T13:30:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 -- 172.21.15.5:0/1730824909 wait complete. 2024-04-22T13:30:00.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.313+0000 7f1419d82740 1 -- start start 2024-04-22T13:30:00.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 Processor -- start 2024-04-22T13:30:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 0x5593844e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x5593844e1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x5593844ec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14101aef00 con 0x559384469580 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ae830 con 0x559384469ea0 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14101aba90 con 0x559384472a80 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1419d82740 10 client.0 _ll_register_callbacks cb 0x7f13fc00b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f140ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 0x5593844e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f140ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 0x5593844e1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f140ffff640 1 -- 172.21.15.5:0/1490470130 learned_addr learned my addr 172.21.15.5:0/1490470130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x5593844e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f140ffff640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x5593844ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1414e5c640 1 -- 172.21.15.5:0/1490470130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 msgr2=0x5593844ec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x5593844ec080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1414e5c640 1 -- 172.21.15.5:0/1490470130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 msgr2=0x5593844e1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.314+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 0x5593844e1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f1414e5c640 1 -- 172.21.15.5:0/1490470130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593844ec5c0 con 0x559384469ea0 2024-04-22T13:30:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f140ffff640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 0x5593844e1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f1414e5c640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x5593844e1540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1404036040 tx=0x7f140402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bac50640cf3bd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f13effff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f13effff640 1 -- 172.21.15.5:0/1490470130 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404054070 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f13effff640 1 -- 172.21.15.5:0/1490470130 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1404004240 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13fc010770 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f13effff640 1 -- 172.21.15.5:0/1490470130 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f140405f050 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f1419d82740 10 client.26303 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593844ed7d0 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.315+0000 7f13effff640 1 -- 172.21.15.5:0/1490470130 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13fc010770 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f13effff640 1 -- 172.21.15.5:0/1490470130 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1404003ca0 con 0x559384469ea0 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 10 client.26303 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 20 client.26303 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:00.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593844edb10 0x5593844eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 4 client.26303 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593843d5a70 con 0x5593844edb10 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f140ffff640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593844edb10 0x5593844eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f13effff640 10 client.26303 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f140ffff640 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593844edb10 0x5593844eff00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b672ca857e9922 server_cookie=d0c73e194e626684 in_seq=0 out_seq=0 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f13effff640 1 -- 172.21.15.5:0/1490470130 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5593843d5a70 con 0x5593844edb10 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f13effff640 10 client.26303 handle_command_reply: tid=0 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.316+0000 7f1419d82740 1 client.26303 shutdown 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 msgr2=0x5593844e1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x5593844e1540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1404036040 tx=0x7f140402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 shutdown_connections 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593844edb10 0x5593844eff00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559384472a80 0x5593844ec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559384469ea0 0x5593844e1540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 --2- 172.21.15.5:0/1490470130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559384469580 0x5593844e1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f13effff640 0 client.26303 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 >> 172.21.15.5:0/1490470130 conn(0x559384463cb0 msgr2=0x559384462d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 shutdown_connections 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 1 -- 172.21.15.5:0/1490470130 wait complete. 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f1419d82740 20 client.26303 trim_cache size 0 max 16384 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stdout: "19277220-82a0-430c-8695-a533993be220": { 2024-04-22T13:30:00.319 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:30:00.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "19277220-82a0-430c-8695-a533993be220", 2024-04-22T13:30:00.320 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:30:00.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:30:00.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:30:00.320 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f14000a3d20 msgr2=0x7f14000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f14000a3d20 0x7f14000a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14080ce4d0 tx=0x7f1408067320 comp rx=0 tx=0).stop 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 msgr2=0x7f141019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 0x7f141019d480 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f140800eb10 tx=0x7f140800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 shutdown_connections 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141019cfd0 0x7f141019d480 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f14000a3d20 0x7f14000a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1410107270 0x7f1410198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.317+0000 7f14170c0640 1 --2- 172.21.15.5:0/1605162498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14101021a0 0x7f1410198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.318+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 >> 172.21.15.5:0/1605162498 conn(0x7f14100fd9c0 msgr2=0x7f14100fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.318+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 shutdown_connections 2024-04-22T13:30:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:00.318+0000 7f14170c0640 1 -- 172.21.15.5:0/1605162498 wait complete. 2024-04-22T13:30:00.325 INFO:tasks.cephfs.filesystem:scrub status for tag:19277220-82a0-430c-8695-a533993be220 - {'path': '/', 'tag': '19277220-82a0-430c-8695-a533993be220', 'options': 'recursive,force'} 2024-04-22T13:30:01.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-04-22T13:30:01.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: overall HEALTH_OK 2024-04-22T13:30:01.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:30:01.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:01.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:30:01.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:01.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:00 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:30:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-04-22T13:30:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: overall HEALTH_OK 2024-04-22T13:30:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:30:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:30:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:01.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:00 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: pgmap v89: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: overall HEALTH_OK 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:00 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:30:03.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:02 smithi049 ceph-mon[31379]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:30:03.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:02 smithi107 ceph-mon[31859]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:30:03.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:02 smithi005 ceph-mon[28006]: pgmap v90: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:30:05.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:04 smithi049 ceph-mon[31379]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 863 KiB/s wr, 14 op/s 2024-04-22T13:30:05.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:04 smithi107 ceph-mon[31859]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 863 KiB/s wr, 14 op/s 2024-04-22T13:30:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:04 smithi005 ceph-mon[28006]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 863 KiB/s wr, 14 op/s 2024-04-22T13:30:07.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:06 smithi049 ceph-mon[31379]: pgmap v92: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.6 MiB/s wr, 358 op/s 2024-04-22T13:30:07.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:06 smithi107 ceph-mon[31859]: pgmap v92: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.6 MiB/s wr, 358 op/s 2024-04-22T13:30:07.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:06 smithi005 ceph-mon[28006]: pgmap v92: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.6 MiB/s wr, 358 op/s 2024-04-22T13:30:08.343 INFO:tasks.workunit.client.0.smithi005.stdout:setting up random write file 2024-04-22T13:30:08.343 INFO:tasks.workunit.client.0.smithi005.stdout:done setting up random write file 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:starting fsync run 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0003s fsync time: 0.0100s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 26.3115s fsync time: 0.0057s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0057s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0053s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0058s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0053s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:30:08.344 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0056s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0060s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0050s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0050s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0003s fsync time: 0.0054s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0061s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0057s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0059s 2024-04-22T13:30:08.345 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0047s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0042s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0043s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0003s fsync time: 0.0050s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stdout:run done 35 fsyncs total, killing random writer 2024-04-22T13:30:08.346 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T13:30:08.347 INFO:tasks.workunit.client.0.smithi005.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T13:30:08.347 INFO:tasks.workunit.client.0.smithi005.stderr:+ whereis lsof 2024-04-22T13:30:08.360 INFO:tasks.workunit.client.0.smithi005.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-04-22T13:30:08.360 INFO:tasks.workunit.client.0.smithi005.stderr:+ lsof 2024-04-22T13:30:08.381 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/b51aa304f451986f651e132e20e762607a3f5207a945553243ec38ba235199fa/merged 2024-04-22T13:30:08.381 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/65a1e6cfcc1b5a51d1066caa6f09909537f03d375278b3424a7b67f4d13cc08c/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/1db7ea603f0e896bd82bad0aebda47df29f989d844f670209f7d85b29fabafc6/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/5e14d2180de4431f46964f2cbb42aa2afcc4a6fffa3b503e040f12d96f9053bb/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6d8221f887313f6987155f1787eb2c68ab34ea61545cb6c06ccbab6c457f5829/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/2eb80e6a318af3832116c68bc8da3f84eb6e7757b6a27ce36b452c282f084dc9/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4e03ed75ed8308d13fa3acf6fe101f5f75f75498fac6356011baa666fb230164/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4a06723d4bc81fea05533cb0abd736f9d59380bb260706ee64e64f0b92586c69/merged 2024-04-22T13:30:08.382 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.383 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/be1e219320d82e18964316d7d3a042b3ba639635ac28741206e52d169a4e2076/merged 2024-04-22T13:30:08.383 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.383 INFO:tasks.workunit.client.0.smithi005.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f04fca97db726a5ebadf56cc4ba11e99ffebc5f7028606cf5f787ac5a4d22de9/merged 2024-04-22T13:30:08.383 INFO:tasks.workunit.client.0.smithi005.stderr: Output information may be incomplete. 2024-04-22T13:30:08.447 INFO:tasks.workunit.client.0.smithi005.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-04-22T13:30:08.447 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-04-22T13:30:08.447 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-04-22T13:30:08.448 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-04-22T13:30:08.449 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-04-22T13:30:08.450 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-04-22T13:30:08.451 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-04-22T13:30:08.452 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-04-22T13:30:08.453 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-04-22T13:30:08.454 INFO:tasks.workunit.client.0.smithi005.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-04-22T13:30:08.455 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-04-22T13:30:08.456 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-04-22T13:30:08.457 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-04-22T13:30:08.458 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-04-22T13:30:08.459 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-04-22T13:30:08.460 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-04-22T13:30:08.461 INFO:tasks.workunit.client.0.smithi005.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-04-22T13:30:08.462 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-04-22T13:30:08.463 INFO:tasks.workunit.client.0.smithi005.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-04-22T13:30:08.464 INFO:tasks.workunit.client.0.smithi005.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:khugepage 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:khugepage 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:khugepage 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:khugepage 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:cryptd 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:cryptd 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:cryptd 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:cryptd 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:kintegrit 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:kintegrit 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-04-22T13:30:08.465 INFO:tasks.workunit.client.0.smithi005.stdout:kintegrit 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:kintegrit 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:kblockd 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:kblockd 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:kblockd 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:kblockd 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:blkcg_pun 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:blkcg_pun 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:blkcg_pun 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:blkcg_pun 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:tpm_dev_w 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:tpm_dev_w 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:tpm_dev_w 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:md 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-04-22T13:30:08.466 INFO:tasks.workunit.client.0.smithi005.stdout:md 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:md 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:md 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:md_bitmap 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:md_bitmap 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:md_bitmap 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:md_bitmap 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:edac-poll 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:edac-poll 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:edac-poll 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:edac-poll 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:watchdogd 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:watchdogd 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:watchdogd 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-04-22T13:30:08.467 INFO:tasks.workunit.client.0.smithi005.stdout:watchdogd 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kswapd0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kswapd0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kswapd0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kswapd0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kthrotld 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kthrotld 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kthrotld 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:kthrotld 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-04-22T13:30:08.468 INFO:tasks.workunit.client.0.smithi005.stdout:acpi_ther 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:acpi_ther 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:acpi_ther 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:acpi_ther 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kmpath_rd 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kmpath_rd 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kmpath_rd 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kmpath_rd 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kaluad 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kaluad 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kaluad 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:kaluad 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:mld 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:mld 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:mld 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-04-22T13:30:08.469 INFO:tasks.workunit.client.0.smithi005.stdout:mld 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:ipv6_addr 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:ipv6_addr 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:ipv6_addr 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:ipv6_addr 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kstrp 125 root cwd unknown /proc/125/cwd (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kstrp 125 root rtd unknown /proc/125/root (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kstrp 125 root txt unknown /proc/125/exe (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:kstrp 125 root NOFD /proc/125/fd (opendir: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:zswap-shr 133 root cwd unknown /proc/133/cwd (readlink: Permission denied) 2024-04-22T13:30:08.470 INFO:tasks.workunit.client.0.smithi005.stdout:zswap-shr 133 root rtd unknown /proc/133/root (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:zswap-shr 133 root txt unknown /proc/133/exe (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:zswap-shr 133 root NOFD /proc/133/fd (opendir: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 134 root cwd unknown /proc/134/cwd (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 134 root rtd unknown /proc/134/root (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 134 root txt unknown /proc/134/exe (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 134 root NOFD /proc/134/fd (opendir: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 241 root cwd unknown /proc/241/cwd (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 241 root rtd unknown /proc/241/root (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 241 root txt unknown /proc/241/exe (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 241 root NOFD /proc/241/fd (opendir: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 283 root cwd unknown /proc/283/cwd (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 283 root rtd unknown /proc/283/root (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 283 root txt unknown /proc/283/exe (readlink: Permission denied) 2024-04-22T13:30:08.471 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 283 root NOFD /proc/283/fd (opendir: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 285 root cwd unknown /proc/285/cwd (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 285 root rtd unknown /proc/285/root (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 285 root txt unknown /proc/285/exe (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 285 root NOFD /proc/285/fd (opendir: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 287 root cwd unknown /proc/287/cwd (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 287 root rtd unknown /proc/287/root (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 287 root txt unknown /proc/287/exe (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 287 root NOFD /proc/287/fd (opendir: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 302 root cwd unknown /proc/302/cwd (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 302 root rtd unknown /proc/302/root (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 302 root txt unknown /proc/302/exe (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 302 root NOFD /proc/302/fd (opendir: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 379 root cwd unknown /proc/379/cwd (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 379 root rtd unknown /proc/379/root (readlink: Permission denied) 2024-04-22T13:30:08.472 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 379 root txt unknown /proc/379/exe (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 379 root NOFD /proc/379/fd (opendir: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 390 root cwd unknown /proc/390/cwd (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 390 root rtd unknown /proc/390/root (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 390 root txt unknown /proc/390/exe (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 390 root NOFD /proc/390/fd (opendir: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ixgbe 435 root cwd unknown /proc/435/cwd (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ixgbe 435 root rtd unknown /proc/435/root (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ixgbe 435 root txt unknown /proc/435/exe (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ixgbe 435 root NOFD /proc/435/fd (opendir: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ata_sff 436 root cwd unknown /proc/436/cwd (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ata_sff 436 root rtd unknown /proc/436/root (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ata_sff 436 root txt unknown /proc/436/exe (readlink: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:ata_sff 436 root NOFD /proc/436/fd (opendir: Permission denied) 2024-04-22T13:30:08.473 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_0 442 root cwd unknown /proc/442/cwd (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_0 442 root rtd unknown /proc/442/root (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_0 442 root txt unknown /proc/442/exe (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_0 442 root NOFD /proc/442/fd (opendir: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_1 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_1 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_1 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_1 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-04-22T13:30:08.474 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_2 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_2 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_2 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_2 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_3 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_3 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_3 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_3 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-04-22T13:30:08.475 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-wq 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-wq 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-wq 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-wq 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-rese 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-rese 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-rese 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-rese 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-dele 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-dele 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-dele 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-dele 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-auth 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-04-22T13:30:08.476 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-auth 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-auth 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:nvme-auth 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_4 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_4 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_4 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_4 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_5 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_5 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_5 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_5 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-04-22T13:30:08.477 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_6 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_6 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_6 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_6 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_7 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_7 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-04-22T13:30:08.478 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_7 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_7 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_8 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_8 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_8 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_8 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_9 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-04-22T13:30:08.479 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_9 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_9 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_eh_9 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 470 root cwd unknown /proc/470/cwd (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 470 root rtd unknown /proc/470/root (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 470 root txt unknown /proc/470/exe (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:scsi_tmf_ 470 root NOFD /proc/470/fd (opendir: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 496 root cwd unknown /proc/496/cwd (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 496 root rtd unknown /proc/496/root (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 496 root txt unknown /proc/496/exe (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 496 root NOFD /proc/496/fd (opendir: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:jbd2/sda1 500 root cwd unknown /proc/500/cwd (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:jbd2/sda1 500 root rtd unknown /proc/500/root (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:jbd2/sda1 500 root txt unknown /proc/500/exe (readlink: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:jbd2/sda1 500 root NOFD /proc/500/fd (opendir: Permission denied) 2024-04-22T13:30:08.480 INFO:tasks.workunit.client.0.smithi005.stdout:ext4-rsv- 501 root cwd unknown /proc/501/cwd (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4-rsv- 501 root rtd unknown /proc/501/root (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4-rsv- 501 root txt unknown /proc/501/exe (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4-rsv- 501 root NOFD /proc/501/fd (opendir: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-j 577 root cwd unknown /proc/577/cwd (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-j 577 root rtd unknown /proc/577/root (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-j 577 root txt unknown /proc/577/exe (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-j 577 root NOFD /proc/577/fd (opendir: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4lazyi 583 root cwd unknown /proc/583/cwd (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4lazyi 583 root rtd unknown /proc/583/root (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4lazyi 583 root txt unknown /proc/583/exe (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:ext4lazyi 583 root NOFD /proc/583/fd (opendir: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 585 root cwd unknown /proc/585/cwd (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 585 root rtd unknown /proc/585/root (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 585 root txt unknown /proc/585/exe (readlink: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 585 root NOFD /proc/585/fd (opendir: Permission denied) 2024-04-22T13:30:08.481 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-u 597 root cwd unknown /proc/597/cwd (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-u 597 root rtd unknown /proc/597/root (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-u 597 root txt unknown /proc/597/exe (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-u 597 root NOFD /proc/597/fd (opendir: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:ipmi-msgh 648 root cwd unknown /proc/648/cwd (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:ipmi-msgh 648 root rtd unknown /proc/648/root (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:ipmi-msgh 648 root txt unknown /proc/648/exe (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:ipmi-msgh 648 root NOFD /proc/648/fd (opendir: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kipmi0 664 root cwd unknown /proc/664/cwd (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kipmi0 664 root rtd unknown /proc/664/root (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kipmi0 664 root txt unknown /proc/664/exe (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kipmi0 664 root NOFD /proc/664/fd (opendir: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 669 root cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 669 root rtd unknown /proc/669/root (readlink: Permission denied) 2024-04-22T13:30:08.482 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 669 root txt unknown /proc/669/exe (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 669 root NOFD /proc/669/fd (opendir: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpcbind 672 rpc cwd unknown /proc/672/cwd (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpcbind 672 rpc rtd unknown /proc/672/root (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpcbind 672 rpc txt unknown /proc/672/exe (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpcbind 672 rpc NOFD /proc/672/fd (opendir: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpciod 676 root cwd unknown /proc/676/cwd (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpciod 676 root rtd unknown /proc/676/root (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpciod 676 root txt unknown /proc/676/exe (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:rpciod 676 root NOFD /proc/676/fd (opendir: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:xprtiod 677 root cwd unknown /proc/677/cwd (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:xprtiod 677 root rtd unknown /proc/677/root (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:xprtiod 677 root txt unknown /proc/677/exe (readlink: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:xprtiod 677 root NOFD /proc/677/fd (opendir: Permission denied) 2024-04-22T13:30:08.483 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 root cwd unknown /proc/678/cwd (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 root rtd unknown /proc/678/root (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 root txt unknown /proc/678/exe (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 root NOFD /proc/678/fd (opendir: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 679 auditd root cwd unknown /proc/678/task/679/cwd (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 679 auditd root rtd unknown /proc/678/task/679/root (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 679 auditd root txt unknown /proc/678/task/679/exe (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 679 auditd root NOFD /proc/678/task/679/fd (opendir: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 683 auditd root cwd unknown /proc/678/task/683/cwd (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 683 auditd root rtd unknown /proc/678/task/683/root (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 683 auditd root txt unknown /proc/678/task/683/exe (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:auditd 678 683 auditd root NOFD /proc/678/task/683/fd (opendir: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:sedispatc 682 root cwd unknown /proc/682/cwd (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:sedispatc 682 root rtd unknown /proc/682/root (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:sedispatc 682 root txt unknown /proc/682/exe (readlink: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:sedispatc 682 root NOFD /proc/682/fd (opendir: Permission denied) 2024-04-22T13:30:08.484 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 720 dbus cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 720 dbus rtd unknown /proc/720/root (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 720 dbus txt unknown /proc/720/exe (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 720 dbus NOFD /proc/720/fd (opendir: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 721 dbus cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 721 dbus rtd unknown /proc/721/root (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 721 dbus txt unknown /proc/721/exe (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:dbus-brok 721 dbus NOFD /proc/721/fd (opendir: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 775 gmain root cwd unknown /proc/722/task/775/cwd (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 775 gmain root rtd unknown /proc/722/task/775/root (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 775 gmain root txt unknown /proc/722/task/775/exe (readlink: Permission denied) 2024-04-22T13:30:08.485 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 775 gmain root NOFD /proc/722/task/775/fd (opendir: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 776 gdbus root cwd unknown /proc/722/task/776/cwd (readlink: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 776 gdbus root rtd unknown /proc/722/task/776/root (readlink: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 776 gdbus root txt unknown /proc/722/task/776/exe (readlink: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:NetworkMa 722 776 gdbus root NOFD /proc/722/task/776/fd (opendir: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-04-22T13:30:08.486 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 729 gmain root cwd unknown /proc/725/task/729/cwd (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 729 gmain root rtd unknown /proc/725/task/729/root (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 729 gmain root txt unknown /proc/725/task/729/exe (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:irqbalanc 725 729 gmain root NOFD /proc/725/task/729/fd (opendir: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:lsmd 726 libstoragemgmt cwd unknown /proc/726/cwd (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:lsmd 726 libstoragemgmt rtd unknown /proc/726/root (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:lsmd 726 libstoragemgmt txt unknown /proc/726/exe (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:lsmd 726 libstoragemgmt NOFD /proc/726/fd (opendir: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:mcelog 727 root cwd unknown /proc/727/cwd (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:mcelog 727 root rtd unknown /proc/727/root (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:mcelog 727 root txt unknown /proc/727/exe (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:mcelog 727 root NOFD /proc/727/fd (opendir: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:smartd 728 root cwd unknown /proc/728/cwd (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:smartd 728 root rtd unknown /proc/728/root (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:smartd 728 root txt unknown /proc/728/exe (readlink: Permission denied) 2024-04-22T13:30:08.487 INFO:tasks.workunit.client.0.smithi005.stdout:smartd 728 root NOFD /proc/728/fd (opendir: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-l 734 root cwd unknown /proc/734/cwd (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-l 734 root rtd unknown /proc/734/root (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-l 734 root txt unknown /proc/734/exe (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:systemd-l 734 root NOFD /proc/734/fd (opendir: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 823 root cwd unknown /proc/823/cwd (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 823 root rtd unknown /proc/823/root (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 823 root txt unknown /proc/823/exe (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 823 root NOFD /proc/823/fd (opendir: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 root cwd unknown /proc/827/cwd (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 root rtd unknown /proc/827/root (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 root txt unknown /proc/827/exe (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 root NOFD /proc/827/fd (opendir: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 828 gssproxy root cwd unknown /proc/827/task/828/cwd (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 828 gssproxy root rtd unknown /proc/827/task/828/root (readlink: Permission denied) 2024-04-22T13:30:08.488 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 828 gssproxy root txt unknown /proc/827/task/828/exe (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 828 gssproxy root NOFD /proc/827/task/828/fd (opendir: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 829 gssproxy root cwd unknown /proc/827/task/829/cwd (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 829 gssproxy root rtd unknown /proc/827/task/829/root (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 829 gssproxy root txt unknown /proc/827/task/829/exe (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 829 gssproxy root NOFD /proc/827/task/829/fd (opendir: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 830 gssproxy root cwd unknown /proc/827/task/830/cwd (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 830 gssproxy root rtd unknown /proc/827/task/830/root (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 830 gssproxy root txt unknown /proc/827/task/830/exe (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 830 gssproxy root NOFD /proc/827/task/830/fd (opendir: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 831 gssproxy root cwd unknown /proc/827/task/831/cwd (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 831 gssproxy root rtd unknown /proc/827/task/831/root (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 831 gssproxy root txt unknown /proc/827/task/831/exe (readlink: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 831 gssproxy root NOFD /proc/827/task/831/fd (opendir: Permission denied) 2024-04-22T13:30:08.489 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 832 gssproxy root cwd unknown /proc/827/task/832/cwd (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 832 gssproxy root rtd unknown /proc/827/task/832/root (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 832 gssproxy root txt unknown /proc/827/task/832/exe (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:gssproxy 827 832 gssproxy root NOFD /proc/827/task/832/fd (opendir: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:atd 834 root cwd unknown /proc/834/cwd (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:atd 834 root rtd unknown /proc/834/root (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:atd 834 root txt unknown /proc/834/exe (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:atd 834 root NOFD /proc/834/fd (opendir: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:crond 835 root cwd unknown /proc/835/cwd (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:crond 835 root rtd unknown /proc/835/root (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:crond 835 root txt unknown /proc/835/exe (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:crond 835 root NOFD /proc/835/fd (opendir: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:nrpe 859 nrpe cwd unknown /proc/859/cwd (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:nrpe 859 nrpe rtd unknown /proc/859/root (readlink: Permission denied) 2024-04-22T13:30:08.490 INFO:tasks.workunit.client.0.smithi005.stdout:nrpe 859 nrpe txt unknown /proc/859/exe (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:nrpe 859 nrpe NOFD /proc/859/fd (opendir: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 5199 root cwd unknown /proc/5199/cwd (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 5199 root rtd unknown /proc/5199/root (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 5199 root txt unknown /proc/5199/exe (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 5199 root NOFD /proc/5199/fd (opendir: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5201 root cwd unknown /proc/5201/cwd (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5201 root rtd unknown /proc/5201/root (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5201 root txt unknown /proc/5201/exe (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5201 root NOFD /proc/5201/fd (opendir: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5203 root cwd unknown /proc/5203/cwd (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5203 root rtd unknown /proc/5203/root (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5203 root txt unknown /proc/5203/exe (readlink: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:agetty 5203 root NOFD /proc/5203/fd (opendir: Permission denied) 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu cwd DIR 8,1 4096 2 / 2024-04-22T13:30:08.491 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu DEL REG 8,1 61581 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu DEL REG 8,1 62167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-22T13:30:08.492 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-22T13:30:08.493 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 1u unix 0x0000000000000000 0t0 26922 type=STREAM (CONNECTED) 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 2u unix 0x0000000000000000 0t0 26922 type=STREAM (CONNECTED) 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 3u unix 0x0000000000000000 0t0 26950 type=DGRAM (CONNECTED) 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 4u a_inode 0,14 0 2074 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 5u a_inode 0,14 0 2074 [signalfd] 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 6r a_inode 0,14 0 2074 inotify 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 7r DIR 0,25 0 3933 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 8u a_inode 0,14 0 2074 [timerfd] 2024-04-22T13:30:08.494 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 9u a_inode 0,14 0 2074 [eventpoll:10,13] 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 10r REG 0,20 0 26939 /proc/5396/mountinfo 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 11r a_inode 0,14 0 2074 inotify 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 12u unix 0x0000000000000000 0t0 26972 /run/user/1000/bus type=STREAM (LISTEN) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 13r a_inode 0,14 0 2074 inotify 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 15u netlink 0t0 26954 KOBJECT_UEVENT 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 16u unix 0x0000000000000000 0t0 26959 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 17u unix 0x0000000000000000 0t0 26960 type=DGRAM (CONNECTED) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 18u unix 0x0000000000000000 0t0 26961 type=DGRAM (CONNECTED) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 19u unix 0x0000000000000000 0t0 26962 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 20u unix 0x0000000000000000 0t0 26964 type=STREAM (CONNECTED) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 5396 ubuntu 21u a_inode 0,14 0 2074 [timerfd] 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 5398 ubuntu cwd unknown /proc/5398/cwd (readlink: Permission denied) 2024-04-22T13:30:08.495 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 5398 ubuntu rtd unknown /proc/5398/root (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 5398 ubuntu txt unknown /proc/5398/exe (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 5398 ubuntu NOFD /proc/5398/fd (opendir: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:tls-strp 5580 root cwd unknown /proc/5580/cwd (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:tls-strp 5580 root rtd unknown /proc/5580/root (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:tls-strp 5580 root txt unknown /proc/5580/exe (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:tls-strp 5580 root NOFD /proc/5580/fd (opendir: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 polkitd cwd unknown /proc/5665/cwd (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 polkitd rtd unknown /proc/5665/root (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 polkitd txt unknown /proc/5665/exe (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 polkitd NOFD /proc/5665/fd (opendir: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5667 gmain polkitd cwd unknown /proc/5665/task/5667/cwd (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5667 gmain polkitd rtd unknown /proc/5665/task/5667/root (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5667 gmain polkitd txt unknown /proc/5665/task/5667/exe (readlink: Permission denied) 2024-04-22T13:30:08.496 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5667 gmain polkitd NOFD /proc/5665/task/5667/fd (opendir: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5668 gdbus polkitd cwd unknown /proc/5665/task/5668/cwd (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5668 gdbus polkitd rtd unknown /proc/5665/task/5668/root (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5668 gdbus polkitd txt unknown /proc/5665/task/5668/exe (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5668 gdbus polkitd NOFD /proc/5665/task/5668/fd (opendir: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5669 JS\x20Hel polkitd cwd unknown /proc/5665/task/5669/cwd (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5669 JS\x20Hel polkitd rtd unknown /proc/5665/task/5669/root (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5669 JS\x20Hel polkitd txt unknown /proc/5665/task/5669/exe (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5669 JS\x20Hel polkitd NOFD /proc/5665/task/5669/fd (opendir: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5670 JS\x20Hel polkitd cwd unknown /proc/5665/task/5670/cwd (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5670 JS\x20Hel polkitd rtd unknown /proc/5665/task/5670/root (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5670 JS\x20Hel polkitd txt unknown /proc/5665/task/5670/exe (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5670 JS\x20Hel polkitd NOFD /proc/5665/task/5670/fd (opendir: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5671 JS\x20Hel polkitd cwd unknown /proc/5665/task/5671/cwd (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5671 JS\x20Hel polkitd rtd unknown /proc/5665/task/5671/root (readlink: Permission denied) 2024-04-22T13:30:08.497 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5671 JS\x20Hel polkitd txt unknown /proc/5665/task/5671/exe (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5671 JS\x20Hel polkitd NOFD /proc/5665/task/5671/fd (opendir: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5672 JS\x20Hel polkitd cwd unknown /proc/5665/task/5672/cwd (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5672 JS\x20Hel polkitd rtd unknown /proc/5665/task/5672/root (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5672 JS\x20Hel polkitd txt unknown /proc/5665/task/5672/exe (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5672 JS\x20Hel polkitd NOFD /proc/5665/task/5672/fd (opendir: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5673 JS\x20Hel polkitd cwd unknown /proc/5665/task/5673/cwd (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5673 JS\x20Hel polkitd rtd unknown /proc/5665/task/5673/root (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5673 JS\x20Hel polkitd txt unknown /proc/5665/task/5673/exe (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5673 JS\x20Hel polkitd NOFD /proc/5665/task/5673/fd (opendir: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5674 JS\x20Hel polkitd cwd unknown /proc/5665/task/5674/cwd (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5674 JS\x20Hel polkitd rtd unknown /proc/5665/task/5674/root (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5674 JS\x20Hel polkitd txt unknown /proc/5665/task/5674/exe (readlink: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5674 JS\x20Hel polkitd NOFD /proc/5665/task/5674/fd (opendir: Permission denied) 2024-04-22T13:30:08.498 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5675 JS\x20Hel polkitd cwd unknown /proc/5665/task/5675/cwd (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5675 JS\x20Hel polkitd rtd unknown /proc/5665/task/5675/root (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5675 JS\x20Hel polkitd txt unknown /proc/5665/task/5675/exe (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5675 JS\x20Hel polkitd NOFD /proc/5665/task/5675/fd (opendir: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5676 JS\x20Hel polkitd cwd unknown /proc/5665/task/5676/cwd (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5676 JS\x20Hel polkitd rtd unknown /proc/5665/task/5676/root (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5676 JS\x20Hel polkitd txt unknown /proc/5665/task/5676/exe (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5676 JS\x20Hel polkitd NOFD /proc/5665/task/5676/fd (opendir: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5677 polkitd polkitd cwd unknown /proc/5665/task/5677/cwd (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5677 polkitd polkitd rtd unknown /proc/5665/task/5677/root (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5677 polkitd polkitd txt unknown /proc/5665/task/5677/exe (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:polkitd 5665 5677 polkitd polkitd NOFD /proc/5665/task/5677/fd (opendir: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 root cwd unknown /proc/5895/cwd (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 root rtd unknown /proc/5895/root (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 root txt unknown /proc/5895/exe (readlink: Permission denied) 2024-04-22T13:30:08.499 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 root NOFD /proc/5895/fd (opendir: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5897 in:imjour root cwd unknown /proc/5895/task/5897/cwd (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5897 in:imjour root rtd unknown /proc/5895/task/5897/root (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5897 in:imjour root txt unknown /proc/5895/task/5897/exe (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5897 in:imjour root NOFD /proc/5895/task/5897/fd (opendir: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5898 rs:main root cwd unknown /proc/5895/task/5898/cwd (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5898 rs:main root rtd unknown /proc/5895/task/5898/root (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5898 rs:main root txt unknown /proc/5895/task/5898/exe (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:rsyslogd 5895 5898 rs:main root NOFD /proc/5895/task/5898/fd (opendir: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 17705 root cwd unknown /proc/17705/cwd (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 17705 root rtd unknown /proc/17705/root (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 17705 root txt unknown /proc/17705/exe (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 17705 root NOFD /proc/17705/fd (opendir: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 18840 root cwd unknown /proc/18840/cwd (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 18840 root rtd unknown /proc/18840/root (readlink: Permission denied) 2024-04-22T13:30:08.500 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 18840 root txt unknown /proc/18840/exe (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 18840 root NOFD /proc/18840/fd (opendir: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 19213 root cwd unknown /proc/19213/cwd (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 19213 root rtd unknown /proc/19213/root (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 19213 root txt unknown /proc/19213/exe (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 19213 root NOFD /proc/19213/fd (opendir: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19283 root cwd unknown /proc/19283/cwd (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19283 root rtd unknown /proc/19283/root (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19283 root txt unknown /proc/19283/exe (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19283 root NOFD /proc/19283/fd (opendir: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19312 root cwd unknown /proc/19312/cwd (readlink: Permission denied) 2024-04-22T13:30:08.501 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19312 root rtd unknown /proc/19312/root (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19312 root txt unknown /proc/19312/exe (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19312 root NOFD /proc/19312/fd (opendir: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19341 root cwd unknown /proc/19341/cwd (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19341 root rtd unknown /proc/19341/root (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19341 root txt unknown /proc/19341/exe (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19341 root NOFD /proc/19341/fd (opendir: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19370 root cwd unknown /proc/19370/cwd (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19370 root rtd unknown /proc/19370/root (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19370 root txt unknown /proc/19370/exe (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19370 root NOFD /proc/19370/fd (opendir: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19399 root cwd unknown /proc/19399/cwd (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19399 root rtd unknown /proc/19399/root (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19399 root txt unknown /proc/19399/exe (readlink: Permission denied) 2024-04-22T13:30:08.502 INFO:tasks.workunit.client.0.smithi005.stdout:kdmflush/ 19399 root NOFD /proc/19399/fd (opendir: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfsalloc 19563 root cwd unknown /proc/19563/cwd (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfsalloc 19563 root rtd unknown /proc/19563/root (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfsalloc 19563 root txt unknown /proc/19563/exe (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfsalloc 19563 root NOFD /proc/19563/fd (opendir: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs_mru_c 19564 root cwd unknown /proc/19564/cwd (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs_mru_c 19564 root rtd unknown /proc/19564/root (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs_mru_c 19564 root txt unknown /proc/19564/exe (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs_mru_c 19564 root NOFD /proc/19564/fd (opendir: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-buf/d 19565 root cwd unknown /proc/19565/cwd (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-buf/d 19565 root rtd unknown /proc/19565/root (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-buf/d 19565 root txt unknown /proc/19565/exe (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-buf/d 19565 root NOFD /proc/19565/fd (opendir: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-conv/ 19566 root cwd unknown /proc/19566/cwd (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-conv/ 19566 root rtd unknown /proc/19566/root (readlink: Permission denied) 2024-04-22T13:30:08.503 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-conv/ 19566 root txt unknown /proc/19566/exe (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-conv/ 19566 root NOFD /proc/19566/fd (opendir: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-recla 19567 root cwd unknown /proc/19567/cwd (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-recla 19567 root rtd unknown /proc/19567/root (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-recla 19567 root txt unknown /proc/19567/exe (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-recla 19567 root NOFD /proc/19567/fd (opendir: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-block 19568 root cwd unknown /proc/19568/cwd (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-block 19568 root rtd unknown /proc/19568/root (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-block 19568 root txt unknown /proc/19568/exe (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-block 19568 root NOFD /proc/19568/fd (opendir: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-inode 19569 root cwd unknown /proc/19569/cwd (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-inode 19569 root rtd unknown /proc/19569/root (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-inode 19569 root txt unknown /proc/19569/exe (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-inode 19569 root NOFD /proc/19569/fd (opendir: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-log/d 19570 root cwd unknown /proc/19570/cwd (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-log/d 19570 root rtd unknown /proc/19570/root (readlink: Permission denied) 2024-04-22T13:30:08.504 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-log/d 19570 root txt unknown /proc/19570/exe (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-log/d 19570 root NOFD /proc/19570/fd (opendir: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-cil/d 19571 root cwd unknown /proc/19571/cwd (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-cil/d 19571 root rtd unknown /proc/19571/root (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-cil/d 19571 root txt unknown /proc/19571/exe (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfs-cil/d 19571 root NOFD /proc/19571/fd (opendir: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfsaild/d 19572 root cwd unknown /proc/19572/cwd (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfsaild/d 19572 root rtd unknown /proc/19572/root (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfsaild/d 19572 root txt unknown /proc/19572/exe (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:xfsaild/d 19572 root NOFD /proc/19572/fd (opendir: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20500 root cwd unknown /proc/20500/cwd (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20500 root rtd unknown /proc/20500/root (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20500 root txt unknown /proc/20500/exe (readlink: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20500 root NOFD /proc/20500/fd (opendir: Permission denied) 2024-04-22T13:30:08.505 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20503 ubuntu cwd unknown /proc/20503/cwd (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20503 ubuntu rtd unknown /proc/20503/root (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20503 ubuntu txt unknown /proc/20503/exe (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 20503 ubuntu NOFD /proc/20503/fd (opendir: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:chronyd 20564 chrony cwd unknown /proc/20564/cwd (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:chronyd 20564 chrony rtd unknown /proc/20564/root (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:chronyd 20564 chrony txt unknown /proc/20564/exe (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:chronyd 20564 chrony NOFD /proc/20564/fd (opendir: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-cras 20875 ceph cwd unknown /proc/20875/cwd (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-cras 20875 ceph rtd unknown /proc/20875/root (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-cras 20875 ceph txt unknown /proc/20875/exe (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-cras 20875 ceph NOFD /proc/20875/fd (opendir: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 26938 root cwd unknown /proc/26938/cwd (readlink: Permission denied) 2024-04-22T13:30:08.506 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 26938 root rtd unknown /proc/26938/root (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 26938 root txt unknown /proc/26938/exe (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 26938 root NOFD /proc/26938/fd (opendir: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 26991 root cwd unknown /proc/26991/cwd (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 26991 root rtd unknown /proc/26991/root (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 26991 root txt unknown /proc/26991/exe (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 26991 root NOFD /proc/26991/fd (opendir: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27008 root cwd unknown /proc/27008/cwd (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27008 root rtd unknown /proc/27008/root (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27008 root txt unknown /proc/27008/exe (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27008 root NOFD /proc/27008/fd (opendir: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 27009 root cwd unknown /proc/27009/cwd (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 27009 root rtd unknown /proc/27009/root (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 27009 root txt unknown /proc/27009/exe (readlink: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 27009 root NOFD /proc/27009/fd (opendir: Permission denied) 2024-04-22T13:30:08.507 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27026 root cwd unknown /proc/27026/cwd (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27026 root rtd unknown /proc/27026/root (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27026 root txt unknown /proc/27026/exe (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 27026 root NOFD /proc/27026/fd (opendir: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28002 root cwd unknown /proc/28002/cwd (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28002 root rtd unknown /proc/28002/root (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28002 root txt unknown /proc/28002/exe (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28002 root NOFD /proc/28002/fd (opendir: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28004 root cwd unknown /proc/28004/cwd (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28004 root rtd unknown /proc/28004/root (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28004 root txt unknown /proc/28004/exe (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28004 root NOFD /proc/28004/fd (opendir: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 ceph cwd unknown /proc/28006/cwd (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 ceph rtd unknown /proc/28006/root (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 ceph txt unknown /proc/28006/exe (readlink: Permission denied) 2024-04-22T13:30:08.508 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 ceph NOFD /proc/28006/fd (opendir: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28007 log ceph cwd unknown /proc/28006/task/28007/cwd (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28007 log ceph rtd unknown /proc/28006/task/28007/root (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28007 log ceph txt unknown /proc/28006/task/28007/exe (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28007 log ceph NOFD /proc/28006/task/28007/fd (opendir: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28008 service ceph cwd unknown /proc/28006/task/28008/cwd (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28008 service ceph rtd unknown /proc/28006/task/28008/root (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28008 service ceph txt unknown /proc/28006/task/28008/exe (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28008 service ceph NOFD /proc/28006/task/28008/fd (opendir: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28009 admin_soc ceph cwd unknown /proc/28006/task/28009/cwd (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28009 admin_soc ceph rtd unknown /proc/28006/task/28009/root (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28009 admin_soc ceph txt unknown /proc/28006/task/28009/exe (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28009 admin_soc ceph NOFD /proc/28006/task/28009/fd (opendir: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28010 signal_ha ceph cwd unknown /proc/28006/task/28010/cwd (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28010 signal_ha ceph rtd unknown /proc/28006/task/28010/root (readlink: Permission denied) 2024-04-22T13:30:08.509 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28010 signal_ha ceph txt unknown /proc/28006/task/28010/exe (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28010 signal_ha ceph NOFD /proc/28006/task/28010/fd (opendir: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28011 rocksdb:l ceph cwd unknown /proc/28006/task/28011/cwd (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28011 rocksdb:l ceph rtd unknown /proc/28006/task/28011/root (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28011 rocksdb:l ceph txt unknown /proc/28006/task/28011/exe (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28011 rocksdb:l ceph NOFD /proc/28006/task/28011/fd (opendir: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28012 rocksdb:h ceph cwd unknown /proc/28006/task/28012/cwd (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28012 rocksdb:h ceph rtd unknown /proc/28006/task/28012/root (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28012 rocksdb:h ceph txt unknown /proc/28006/task/28012/exe (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28012 rocksdb:h ceph NOFD /proc/28006/task/28012/fd (opendir: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28028 ceph-mon ceph cwd unknown /proc/28006/task/28028/cwd (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28028 ceph-mon ceph rtd unknown /proc/28006/task/28028/root (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28028 ceph-mon ceph txt unknown /proc/28006/task/28028/exe (readlink: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28028 ceph-mon ceph NOFD /proc/28006/task/28028/fd (opendir: Permission denied) 2024-04-22T13:30:08.510 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28029 fn_monsto ceph cwd unknown /proc/28006/task/28029/cwd (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28029 fn_monsto ceph rtd unknown /proc/28006/task/28029/root (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28029 fn_monsto ceph txt unknown /proc/28006/task/28029/exe (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28029 fn_monsto ceph NOFD /proc/28006/task/28029/fd (opendir: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28030 msgr-work ceph cwd unknown /proc/28006/task/28030/cwd (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28030 msgr-work ceph rtd unknown /proc/28006/task/28030/root (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28030 msgr-work ceph txt unknown /proc/28006/task/28030/exe (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28030 msgr-work ceph NOFD /proc/28006/task/28030/fd (opendir: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28031 msgr-work ceph cwd unknown /proc/28006/task/28031/cwd (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28031 msgr-work ceph rtd unknown /proc/28006/task/28031/root (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28031 msgr-work ceph txt unknown /proc/28006/task/28031/exe (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28031 msgr-work ceph NOFD /proc/28006/task/28031/fd (opendir: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28032 msgr-work ceph cwd unknown /proc/28006/task/28032/cwd (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28032 msgr-work ceph rtd unknown /proc/28006/task/28032/root (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28032 msgr-work ceph txt unknown /proc/28006/task/28032/exe (readlink: Permission denied) 2024-04-22T13:30:08.511 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28032 msgr-work ceph NOFD /proc/28006/task/28032/fd (opendir: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28033 OpHistory ceph cwd unknown /proc/28006/task/28033/cwd (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28033 OpHistory ceph rtd unknown /proc/28006/task/28033/root (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28033 OpHistory ceph txt unknown /proc/28006/task/28033/exe (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28033 OpHistory ceph NOFD /proc/28006/task/28033/fd (opendir: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28034 fin ceph cwd unknown /proc/28006/task/28034/cwd (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28034 fin ceph rtd unknown /proc/28006/task/28034/root (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28034 fin ceph txt unknown /proc/28006/task/28034/exe (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28034 fin ceph NOFD /proc/28006/task/28034/fd (opendir: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28035 safe_time ceph cwd unknown /proc/28006/task/28035/cwd (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28035 safe_time ceph rtd unknown /proc/28006/task/28035/root (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28035 safe_time ceph txt unknown /proc/28006/task/28035/exe (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28035 safe_time ceph NOFD /proc/28006/task/28035/fd (opendir: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28036 cpu_tp ceph cwd unknown /proc/28006/task/28036/cwd (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28036 cpu_tp ceph rtd unknown /proc/28006/task/28036/root (readlink: Permission denied) 2024-04-22T13:30:08.512 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28036 cpu_tp ceph txt unknown /proc/28006/task/28036/exe (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28036 cpu_tp ceph NOFD /proc/28006/task/28036/fd (opendir: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28037 cpu_tp ceph cwd unknown /proc/28006/task/28037/cwd (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28037 cpu_tp ceph rtd unknown /proc/28006/task/28037/root (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28037 cpu_tp ceph txt unknown /proc/28006/task/28037/exe (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28037 cpu_tp ceph NOFD /proc/28006/task/28037/fd (opendir: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28038 cpu_tp ceph cwd unknown /proc/28006/task/28038/cwd (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28038 cpu_tp ceph rtd unknown /proc/28006/task/28038/root (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28038 cpu_tp ceph txt unknown /proc/28006/task/28038/exe (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28038 cpu_tp ceph NOFD /proc/28006/task/28038/fd (opendir: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28039 cpu_tp ceph cwd unknown /proc/28006/task/28039/cwd (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28039 cpu_tp ceph rtd unknown /proc/28006/task/28039/root (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28039 cpu_tp ceph txt unknown /proc/28006/task/28039/exe (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28039 cpu_tp ceph NOFD /proc/28006/task/28039/fd (opendir: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28040 ms_dispat ceph cwd unknown /proc/28006/task/28040/cwd (readlink: Permission denied) 2024-04-22T13:30:08.513 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28040 ms_dispat ceph rtd unknown /proc/28006/task/28040/root (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28040 ms_dispat ceph txt unknown /proc/28006/task/28040/exe (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28040 ms_dispat ceph NOFD /proc/28006/task/28040/fd (opendir: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28041 ms_local ceph cwd unknown /proc/28006/task/28041/cwd (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28041 ms_local ceph rtd unknown /proc/28006/task/28041/root (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28041 ms_local ceph txt unknown /proc/28006/task/28041/exe (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28041 ms_local ceph NOFD /proc/28006/task/28041/fd (opendir: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28042 safe_time ceph cwd unknown /proc/28006/task/28042/cwd (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28042 safe_time ceph rtd unknown /proc/28006/task/28042/root (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28042 safe_time ceph txt unknown /proc/28006/task/28042/exe (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28042 safe_time ceph NOFD /proc/28006/task/28042/fd (opendir: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28043 ms_dispat ceph cwd unknown /proc/28006/task/28043/cwd (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28043 ms_dispat ceph rtd unknown /proc/28006/task/28043/root (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28043 ms_dispat ceph txt unknown /proc/28006/task/28043/exe (readlink: Permission denied) 2024-04-22T13:30:08.514 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28043 ms_dispat ceph NOFD /proc/28006/task/28043/fd (opendir: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28044 ms_local ceph cwd unknown /proc/28006/task/28044/cwd (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28044 ms_local ceph rtd unknown /proc/28006/task/28044/root (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28044 ms_local ceph txt unknown /proc/28006/task/28044/exe (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 28044 ms_local ceph NOFD /proc/28006/task/28044/fd (opendir: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 67819 rstore_co ceph cwd unknown /proc/28006/task/67819/cwd (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 67819 rstore_co ceph rtd unknown /proc/28006/task/67819/root (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 67819 rstore_co ceph txt unknown /proc/28006/task/67819/exe (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mon 28006 67819 rstore_co ceph NOFD /proc/28006/task/67819/fd (opendir: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28236 root cwd unknown /proc/28236/cwd (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28236 root rtd unknown /proc/28236/root (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28236 root txt unknown /proc/28236/exe (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 28236 root NOFD /proc/28236/fd (opendir: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28238 root cwd unknown /proc/28238/cwd (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28238 root rtd unknown /proc/28238/root (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28238 root txt unknown /proc/28238/exe (readlink: Permission denied) 2024-04-22T13:30:08.515 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 28238 root NOFD /proc/28238/fd (opendir: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 ceph cwd unknown /proc/28240/cwd (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 ceph rtd unknown /proc/28240/root (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 ceph txt unknown /proc/28240/exe (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 ceph NOFD /proc/28240/fd (opendir: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80255 log ceph cwd unknown /proc/28240/task/80255/cwd (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80255 log ceph rtd unknown /proc/28240/task/80255/root (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80255 log ceph txt unknown /proc/28240/task/80255/exe (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80255 log ceph NOFD /proc/28240/task/80255/fd (opendir: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80257 msgr-work ceph cwd unknown /proc/28240/task/80257/cwd (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80257 msgr-work ceph rtd unknown /proc/28240/task/80257/root (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80257 msgr-work ceph txt unknown /proc/28240/task/80257/exe (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80257 msgr-work ceph NOFD /proc/28240/task/80257/fd (opendir: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80258 msgr-work ceph cwd unknown /proc/28240/task/80258/cwd (readlink: Permission denied) 2024-04-22T13:30:08.516 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80258 msgr-work ceph rtd unknown /proc/28240/task/80258/root (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80258 msgr-work ceph txt unknown /proc/28240/task/80258/exe (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80258 msgr-work ceph NOFD /proc/28240/task/80258/fd (opendir: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80259 msgr-work ceph cwd unknown /proc/28240/task/80259/cwd (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80259 msgr-work ceph rtd unknown /proc/28240/task/80259/root (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80259 msgr-work ceph txt unknown /proc/28240/task/80259/exe (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80259 msgr-work ceph NOFD /proc/28240/task/80259/fd (opendir: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80263 service ceph cwd unknown /proc/28240/task/80263/cwd (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80263 service ceph rtd unknown /proc/28240/task/80263/root (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80263 service ceph txt unknown /proc/28240/task/80263/exe (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80263 service ceph NOFD /proc/28240/task/80263/fd (opendir: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80264 admin_soc ceph cwd unknown /proc/28240/task/80264/cwd (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80264 admin_soc ceph rtd unknown /proc/28240/task/80264/root (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80264 admin_soc ceph txt unknown /proc/28240/task/80264/exe (readlink: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80264 admin_soc ceph NOFD /proc/28240/task/80264/fd (opendir: Permission denied) 2024-04-22T13:30:08.517 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80265 ceph_time ceph cwd unknown /proc/28240/task/80265/cwd (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80265 ceph_time ceph rtd unknown /proc/28240/task/80265/root (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80265 ceph_time ceph txt unknown /proc/28240/task/80265/exe (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80265 ceph_time ceph NOFD /proc/28240/task/80265/fd (opendir: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80266 fn_anonym ceph cwd unknown /proc/28240/task/80266/cwd (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80266 fn_anonym ceph rtd unknown /proc/28240/task/80266/root (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80266 fn_anonym ceph txt unknown /proc/28240/task/80266/exe (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80266 fn_anonym ceph NOFD /proc/28240/task/80266/fd (opendir: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80267 signal_ha ceph cwd unknown /proc/28240/task/80267/cwd (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80267 signal_ha ceph rtd unknown /proc/28240/task/80267/root (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80267 signal_ha ceph txt unknown /proc/28240/task/80267/exe (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80267 signal_ha ceph NOFD /proc/28240/task/80267/fd (opendir: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80268 mgrsb-fin ceph cwd unknown /proc/28240/task/80268/cwd (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80268 mgrsb-fin ceph rtd unknown /proc/28240/task/80268/root (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80268 mgrsb-fin ceph txt unknown /proc/28240/task/80268/exe (readlink: Permission denied) 2024-04-22T13:30:08.518 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80268 mgrsb-fin ceph NOFD /proc/28240/task/80268/fd (opendir: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80269 ms_dispat ceph cwd unknown /proc/28240/task/80269/cwd (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80269 ms_dispat ceph rtd unknown /proc/28240/task/80269/root (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80269 ms_dispat ceph txt unknown /proc/28240/task/80269/exe (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80269 ms_dispat ceph NOFD /proc/28240/task/80269/fd (opendir: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80270 ms_local ceph cwd unknown /proc/28240/task/80270/cwd (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80270 ms_local ceph rtd unknown /proc/28240/task/80270/root (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80270 ms_local ceph txt unknown /proc/28240/task/80270/exe (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80270 ms_local ceph NOFD /proc/28240/task/80270/fd (opendir: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80271 io_contex ceph cwd unknown /proc/28240/task/80271/cwd (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80271 io_contex ceph rtd unknown /proc/28240/task/80271/root (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80271 io_contex ceph txt unknown /proc/28240/task/80271/exe (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80271 io_contex ceph NOFD /proc/28240/task/80271/fd (opendir: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80272 io_contex ceph cwd unknown /proc/28240/task/80272/cwd (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80272 io_contex ceph rtd unknown /proc/28240/task/80272/root (readlink: Permission denied) 2024-04-22T13:30:08.519 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80272 io_contex ceph txt unknown /proc/28240/task/80272/exe (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80272 io_contex ceph NOFD /proc/28240/task/80272/fd (opendir: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80273 safe_time ceph cwd unknown /proc/28240/task/80273/cwd (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80273 safe_time ceph rtd unknown /proc/28240/task/80273/root (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80273 safe_time ceph txt unknown /proc/28240/task/80273/exe (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80273 safe_time ceph NOFD /proc/28240/task/80273/fd (opendir: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80274 safe_time ceph cwd unknown /proc/28240/task/80274/cwd (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80274 safe_time ceph rtd unknown /proc/28240/task/80274/root (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80274 safe_time ceph txt unknown /proc/28240/task/80274/exe (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80274 safe_time ceph NOFD /proc/28240/task/80274/fd (opendir: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80275 safe_time ceph cwd unknown /proc/28240/task/80275/cwd (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80275 safe_time ceph rtd unknown /proc/28240/task/80275/root (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80275 safe_time ceph txt unknown /proc/28240/task/80275/exe (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80275 safe_time ceph NOFD /proc/28240/task/80275/fd (opendir: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80276 fn_anonym ceph cwd unknown /proc/28240/task/80276/cwd (readlink: Permission denied) 2024-04-22T13:30:08.520 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80276 fn_anonym ceph rtd unknown /proc/28240/task/80276/root (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80276 fn_anonym ceph txt unknown /proc/28240/task/80276/exe (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80276 fn_anonym ceph NOFD /proc/28240/task/80276/fd (opendir: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80277 flusher ceph cwd unknown /proc/28240/task/80277/cwd (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80277 flusher ceph rtd unknown /proc/28240/task/80277/root (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80277 flusher ceph txt unknown /proc/28240/task/80277/exe (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80277 flusher ceph NOFD /proc/28240/task/80277/fd (opendir: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80278 safe_time ceph cwd unknown /proc/28240/task/80278/cwd (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80278 safe_time ceph rtd unknown /proc/28240/task/80278/root (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80278 safe_time ceph txt unknown /proc/28240/task/80278/exe (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 80278 safe_time ceph NOFD /proc/28240/task/80278/fd (opendir: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81118 mgr-fin ceph cwd unknown /proc/28240/task/81118/cwd (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81118 mgr-fin ceph rtd unknown /proc/28240/task/81118/root (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81118 mgr-fin ceph txt unknown /proc/28240/task/81118/exe (readlink: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81118 mgr-fin ceph NOFD /proc/28240/task/81118/fd (opendir: Permission denied) 2024-04-22T13:30:08.521 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81119 ms_dispat ceph cwd unknown /proc/28240/task/81119/cwd (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81119 ms_dispat ceph rtd unknown /proc/28240/task/81119/root (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81119 ms_dispat ceph txt unknown /proc/28240/task/81119/exe (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81119 ms_dispat ceph NOFD /proc/28240/task/81119/fd (opendir: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81120 ms_local ceph cwd unknown /proc/28240/task/81120/cwd (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81120 ms_local ceph rtd unknown /proc/28240/task/81120/root (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81120 ms_local ceph txt unknown /proc/28240/task/81120/exe (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81120 ms_local ceph NOFD /proc/28240/task/81120/fd (opendir: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81121 safe_time ceph cwd unknown /proc/28240/task/81121/cwd (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81121 safe_time ceph rtd unknown /proc/28240/task/81121/root (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81121 safe_time ceph txt unknown /proc/28240/task/81121/exe (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81121 safe_time ceph NOFD /proc/28240/task/81121/fd (opendir: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81122 cmdfin ceph cwd unknown /proc/28240/task/81122/cwd (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81122 cmdfin ceph rtd unknown /proc/28240/task/81122/root (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81122 cmdfin ceph txt unknown /proc/28240/task/81122/exe (readlink: Permission denied) 2024-04-22T13:30:08.522 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81122 cmdfin ceph NOFD /proc/28240/task/81122/fd (opendir: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81126 io_contex ceph cwd unknown /proc/28240/task/81126/cwd (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81126 io_contex ceph rtd unknown /proc/28240/task/81126/root (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81126 io_contex ceph txt unknown /proc/28240/task/81126/exe (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81126 io_contex ceph NOFD /proc/28240/task/81126/fd (opendir: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81127 io_contex ceph cwd unknown /proc/28240/task/81127/cwd (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81127 io_contex ceph rtd unknown /proc/28240/task/81127/root (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81127 io_contex ceph txt unknown /proc/28240/task/81127/exe (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81127 io_contex ceph NOFD /proc/28240/task/81127/fd (opendir: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81128 ceph_time ceph cwd unknown /proc/28240/task/81128/cwd (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81128 ceph_time ceph rtd unknown /proc/28240/task/81128/root (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81128 ceph_time ceph txt unknown /proc/28240/task/81128/exe (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81128 ceph_time ceph NOFD /proc/28240/task/81128/fd (opendir: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81129 ms_dispat ceph cwd unknown /proc/28240/task/81129/cwd (readlink: Permission denied) 2024-04-22T13:30:08.523 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81129 ms_dispat ceph rtd unknown /proc/28240/task/81129/root (readlink: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81129 ms_dispat ceph txt unknown /proc/28240/task/81129/exe (readlink: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81129 ms_dispat ceph NOFD /proc/28240/task/81129/fd (opendir: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81130 ms_local ceph cwd unknown /proc/28240/task/81130/cwd (readlink: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81130 ms_local ceph rtd unknown /proc/28240/task/81130/root (readlink: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81130 ms_local ceph txt unknown /proc/28240/task/81130/exe (readlink: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81130 ms_local ceph NOFD /proc/28240/task/81130/fd (opendir: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81131 safe_time ceph cwd unknown /proc/28240/task/81131/cwd (readlink: Permission denied) 2024-04-22T13:30:08.524 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81131 safe_time ceph rtd unknown /proc/28240/task/81131/root (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81131 safe_time ceph txt unknown /proc/28240/task/81131/exe (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81131 safe_time ceph NOFD /proc/28240/task/81131/fd (opendir: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81132 safe_time ceph cwd unknown /proc/28240/task/81132/cwd (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81132 safe_time ceph rtd unknown /proc/28240/task/81132/root (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81132 safe_time ceph txt unknown /proc/28240/task/81132/exe (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81132 safe_time ceph NOFD /proc/28240/task/81132/fd (opendir: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81133 balancer ceph cwd unknown /proc/28240/task/81133/cwd (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81133 balancer ceph rtd unknown /proc/28240/task/81133/root (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81133 balancer ceph txt unknown /proc/28240/task/81133/exe (readlink: Permission denied) 2024-04-22T13:30:08.525 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81133 balancer ceph NOFD /proc/28240/task/81133/fd (opendir: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81134 m-fin-bal ceph cwd unknown /proc/28240/task/81134/cwd (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81134 m-fin-bal ceph rtd unknown /proc/28240/task/81134/root (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81134 m-fin-bal ceph txt unknown /proc/28240/task/81134/exe (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81134 m-fin-bal ceph NOFD /proc/28240/task/81134/fd (opendir: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81135 mgr-fin ceph cwd unknown /proc/28240/task/81135/cwd (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81135 mgr-fin ceph rtd unknown /proc/28240/task/81135/root (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81135 mgr-fin ceph txt unknown /proc/28240/task/81135/exe (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81135 mgr-fin ceph NOFD /proc/28240/task/81135/fd (opendir: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81136 mgr-fin ceph cwd unknown /proc/28240/task/81136/cwd (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81136 mgr-fin ceph rtd unknown /proc/28240/task/81136/root (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81136 mgr-fin ceph txt unknown /proc/28240/task/81136/exe (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81136 mgr-fin ceph NOFD /proc/28240/task/81136/fd (opendir: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81137 mgr-fin ceph cwd unknown /proc/28240/task/81137/cwd (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81137 mgr-fin ceph rtd unknown /proc/28240/task/81137/root (readlink: Permission denied) 2024-04-22T13:30:08.526 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81137 mgr-fin ceph txt unknown /proc/28240/task/81137/exe (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81137 mgr-fin ceph NOFD /proc/28240/task/81137/fd (opendir: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81138 mgr-fin ceph cwd unknown /proc/28240/task/81138/cwd (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81138 mgr-fin ceph rtd unknown /proc/28240/task/81138/root (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81138 mgr-fin ceph txt unknown /proc/28240/task/81138/exe (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81138 mgr-fin ceph NOFD /proc/28240/task/81138/fd (opendir: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81139 mgr-fin ceph cwd unknown /proc/28240/task/81139/cwd (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81139 mgr-fin ceph rtd unknown /proc/28240/task/81139/root (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81139 mgr-fin ceph txt unknown /proc/28240/task/81139/exe (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81139 mgr-fin ceph NOFD /proc/28240/task/81139/fd (opendir: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81140 mgr-fin ceph cwd unknown /proc/28240/task/81140/cwd (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81140 mgr-fin ceph rtd unknown /proc/28240/task/81140/root (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81140 mgr-fin ceph txt unknown /proc/28240/task/81140/exe (readlink: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81140 mgr-fin ceph NOFD /proc/28240/task/81140/fd (opendir: Permission denied) 2024-04-22T13:30:08.527 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81141 mgr-fin ceph cwd unknown /proc/28240/task/81141/cwd (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81141 mgr-fin ceph rtd unknown /proc/28240/task/81141/root (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81141 mgr-fin ceph txt unknown /proc/28240/task/81141/exe (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81141 mgr-fin ceph NOFD /proc/28240/task/81141/fd (opendir: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81142 mgr-fin ceph cwd unknown /proc/28240/task/81142/cwd (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81142 mgr-fin ceph rtd unknown /proc/28240/task/81142/root (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81142 mgr-fin ceph txt unknown /proc/28240/task/81142/exe (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81142 mgr-fin ceph NOFD /proc/28240/task/81142/fd (opendir: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81143 mgr-fin ceph cwd unknown /proc/28240/task/81143/cwd (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81143 mgr-fin ceph rtd unknown /proc/28240/task/81143/root (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81143 mgr-fin ceph txt unknown /proc/28240/task/81143/exe (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81143 mgr-fin ceph NOFD /proc/28240/task/81143/fd (opendir: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81144 mgr-fin ceph cwd unknown /proc/28240/task/81144/cwd (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81144 mgr-fin ceph rtd unknown /proc/28240/task/81144/root (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81144 mgr-fin ceph txt unknown /proc/28240/task/81144/exe (readlink: Permission denied) 2024-04-22T13:30:08.528 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81144 mgr-fin ceph NOFD /proc/28240/task/81144/fd (opendir: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81145 mgr-fin ceph cwd unknown /proc/28240/task/81145/cwd (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81145 mgr-fin ceph rtd unknown /proc/28240/task/81145/root (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81145 mgr-fin ceph txt unknown /proc/28240/task/81145/exe (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81145 mgr-fin ceph NOFD /proc/28240/task/81145/fd (opendir: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81146 mgr-fin ceph cwd unknown /proc/28240/task/81146/cwd (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81146 mgr-fin ceph rtd unknown /proc/28240/task/81146/root (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81146 mgr-fin ceph txt unknown /proc/28240/task/81146/exe (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81146 mgr-fin ceph NOFD /proc/28240/task/81146/fd (opendir: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81147 mgr-fin ceph cwd unknown /proc/28240/task/81147/cwd (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81147 mgr-fin ceph rtd unknown /proc/28240/task/81147/root (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81147 mgr-fin ceph txt unknown /proc/28240/task/81147/exe (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81147 mgr-fin ceph NOFD /proc/28240/task/81147/fd (opendir: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81148 mgr-fin ceph cwd unknown /proc/28240/task/81148/cwd (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81148 mgr-fin ceph rtd unknown /proc/28240/task/81148/root (readlink: Permission denied) 2024-04-22T13:30:08.529 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81148 mgr-fin ceph txt unknown /proc/28240/task/81148/exe (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81148 mgr-fin ceph NOFD /proc/28240/task/81148/fd (opendir: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81149 mgr-fin ceph cwd unknown /proc/28240/task/81149/cwd (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81149 mgr-fin ceph rtd unknown /proc/28240/task/81149/root (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81149 mgr-fin ceph txt unknown /proc/28240/task/81149/exe (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81149 mgr-fin ceph NOFD /proc/28240/task/81149/fd (opendir: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81150 mgr-fin ceph cwd unknown /proc/28240/task/81150/cwd (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81150 mgr-fin ceph rtd unknown /proc/28240/task/81150/root (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81150 mgr-fin ceph txt unknown /proc/28240/task/81150/exe (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81150 mgr-fin ceph NOFD /proc/28240/task/81150/fd (opendir: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81151 cephadm ceph cwd unknown /proc/28240/task/81151/cwd (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81151 cephadm ceph rtd unknown /proc/28240/task/81151/root (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81151 cephadm ceph txt unknown /proc/28240/task/81151/exe (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81151 cephadm ceph NOFD /proc/28240/task/81151/fd (opendir: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81152 m-fin-cep ceph cwd unknown /proc/28240/task/81152/cwd (readlink: Permission denied) 2024-04-22T13:30:08.530 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81152 m-fin-cep ceph rtd unknown /proc/28240/task/81152/root (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81152 m-fin-cep ceph txt unknown /proc/28240/task/81152/exe (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81152 m-fin-cep ceph NOFD /proc/28240/task/81152/fd (opendir: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81153 mgr-fin ceph cwd unknown /proc/28240/task/81153/cwd (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81153 mgr-fin ceph rtd unknown /proc/28240/task/81153/root (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81153 mgr-fin ceph txt unknown /proc/28240/task/81153/exe (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81153 mgr-fin ceph NOFD /proc/28240/task/81153/fd (opendir: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81154 mgr-fin ceph cwd unknown /proc/28240/task/81154/cwd (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81154 mgr-fin ceph rtd unknown /proc/28240/task/81154/root (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81154 mgr-fin ceph txt unknown /proc/28240/task/81154/exe (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81154 mgr-fin ceph NOFD /proc/28240/task/81154/fd (opendir: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81155 crash ceph cwd unknown /proc/28240/task/81155/cwd (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81155 crash ceph rtd unknown /proc/28240/task/81155/root (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81155 crash ceph txt unknown /proc/28240/task/81155/exe (readlink: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81155 crash ceph NOFD /proc/28240/task/81155/fd (opendir: Permission denied) 2024-04-22T13:30:08.531 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81156 m-fin-cra ceph cwd unknown /proc/28240/task/81156/cwd (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81156 m-fin-cra ceph rtd unknown /proc/28240/task/81156/root (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81156 m-fin-cra ceph txt unknown /proc/28240/task/81156/exe (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81156 m-fin-cra ceph NOFD /proc/28240/task/81156/fd (opendir: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81157 mgr-fin ceph cwd unknown /proc/28240/task/81157/cwd (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81157 mgr-fin ceph rtd unknown /proc/28240/task/81157/root (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81157 mgr-fin ceph txt unknown /proc/28240/task/81157/exe (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81157 mgr-fin ceph NOFD /proc/28240/task/81157/fd (opendir: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81158 cephadm ceph cwd unknown /proc/28240/task/81158/cwd (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81158 cephadm ceph rtd unknown /proc/28240/task/81158/root (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81158 cephadm ceph txt unknown /proc/28240/task/81158/exe (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81158 cephadm ceph NOFD /proc/28240/task/81158/fd (opendir: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81159 dashboard ceph cwd unknown /proc/28240/task/81159/cwd (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81159 dashboard ceph rtd unknown /proc/28240/task/81159/root (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81159 dashboard ceph txt unknown /proc/28240/task/81159/exe (readlink: Permission denied) 2024-04-22T13:30:08.532 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81159 dashboard ceph NOFD /proc/28240/task/81159/fd (opendir: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81160 m-fin-das ceph cwd unknown /proc/28240/task/81160/cwd (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81160 m-fin-das ceph rtd unknown /proc/28240/task/81160/root (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81160 m-fin-das ceph txt unknown /proc/28240/task/81160/exe (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81160 m-fin-das ceph NOFD /proc/28240/task/81160/fd (opendir: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81161 mgr-fin ceph cwd unknown /proc/28240/task/81161/cwd (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81161 mgr-fin ceph rtd unknown /proc/28240/task/81161/root (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81161 mgr-fin ceph txt unknown /proc/28240/task/81161/exe (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81161 mgr-fin ceph NOFD /proc/28240/task/81161/fd (opendir: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81162 mgr-fin ceph cwd unknown /proc/28240/task/81162/cwd (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81162 mgr-fin ceph rtd unknown /proc/28240/task/81162/root (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81162 mgr-fin ceph txt unknown /proc/28240/task/81162/exe (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81162 mgr-fin ceph NOFD /proc/28240/task/81162/fd (opendir: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81163 devicehea ceph cwd unknown /proc/28240/task/81163/cwd (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81163 devicehea ceph rtd unknown /proc/28240/task/81163/root (readlink: Permission denied) 2024-04-22T13:30:08.533 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81163 devicehea ceph txt unknown /proc/28240/task/81163/exe (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81163 devicehea ceph NOFD /proc/28240/task/81163/fd (opendir: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81164 m-fin-dev ceph cwd unknown /proc/28240/task/81164/cwd (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81164 m-fin-dev ceph rtd unknown /proc/28240/task/81164/root (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81164 m-fin-dev ceph txt unknown /proc/28240/task/81164/exe (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81164 m-fin-dev ceph NOFD /proc/28240/task/81164/fd (opendir: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81165 mgr-fin ceph cwd unknown /proc/28240/task/81165/cwd (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81165 mgr-fin ceph rtd unknown /proc/28240/task/81165/root (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81165 mgr-fin ceph txt unknown /proc/28240/task/81165/exe (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81165 mgr-fin ceph NOFD /proc/28240/task/81165/fd (opendir: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81167 m-fin-ios ceph cwd unknown /proc/28240/task/81167/cwd (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81167 m-fin-ios ceph rtd unknown /proc/28240/task/81167/root (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81167 m-fin-ios ceph txt unknown /proc/28240/task/81167/exe (readlink: Permission denied) 2024-04-22T13:30:08.534 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81167 m-fin-ios ceph NOFD /proc/28240/task/81167/fd (opendir: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81168 mgr-fin ceph cwd unknown /proc/28240/task/81168/cwd (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81168 mgr-fin ceph rtd unknown /proc/28240/task/81168/root (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81168 mgr-fin ceph txt unknown /proc/28240/task/81168/exe (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81168 mgr-fin ceph NOFD /proc/28240/task/81168/fd (opendir: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81169 mgr-fin ceph cwd unknown /proc/28240/task/81169/cwd (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81169 mgr-fin ceph rtd unknown /proc/28240/task/81169/root (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81169 mgr-fin ceph txt unknown /proc/28240/task/81169/exe (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81169 mgr-fin ceph NOFD /proc/28240/task/81169/fd (opendir: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81171 m-fin-nfs ceph cwd unknown /proc/28240/task/81171/cwd (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81171 m-fin-nfs ceph rtd unknown /proc/28240/task/81171/root (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81171 m-fin-nfs ceph txt unknown /proc/28240/task/81171/exe (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81171 m-fin-nfs ceph NOFD /proc/28240/task/81171/fd (opendir: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81172 mgr-fin ceph cwd unknown /proc/28240/task/81172/cwd (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81172 mgr-fin ceph rtd unknown /proc/28240/task/81172/root (readlink: Permission denied) 2024-04-22T13:30:08.535 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81172 mgr-fin ceph txt unknown /proc/28240/task/81172/exe (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81172 mgr-fin ceph NOFD /proc/28240/task/81172/fd (opendir: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81173 mgr-fin ceph cwd unknown /proc/28240/task/81173/cwd (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81173 mgr-fin ceph rtd unknown /proc/28240/task/81173/root (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81173 mgr-fin ceph txt unknown /proc/28240/task/81173/exe (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81173 mgr-fin ceph NOFD /proc/28240/task/81173/fd (opendir: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81175 m-fin-orc ceph cwd unknown /proc/28240/task/81175/cwd (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81175 m-fin-orc ceph rtd unknown /proc/28240/task/81175/root (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81175 m-fin-orc ceph txt unknown /proc/28240/task/81175/exe (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81175 m-fin-orc ceph NOFD /proc/28240/task/81175/fd (opendir: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81176 mgr-fin ceph cwd unknown /proc/28240/task/81176/cwd (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81176 mgr-fin ceph rtd unknown /proc/28240/task/81176/root (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81176 mgr-fin ceph txt unknown /proc/28240/task/81176/exe (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81176 mgr-fin ceph NOFD /proc/28240/task/81176/fd (opendir: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81177 mgr-fin ceph cwd unknown /proc/28240/task/81177/cwd (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81177 mgr-fin ceph rtd unknown /proc/28240/task/81177/root (readlink: Permission denied) 2024-04-22T13:30:08.536 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81177 mgr-fin ceph txt unknown /proc/28240/task/81177/exe (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81177 mgr-fin ceph NOFD /proc/28240/task/81177/fd (opendir: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81178 pg_autosc ceph cwd unknown /proc/28240/task/81178/cwd (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81178 pg_autosc ceph rtd unknown /proc/28240/task/81178/root (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81178 pg_autosc ceph txt unknown /proc/28240/task/81178/exe (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81178 pg_autosc ceph NOFD /proc/28240/task/81178/fd (opendir: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81179 m-fin-pg_ ceph cwd unknown /proc/28240/task/81179/cwd (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81179 m-fin-pg_ ceph rtd unknown /proc/28240/task/81179/root (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81179 m-fin-pg_ ceph txt unknown /proc/28240/task/81179/exe (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81179 m-fin-pg_ ceph NOFD /proc/28240/task/81179/fd (opendir: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81180 progress ceph cwd unknown /proc/28240/task/81180/cwd (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81180 progress ceph rtd unknown /proc/28240/task/81180/root (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81180 progress ceph txt unknown /proc/28240/task/81180/exe (readlink: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81180 progress ceph NOFD /proc/28240/task/81180/fd (opendir: Permission denied) 2024-04-22T13:30:08.537 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81181 m-fin-pro ceph cwd unknown /proc/28240/task/81181/cwd (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81181 m-fin-pro ceph rtd unknown /proc/28240/task/81181/root (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81181 m-fin-pro ceph txt unknown /proc/28240/task/81181/exe (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81181 m-fin-pro ceph NOFD /proc/28240/task/81181/fd (opendir: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81185 io_contex ceph cwd unknown /proc/28240/task/81185/cwd (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81185 io_contex ceph rtd unknown /proc/28240/task/81185/root (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81185 io_contex ceph txt unknown /proc/28240/task/81185/exe (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81185 io_contex ceph NOFD /proc/28240/task/81185/fd (opendir: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81186 io_contex ceph cwd unknown /proc/28240/task/81186/cwd (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81186 io_contex ceph rtd unknown /proc/28240/task/81186/root (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81186 io_contex ceph txt unknown /proc/28240/task/81186/exe (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81186 io_contex ceph NOFD /proc/28240/task/81186/fd (opendir: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81187 ceph_time ceph cwd unknown /proc/28240/task/81187/cwd (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81187 ceph_time ceph rtd unknown /proc/28240/task/81187/root (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81187 ceph_time ceph txt unknown /proc/28240/task/81187/exe (readlink: Permission denied) 2024-04-22T13:30:08.538 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81187 ceph_time ceph NOFD /proc/28240/task/81187/fd (opendir: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81188 ms_dispat ceph cwd unknown /proc/28240/task/81188/cwd (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81188 ms_dispat ceph rtd unknown /proc/28240/task/81188/root (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81188 ms_dispat ceph txt unknown /proc/28240/task/81188/exe (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81188 ms_dispat ceph NOFD /proc/28240/task/81188/fd (opendir: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81189 ms_local ceph cwd unknown /proc/28240/task/81189/cwd (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81189 ms_local ceph rtd unknown /proc/28240/task/81189/root (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81189 ms_local ceph txt unknown /proc/28240/task/81189/exe (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81189 ms_local ceph NOFD /proc/28240/task/81189/fd (opendir: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81190 safe_time ceph cwd unknown /proc/28240/task/81190/cwd (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81190 safe_time ceph rtd unknown /proc/28240/task/81190/root (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81190 safe_time ceph txt unknown /proc/28240/task/81190/exe (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81190 safe_time ceph NOFD /proc/28240/task/81190/fd (opendir: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81191 safe_time ceph cwd unknown /proc/28240/task/81191/cwd (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81191 safe_time ceph rtd unknown /proc/28240/task/81191/root (readlink: Permission denied) 2024-04-22T13:30:08.539 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81191 safe_time ceph txt unknown /proc/28240/task/81191/exe (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81191 safe_time ceph NOFD /proc/28240/task/81191/fd (opendir: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81192 mgr-fin ceph cwd unknown /proc/28240/task/81192/cwd (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81192 mgr-fin ceph rtd unknown /proc/28240/task/81192/root (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81192 mgr-fin ceph txt unknown /proc/28240/task/81192/exe (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81192 mgr-fin ceph NOFD /proc/28240/task/81192/fd (opendir: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81194 m-fin-rbd ceph cwd unknown /proc/28240/task/81194/cwd (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81194 m-fin-rbd ceph rtd unknown /proc/28240/task/81194/root (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81194 m-fin-rbd ceph txt unknown /proc/28240/task/81194/exe (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81194 m-fin-rbd ceph NOFD /proc/28240/task/81194/fd (opendir: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81195 restful ceph cwd unknown /proc/28240/task/81195/cwd (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81195 restful ceph rtd unknown /proc/28240/task/81195/root (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81195 restful ceph txt unknown /proc/28240/task/81195/exe (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81195 restful ceph NOFD /proc/28240/task/81195/fd (opendir: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81196 m-fin-res ceph cwd unknown /proc/28240/task/81196/cwd (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81196 m-fin-res ceph rtd unknown /proc/28240/task/81196/root (readlink: Permission denied) 2024-04-22T13:30:08.540 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81196 m-fin-res ceph txt unknown /proc/28240/task/81196/exe (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81196 m-fin-res ceph NOFD /proc/28240/task/81196/fd (opendir: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81197 mgr-fin ceph cwd unknown /proc/28240/task/81197/cwd (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81197 mgr-fin ceph rtd unknown /proc/28240/task/81197/root (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81197 mgr-fin ceph txt unknown /proc/28240/task/81197/exe (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81197 mgr-fin ceph NOFD /proc/28240/task/81197/fd (opendir: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81198 mgr-fin ceph cwd unknown /proc/28240/task/81198/cwd (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81198 mgr-fin ceph rtd unknown /proc/28240/task/81198/root (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81198 mgr-fin ceph txt unknown /proc/28240/task/81198/exe (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81198 mgr-fin ceph NOFD /proc/28240/task/81198/fd (opendir: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81199 mgr-fin ceph cwd unknown /proc/28240/task/81199/cwd (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81199 mgr-fin ceph rtd unknown /proc/28240/task/81199/root (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81199 mgr-fin ceph txt unknown /proc/28240/task/81199/exe (readlink: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81199 mgr-fin ceph NOFD /proc/28240/task/81199/fd (opendir: Permission denied) 2024-04-22T13:30:08.541 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81200 mgr-fin ceph cwd unknown /proc/28240/task/81200/cwd (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81200 mgr-fin ceph rtd unknown /proc/28240/task/81200/root (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81200 mgr-fin ceph txt unknown /proc/28240/task/81200/exe (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81200 mgr-fin ceph NOFD /proc/28240/task/81200/fd (opendir: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81201 mgr-fin ceph cwd unknown /proc/28240/task/81201/cwd (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81201 mgr-fin ceph rtd unknown /proc/28240/task/81201/root (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81201 mgr-fin ceph txt unknown /proc/28240/task/81201/exe (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81201 mgr-fin ceph NOFD /proc/28240/task/81201/fd (opendir: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81202 mgr-fin ceph cwd unknown /proc/28240/task/81202/cwd (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81202 mgr-fin ceph rtd unknown /proc/28240/task/81202/root (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81202 mgr-fin ceph txt unknown /proc/28240/task/81202/exe (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81202 mgr-fin ceph NOFD /proc/28240/task/81202/fd (opendir: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81206 io_contex ceph cwd unknown /proc/28240/task/81206/cwd (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81206 io_contex ceph rtd unknown /proc/28240/task/81206/root (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81206 io_contex ceph txt unknown /proc/28240/task/81206/exe (readlink: Permission denied) 2024-04-22T13:30:08.542 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81206 io_contex ceph NOFD /proc/28240/task/81206/fd (opendir: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81207 io_contex ceph cwd unknown /proc/28240/task/81207/cwd (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81207 io_contex ceph rtd unknown /proc/28240/task/81207/root (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81207 io_contex ceph txt unknown /proc/28240/task/81207/exe (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81207 io_contex ceph NOFD /proc/28240/task/81207/fd (opendir: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81208 ceph_time ceph cwd unknown /proc/28240/task/81208/cwd (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81208 ceph_time ceph rtd unknown /proc/28240/task/81208/root (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81208 ceph_time ceph txt unknown /proc/28240/task/81208/exe (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81208 ceph_time ceph NOFD /proc/28240/task/81208/fd (opendir: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81209 ms_dispat ceph cwd unknown /proc/28240/task/81209/cwd (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81209 ms_dispat ceph rtd unknown /proc/28240/task/81209/root (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81209 ms_dispat ceph txt unknown /proc/28240/task/81209/exe (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81209 ms_dispat ceph NOFD /proc/28240/task/81209/fd (opendir: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81210 ms_local ceph cwd unknown /proc/28240/task/81210/cwd (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81210 ms_local ceph rtd unknown /proc/28240/task/81210/root (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81210 ms_local ceph txt unknown /proc/28240/task/81210/exe (readlink: Permission denied) 2024-04-22T13:30:08.543 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81210 ms_local ceph NOFD /proc/28240/task/81210/fd (opendir: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81211 safe_time ceph cwd unknown /proc/28240/task/81211/cwd (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81211 safe_time ceph rtd unknown /proc/28240/task/81211/root (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81211 safe_time ceph txt unknown /proc/28240/task/81211/exe (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81211 safe_time ceph NOFD /proc/28240/task/81211/fd (opendir: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81212 safe_time ceph cwd unknown /proc/28240/task/81212/cwd (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81212 safe_time ceph rtd unknown /proc/28240/task/81212/root (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81212 safe_time ceph txt unknown /proc/28240/task/81212/exe (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81212 safe_time ceph NOFD /proc/28240/task/81212/fd (opendir: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81216 m-fin-sna ceph cwd unknown /proc/28240/task/81216/cwd (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81216 m-fin-sna ceph rtd unknown /proc/28240/task/81216/root (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81216 m-fin-sna ceph txt unknown /proc/28240/task/81216/exe (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81216 m-fin-sna ceph NOFD /proc/28240/task/81216/fd (opendir: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81218 m-fin-sta ceph cwd unknown /proc/28240/task/81218/cwd (readlink: Permission denied) 2024-04-22T13:30:08.544 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81218 m-fin-sta ceph rtd unknown /proc/28240/task/81218/root (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81218 m-fin-sta ceph txt unknown /proc/28240/task/81218/exe (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81218 m-fin-sta ceph NOFD /proc/28240/task/81218/fd (opendir: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81219 telemetry ceph cwd unknown /proc/28240/task/81219/cwd (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81219 telemetry ceph rtd unknown /proc/28240/task/81219/root (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81219 telemetry ceph txt unknown /proc/28240/task/81219/exe (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81219 telemetry ceph NOFD /proc/28240/task/81219/fd (opendir: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81220 m-fin-tel ceph cwd unknown /proc/28240/task/81220/cwd (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81220 m-fin-tel ceph rtd unknown /proc/28240/task/81220/root (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81220 m-fin-tel ceph txt unknown /proc/28240/task/81220/exe (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81220 m-fin-tel ceph NOFD /proc/28240/task/81220/fd (opendir: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81221 mgr-fin ceph cwd unknown /proc/28240/task/81221/cwd (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81221 mgr-fin ceph rtd unknown /proc/28240/task/81221/root (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81221 mgr-fin ceph txt unknown /proc/28240/task/81221/exe (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81221 mgr-fin ceph NOFD /proc/28240/task/81221/fd (opendir: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81225 io_contex ceph cwd unknown /proc/28240/task/81225/cwd (readlink: Permission denied) 2024-04-22T13:30:08.545 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81225 io_contex ceph rtd unknown /proc/28240/task/81225/root (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81225 io_contex ceph txt unknown /proc/28240/task/81225/exe (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81225 io_contex ceph NOFD /proc/28240/task/81225/fd (opendir: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81226 io_contex ceph cwd unknown /proc/28240/task/81226/cwd (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81226 io_contex ceph rtd unknown /proc/28240/task/81226/root (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81226 io_contex ceph txt unknown /proc/28240/task/81226/exe (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81226 io_contex ceph NOFD /proc/28240/task/81226/fd (opendir: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81227 ceph_time ceph cwd unknown /proc/28240/task/81227/cwd (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81227 ceph_time ceph rtd unknown /proc/28240/task/81227/root (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81227 ceph_time ceph txt unknown /proc/28240/task/81227/exe (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81227 ceph_time ceph NOFD /proc/28240/task/81227/fd (opendir: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81228 ms_dispat ceph cwd unknown /proc/28240/task/81228/cwd (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81228 ms_dispat ceph rtd unknown /proc/28240/task/81228/root (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81228 ms_dispat ceph txt unknown /proc/28240/task/81228/exe (readlink: Permission denied) 2024-04-22T13:30:08.546 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81228 ms_dispat ceph NOFD /proc/28240/task/81228/fd (opendir: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81229 ms_local ceph cwd unknown /proc/28240/task/81229/cwd (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81229 ms_local ceph rtd unknown /proc/28240/task/81229/root (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81229 ms_local ceph txt unknown /proc/28240/task/81229/exe (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81229 ms_local ceph NOFD /proc/28240/task/81229/fd (opendir: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81230 safe_time ceph cwd unknown /proc/28240/task/81230/cwd (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81230 safe_time ceph rtd unknown /proc/28240/task/81230/root (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81230 safe_time ceph txt unknown /proc/28240/task/81230/exe (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81230 safe_time ceph NOFD /proc/28240/task/81230/fd (opendir: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81231 safe_time ceph cwd unknown /proc/28240/task/81231/cwd (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81231 safe_time ceph rtd unknown /proc/28240/task/81231/root (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81231 safe_time ceph txt unknown /proc/28240/task/81231/exe (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81231 safe_time ceph NOFD /proc/28240/task/81231/fd (opendir: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81232 mgr-fin ceph cwd unknown /proc/28240/task/81232/cwd (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81232 mgr-fin ceph rtd unknown /proc/28240/task/81232/root (readlink: Permission denied) 2024-04-22T13:30:08.547 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81232 mgr-fin ceph txt unknown /proc/28240/task/81232/exe (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81232 mgr-fin ceph NOFD /proc/28240/task/81232/fd (opendir: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81233 mgr-fin ceph cwd unknown /proc/28240/task/81233/cwd (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81233 mgr-fin ceph rtd unknown /proc/28240/task/81233/root (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81233 mgr-fin ceph txt unknown /proc/28240/task/81233/exe (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81233 mgr-fin ceph NOFD /proc/28240/task/81233/fd (opendir: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81234 mgr-fin ceph cwd unknown /proc/28240/task/81234/cwd (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81234 mgr-fin ceph rtd unknown /proc/28240/task/81234/root (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81234 mgr-fin ceph txt unknown /proc/28240/task/81234/exe (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81234 mgr-fin ceph NOFD /proc/28240/task/81234/fd (opendir: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81235 mgr-fin ceph cwd unknown /proc/28240/task/81235/cwd (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81235 mgr-fin ceph rtd unknown /proc/28240/task/81235/root (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81235 mgr-fin ceph txt unknown /proc/28240/task/81235/exe (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81235 mgr-fin ceph NOFD /proc/28240/task/81235/fd (opendir: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81236 mgr-fin ceph cwd unknown /proc/28240/task/81236/cwd (readlink: Permission denied) 2024-04-22T13:30:08.548 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81236 mgr-fin ceph rtd unknown /proc/28240/task/81236/root (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81236 mgr-fin ceph txt unknown /proc/28240/task/81236/exe (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81236 mgr-fin ceph NOFD /proc/28240/task/81236/fd (opendir: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81237 mgr-fin ceph cwd unknown /proc/28240/task/81237/cwd (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81237 mgr-fin ceph rtd unknown /proc/28240/task/81237/root (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81237 mgr-fin ceph txt unknown /proc/28240/task/81237/exe (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81237 mgr-fin ceph NOFD /proc/28240/task/81237/fd (opendir: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81238 mgr-fin ceph cwd unknown /proc/28240/task/81238/cwd (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81238 mgr-fin ceph rtd unknown /proc/28240/task/81238/root (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81238 mgr-fin ceph txt unknown /proc/28240/task/81238/exe (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81238 mgr-fin ceph NOFD /proc/28240/task/81238/fd (opendir: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81239 mgr-fin ceph cwd unknown /proc/28240/task/81239/cwd (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81239 mgr-fin ceph rtd unknown /proc/28240/task/81239/root (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81239 mgr-fin ceph txt unknown /proc/28240/task/81239/exe (readlink: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81239 mgr-fin ceph NOFD /proc/28240/task/81239/fd (opendir: Permission denied) 2024-04-22T13:30:08.549 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81240 mgr-fin ceph cwd unknown /proc/28240/task/81240/cwd (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81240 mgr-fin ceph rtd unknown /proc/28240/task/81240/root (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81240 mgr-fin ceph txt unknown /proc/28240/task/81240/exe (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81240 mgr-fin ceph NOFD /proc/28240/task/81240/fd (opendir: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81241 mgr-fin ceph cwd unknown /proc/28240/task/81241/cwd (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81241 mgr-fin ceph rtd unknown /proc/28240/task/81241/root (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81241 mgr-fin ceph txt unknown /proc/28240/task/81241/exe (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81241 mgr-fin ceph NOFD /proc/28240/task/81241/fd (opendir: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81242 mgr-fin ceph cwd unknown /proc/28240/task/81242/cwd (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81242 mgr-fin ceph rtd unknown /proc/28240/task/81242/root (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81242 mgr-fin ceph txt unknown /proc/28240/task/81242/exe (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81242 mgr-fin ceph NOFD /proc/28240/task/81242/fd (opendir: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81243 mgr-fin ceph cwd unknown /proc/28240/task/81243/cwd (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81243 mgr-fin ceph rtd unknown /proc/28240/task/81243/root (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81243 mgr-fin ceph txt unknown /proc/28240/task/81243/exe (readlink: Permission denied) 2024-04-22T13:30:08.550 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81243 mgr-fin ceph NOFD /proc/28240/task/81243/fd (opendir: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81245 m-fin-vol ceph cwd unknown /proc/28240/task/81245/cwd (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81245 m-fin-vol ceph rtd unknown /proc/28240/task/81245/root (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81245 m-fin-vol ceph txt unknown /proc/28240/task/81245/exe (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81245 m-fin-vol ceph NOFD /proc/28240/task/81245/fd (opendir: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81246 io_contex ceph cwd unknown /proc/28240/task/81246/cwd (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81246 io_contex ceph rtd unknown /proc/28240/task/81246/root (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81246 io_contex ceph txt unknown /proc/28240/task/81246/exe (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81246 io_contex ceph NOFD /proc/28240/task/81246/fd (opendir: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81247 io_contex ceph cwd unknown /proc/28240/task/81247/cwd (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81247 io_contex ceph rtd unknown /proc/28240/task/81247/root (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81247 io_contex ceph txt unknown /proc/28240/task/81247/exe (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81247 io_contex ceph NOFD /proc/28240/task/81247/fd (opendir: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81291 dashboard ceph cwd unknown /proc/28240/task/81291/cwd (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81291 dashboard ceph rtd unknown /proc/28240/task/81291/root (readlink: Permission denied) 2024-04-22T13:30:08.551 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81291 dashboard ceph txt unknown /proc/28240/task/81291/exe (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81291 dashboard ceph NOFD /proc/28240/task/81291/fd (opendir: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81292 dashboard ceph cwd unknown /proc/28240/task/81292/cwd (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81292 dashboard ceph rtd unknown /proc/28240/task/81292/root (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81292 dashboard ceph txt unknown /proc/28240/task/81292/exe (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81292 dashboard ceph NOFD /proc/28240/task/81292/fd (opendir: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81293 dashboard ceph cwd unknown /proc/28240/task/81293/cwd (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81293 dashboard ceph rtd unknown /proc/28240/task/81293/root (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81293 dashboard ceph txt unknown /proc/28240/task/81293/exe (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81293 dashboard ceph NOFD /proc/28240/task/81293/fd (opendir: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81294 dashboard ceph cwd unknown /proc/28240/task/81294/cwd (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81294 dashboard ceph rtd unknown /proc/28240/task/81294/root (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81294 dashboard ceph txt unknown /proc/28240/task/81294/exe (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81294 dashboard ceph NOFD /proc/28240/task/81294/fd (opendir: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81296 dashboard ceph cwd unknown /proc/28240/task/81296/cwd (readlink: Permission denied) 2024-04-22T13:30:08.552 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81296 dashboard ceph rtd unknown /proc/28240/task/81296/root (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81296 dashboard ceph txt unknown /proc/28240/task/81296/exe (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81296 dashboard ceph NOFD /proc/28240/task/81296/fd (opendir: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81297 dashboard ceph cwd unknown /proc/28240/task/81297/cwd (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81297 dashboard ceph rtd unknown /proc/28240/task/81297/root (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81297 dashboard ceph txt unknown /proc/28240/task/81297/exe (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81297 dashboard ceph NOFD /proc/28240/task/81297/fd (opendir: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81298 dashboard ceph cwd unknown /proc/28240/task/81298/cwd (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81298 dashboard ceph rtd unknown /proc/28240/task/81298/root (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81298 dashboard ceph txt unknown /proc/28240/task/81298/exe (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81298 dashboard ceph NOFD /proc/28240/task/81298/fd (opendir: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81299 dashboard ceph cwd unknown /proc/28240/task/81299/cwd (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81299 dashboard ceph rtd unknown /proc/28240/task/81299/root (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81299 dashboard ceph txt unknown /proc/28240/task/81299/exe (readlink: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81299 dashboard ceph NOFD /proc/28240/task/81299/fd (opendir: Permission denied) 2024-04-22T13:30:08.553 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81300 dashboard ceph cwd unknown /proc/28240/task/81300/cwd (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81300 dashboard ceph rtd unknown /proc/28240/task/81300/root (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81300 dashboard ceph txt unknown /proc/28240/task/81300/exe (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81300 dashboard ceph NOFD /proc/28240/task/81300/fd (opendir: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81302 dashboard ceph cwd unknown /proc/28240/task/81302/cwd (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81302 dashboard ceph rtd unknown /proc/28240/task/81302/root (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81302 dashboard ceph txt unknown /proc/28240/task/81302/exe (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81302 dashboard ceph NOFD /proc/28240/task/81302/fd (opendir: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81304 dashboard ceph cwd unknown /proc/28240/task/81304/cwd (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81304 dashboard ceph rtd unknown /proc/28240/task/81304/root (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81304 dashboard ceph txt unknown /proc/28240/task/81304/exe (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81304 dashboard ceph NOFD /proc/28240/task/81304/fd (opendir: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81351 mgr-fin ceph cwd unknown /proc/28240/task/81351/cwd (readlink: Permission denied) 2024-04-22T13:30:08.554 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81351 mgr-fin ceph rtd unknown /proc/28240/task/81351/root (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81351 mgr-fin ceph txt unknown /proc/28240/task/81351/exe (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81351 mgr-fin ceph NOFD /proc/28240/task/81351/fd (opendir: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81353 mgr-fin ceph cwd unknown /proc/28240/task/81353/cwd (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81353 mgr-fin ceph rtd unknown /proc/28240/task/81353/root (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81353 mgr-fin ceph txt unknown /proc/28240/task/81353/exe (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81353 mgr-fin ceph NOFD /proc/28240/task/81353/fd (opendir: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81354 mgr-fin ceph cwd unknown /proc/28240/task/81354/cwd (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81354 mgr-fin ceph rtd unknown /proc/28240/task/81354/root (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81354 mgr-fin ceph txt unknown /proc/28240/task/81354/exe (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81354 mgr-fin ceph NOFD /proc/28240/task/81354/fd (opendir: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81355 mgr-fin ceph cwd unknown /proc/28240/task/81355/cwd (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81355 mgr-fin ceph rtd unknown /proc/28240/task/81355/root (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81355 mgr-fin ceph txt unknown /proc/28240/task/81355/exe (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81355 mgr-fin ceph NOFD /proc/28240/task/81355/fd (opendir: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81356 mgr-fin ceph cwd unknown /proc/28240/task/81356/cwd (readlink: Permission denied) 2024-04-22T13:30:08.555 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81356 mgr-fin ceph rtd unknown /proc/28240/task/81356/root (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81356 mgr-fin ceph txt unknown /proc/28240/task/81356/exe (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81356 mgr-fin ceph NOFD /proc/28240/task/81356/fd (opendir: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81357 mgr-fin ceph cwd unknown /proc/28240/task/81357/cwd (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81357 mgr-fin ceph rtd unknown /proc/28240/task/81357/root (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81357 mgr-fin ceph txt unknown /proc/28240/task/81357/exe (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81357 mgr-fin ceph NOFD /proc/28240/task/81357/fd (opendir: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81358 mgr-fin ceph cwd unknown /proc/28240/task/81358/cwd (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81358 mgr-fin ceph rtd unknown /proc/28240/task/81358/root (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81358 mgr-fin ceph txt unknown /proc/28240/task/81358/exe (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81358 mgr-fin ceph NOFD /proc/28240/task/81358/fd (opendir: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81359 mgr-fin ceph cwd unknown /proc/28240/task/81359/cwd (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81359 mgr-fin ceph rtd unknown /proc/28240/task/81359/root (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81359 mgr-fin ceph txt unknown /proc/28240/task/81359/exe (readlink: Permission denied) 2024-04-22T13:30:08.556 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81359 mgr-fin ceph NOFD /proc/28240/task/81359/fd (opendir: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81360 mgr-fin ceph cwd unknown /proc/28240/task/81360/cwd (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81360 mgr-fin ceph rtd unknown /proc/28240/task/81360/root (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81360 mgr-fin ceph txt unknown /proc/28240/task/81360/exe (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81360 mgr-fin ceph NOFD /proc/28240/task/81360/fd (opendir: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81361 mgr-fin ceph cwd unknown /proc/28240/task/81361/cwd (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81361 mgr-fin ceph rtd unknown /proc/28240/task/81361/root (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81361 mgr-fin ceph txt unknown /proc/28240/task/81361/exe (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81361 mgr-fin ceph NOFD /proc/28240/task/81361/fd (opendir: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81362 mgr-fin ceph cwd unknown /proc/28240/task/81362/cwd (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81362 mgr-fin ceph rtd unknown /proc/28240/task/81362/root (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81362 mgr-fin ceph txt unknown /proc/28240/task/81362/exe (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81362 mgr-fin ceph NOFD /proc/28240/task/81362/fd (opendir: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81363 dashboard ceph cwd unknown /proc/28240/task/81363/cwd (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81363 dashboard ceph rtd unknown /proc/28240/task/81363/root (readlink: Permission denied) 2024-04-22T13:30:08.557 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81363 dashboard ceph txt unknown /proc/28240/task/81363/exe (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81363 dashboard ceph NOFD /proc/28240/task/81363/fd (opendir: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81365 mgr-fin ceph cwd unknown /proc/28240/task/81365/cwd (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81365 mgr-fin ceph rtd unknown /proc/28240/task/81365/root (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81365 mgr-fin ceph txt unknown /proc/28240/task/81365/exe (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81365 mgr-fin ceph NOFD /proc/28240/task/81365/fd (opendir: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81366 mgr-fin ceph cwd unknown /proc/28240/task/81366/cwd (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81366 mgr-fin ceph rtd unknown /proc/28240/task/81366/root (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81366 mgr-fin ceph txt unknown /proc/28240/task/81366/exe (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81366 mgr-fin ceph NOFD /proc/28240/task/81366/fd (opendir: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81367 mgr-fin ceph cwd unknown /proc/28240/task/81367/cwd (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81367 mgr-fin ceph rtd unknown /proc/28240/task/81367/root (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81367 mgr-fin ceph txt unknown /proc/28240/task/81367/exe (readlink: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81367 mgr-fin ceph NOFD /proc/28240/task/81367/fd (opendir: Permission denied) 2024-04-22T13:30:08.558 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81368 mgr-fin ceph cwd unknown /proc/28240/task/81368/cwd (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81368 mgr-fin ceph rtd unknown /proc/28240/task/81368/root (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81368 mgr-fin ceph txt unknown /proc/28240/task/81368/exe (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81368 mgr-fin ceph NOFD /proc/28240/task/81368/fd (opendir: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81369 mgr-fin ceph cwd unknown /proc/28240/task/81369/cwd (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81369 mgr-fin ceph rtd unknown /proc/28240/task/81369/root (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81369 mgr-fin ceph txt unknown /proc/28240/task/81369/exe (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81369 mgr-fin ceph NOFD /proc/28240/task/81369/fd (opendir: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81370 mgr-fin ceph cwd unknown /proc/28240/task/81370/cwd (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81370 mgr-fin ceph rtd unknown /proc/28240/task/81370/root (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81370 mgr-fin ceph txt unknown /proc/28240/task/81370/exe (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81370 mgr-fin ceph NOFD /proc/28240/task/81370/fd (opendir: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81371 mgr-fin ceph cwd unknown /proc/28240/task/81371/cwd (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81371 mgr-fin ceph rtd unknown /proc/28240/task/81371/root (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81371 mgr-fin ceph txt unknown /proc/28240/task/81371/exe (readlink: Permission denied) 2024-04-22T13:30:08.559 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81371 mgr-fin ceph NOFD /proc/28240/task/81371/fd (opendir: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81372 mgr-fin ceph cwd unknown /proc/28240/task/81372/cwd (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81372 mgr-fin ceph rtd unknown /proc/28240/task/81372/root (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81372 mgr-fin ceph txt unknown /proc/28240/task/81372/exe (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81372 mgr-fin ceph NOFD /proc/28240/task/81372/fd (opendir: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81373 mgr-fin ceph cwd unknown /proc/28240/task/81373/cwd (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81373 mgr-fin ceph rtd unknown /proc/28240/task/81373/root (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81373 mgr-fin ceph txt unknown /proc/28240/task/81373/exe (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81373 mgr-fin ceph NOFD /proc/28240/task/81373/fd (opendir: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81374 mgr-fin ceph cwd unknown /proc/28240/task/81374/cwd (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81374 mgr-fin ceph rtd unknown /proc/28240/task/81374/root (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81374 mgr-fin ceph txt unknown /proc/28240/task/81374/exe (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81374 mgr-fin ceph NOFD /proc/28240/task/81374/fd (opendir: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81375 mgr-fin ceph cwd unknown /proc/28240/task/81375/cwd (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81375 mgr-fin ceph rtd unknown /proc/28240/task/81375/root (readlink: Permission denied) 2024-04-22T13:30:08.560 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81375 mgr-fin ceph txt unknown /proc/28240/task/81375/exe (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81375 mgr-fin ceph NOFD /proc/28240/task/81375/fd (opendir: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81754 devicehea ceph cwd unknown /proc/28240/task/81754/cwd (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81754 devicehea ceph rtd unknown /proc/28240/task/81754/root (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81754 devicehea ceph txt unknown /proc/28240/task/81754/exe (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81754 devicehea ceph NOFD /proc/28240/task/81754/fd (opendir: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81767 io_contex ceph cwd unknown /proc/28240/task/81767/cwd (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81767 io_contex ceph rtd unknown /proc/28240/task/81767/root (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81767 io_contex ceph txt unknown /proc/28240/task/81767/exe (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81767 io_contex ceph NOFD /proc/28240/task/81767/fd (opendir: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81768 io_contex ceph cwd unknown /proc/28240/task/81768/cwd (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81768 io_contex ceph rtd unknown /proc/28240/task/81768/root (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81768 io_contex ceph txt unknown /proc/28240/task/81768/exe (readlink: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81768 io_contex ceph NOFD /proc/28240/task/81768/fd (opendir: Permission denied) 2024-04-22T13:30:08.561 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81769 ceph_time ceph cwd unknown /proc/28240/task/81769/cwd (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81769 ceph_time ceph rtd unknown /proc/28240/task/81769/root (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81769 ceph_time ceph txt unknown /proc/28240/task/81769/exe (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81769 ceph_time ceph NOFD /proc/28240/task/81769/fd (opendir: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81770 ms_dispat ceph cwd unknown /proc/28240/task/81770/cwd (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81770 ms_dispat ceph rtd unknown /proc/28240/task/81770/root (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81770 ms_dispat ceph txt unknown /proc/28240/task/81770/exe (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81770 ms_dispat ceph NOFD /proc/28240/task/81770/fd (opendir: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81771 ms_local ceph cwd unknown /proc/28240/task/81771/cwd (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81771 ms_local ceph rtd unknown /proc/28240/task/81771/root (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81771 ms_local ceph txt unknown /proc/28240/task/81771/exe (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81771 ms_local ceph NOFD /proc/28240/task/81771/fd (opendir: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81772 safe_time ceph cwd unknown /proc/28240/task/81772/cwd (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81772 safe_time ceph rtd unknown /proc/28240/task/81772/root (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81772 safe_time ceph txt unknown /proc/28240/task/81772/exe (readlink: Permission denied) 2024-04-22T13:30:08.562 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81772 safe_time ceph NOFD /proc/28240/task/81772/fd (opendir: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81773 safe_time ceph cwd unknown /proc/28240/task/81773/cwd (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81773 safe_time ceph rtd unknown /proc/28240/task/81773/root (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81773 safe_time ceph txt unknown /proc/28240/task/81773/exe (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 81773 safe_time ceph NOFD /proc/28240/task/81773/fd (opendir: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85910 ceph_time ceph cwd unknown /proc/28240/task/85910/cwd (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85910 ceph_time ceph rtd unknown /proc/28240/task/85910/root (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85910 ceph_time ceph txt unknown /proc/28240/task/85910/exe (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85910 ceph_time ceph NOFD /proc/28240/task/85910/fd (opendir: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85911 fn_anonym ceph cwd unknown /proc/28240/task/85911/cwd (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85911 fn_anonym ceph rtd unknown /proc/28240/task/85911/root (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85911 fn_anonym ceph txt unknown /proc/28240/task/85911/exe (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85911 fn_anonym ceph NOFD /proc/28240/task/85911/fd (opendir: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85912 safe_time ceph cwd unknown /proc/28240/task/85912/cwd (readlink: Permission denied) 2024-04-22T13:30:08.563 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85912 safe_time ceph rtd unknown /proc/28240/task/85912/root (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85912 safe_time ceph txt unknown /proc/28240/task/85912/exe (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85912 safe_time ceph NOFD /proc/28240/task/85912/fd (opendir: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85913 fn_anonym ceph cwd unknown /proc/28240/task/85913/cwd (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85913 fn_anonym ceph rtd unknown /proc/28240/task/85913/root (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85913 fn_anonym ceph txt unknown /proc/28240/task/85913/exe (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85913 fn_anonym ceph NOFD /proc/28240/task/85913/fd (opendir: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85914 flusher ceph cwd unknown /proc/28240/task/85914/cwd (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85914 flusher ceph rtd unknown /proc/28240/task/85914/root (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85914 flusher ceph txt unknown /proc/28240/task/85914/exe (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85914 flusher ceph NOFD /proc/28240/task/85914/fd (opendir: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85915 ms_dispat ceph cwd unknown /proc/28240/task/85915/cwd (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85915 ms_dispat ceph rtd unknown /proc/28240/task/85915/root (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85915 ms_dispat ceph txt unknown /proc/28240/task/85915/exe (readlink: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85915 ms_dispat ceph NOFD /proc/28240/task/85915/fd (opendir: Permission denied) 2024-04-22T13:30:08.564 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85916 ms_local ceph cwd unknown /proc/28240/task/85916/cwd (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85916 ms_local ceph rtd unknown /proc/28240/task/85916/root (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85916 ms_local ceph txt unknown /proc/28240/task/85916/exe (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85916 ms_local ceph NOFD /proc/28240/task/85916/fd (opendir: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85917 safe_time ceph cwd unknown /proc/28240/task/85917/cwd (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85917 safe_time ceph rtd unknown /proc/28240/task/85917/root (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85917 safe_time ceph txt unknown /proc/28240/task/85917/exe (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85917 safe_time ceph NOFD /proc/28240/task/85917/fd (opendir: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85918 m-fin-sna ceph cwd unknown /proc/28240/task/85918/cwd (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85918 m-fin-sna ceph rtd unknown /proc/28240/task/85918/root (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85918 m-fin-sna ceph txt unknown /proc/28240/task/85918/exe (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 85918 m-fin-sna ceph NOFD /proc/28240/task/85918/fd (opendir: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 89717 m-fin-sna ceph cwd unknown /proc/28240/task/89717/cwd (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 89717 m-fin-sna ceph rtd unknown /proc/28240/task/89717/root (readlink: Permission denied) 2024-04-22T13:30:08.565 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 89717 m-fin-sna ceph txt unknown /proc/28240/task/89717/exe (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mgr 28240 89717 m-fin-sna ceph NOFD /proc/28240/task/89717/fd (opendir: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 28365 root cwd unknown /proc/28365/cwd (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 28365 root rtd unknown /proc/28365/root (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 28365 root txt unknown /proc/28365/exe (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 28365 root NOFD /proc/28365/fd (opendir: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 29389 root cwd unknown /proc/29389/cwd (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 29389 root rtd unknown /proc/29389/root (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 29389 root txt unknown /proc/29389/exe (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:systemd 29389 root NOFD /proc/29389/fd (opendir: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 29391 root cwd unknown /proc/29391/cwd (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 29391 root rtd unknown /proc/29391/root (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 29391 root txt unknown /proc/29391/exe (readlink: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:(sd-pam) 29391 root NOFD /proc/29391/fd (opendir: Permission denied) 2024-04-22T13:30:08.566 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 30973 root cwd unknown /proc/30973/cwd (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 30973 root rtd unknown /proc/30973/root (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 30973 root txt unknown /proc/30973/exe (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 30973 root NOFD /proc/30973/fd (opendir: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 31757 root cwd unknown /proc/31757/cwd (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 31757 root rtd unknown /proc/31757/root (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 31757 root txt unknown /proc/31757/exe (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 31757 root NOFD /proc/31757/fd (opendir: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 39660 root cwd unknown /proc/39660/cwd (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 39660 root rtd unknown /proc/39660/root (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 39660 root txt unknown /proc/39660/exe (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/0 39660 root NOFD /proc/39660/fd (opendir: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 42998 root cwd unknown /proc/42998/cwd (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 42998 root rtd unknown /proc/42998/root (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 42998 root txt unknown /proc/42998/exe (readlink: Permission denied) 2024-04-22T13:30:08.567 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 42998 root NOFD /proc/42998/fd (opendir: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 43000 root cwd unknown /proc/43000/cwd (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 43000 root rtd unknown /proc/43000/root (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 43000 root txt unknown /proc/43000/exe (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 43000 root NOFD /proc/43000/fd (opendir: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 ceph cwd unknown /proc/43002/cwd (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 ceph rtd unknown /proc/43002/root (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 ceph txt unknown /proc/43002/exe (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 ceph NOFD /proc/43002/fd (opendir: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43003 log ceph cwd unknown /proc/43002/task/43003/cwd (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43003 log ceph rtd unknown /proc/43002/task/43003/root (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43003 log ceph txt unknown /proc/43002/task/43003/exe (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43003 log ceph NOFD /proc/43002/task/43003/fd (opendir: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43005 msgr-work ceph cwd unknown /proc/43002/task/43005/cwd (readlink: Permission denied) 2024-04-22T13:30:08.568 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43005 msgr-work ceph rtd unknown /proc/43002/task/43005/root (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43005 msgr-work ceph txt unknown /proc/43002/task/43005/exe (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43005 msgr-work ceph NOFD /proc/43002/task/43005/fd (opendir: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43006 msgr-work ceph cwd unknown /proc/43002/task/43006/cwd (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43006 msgr-work ceph rtd unknown /proc/43002/task/43006/root (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43006 msgr-work ceph txt unknown /proc/43002/task/43006/exe (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43006 msgr-work ceph NOFD /proc/43002/task/43006/fd (opendir: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43007 msgr-work ceph cwd unknown /proc/43002/task/43007/cwd (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43007 msgr-work ceph rtd unknown /proc/43002/task/43007/root (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43007 msgr-work ceph txt unknown /proc/43002/task/43007/exe (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43007 msgr-work ceph NOFD /proc/43002/task/43007/fd (opendir: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43011 service ceph cwd unknown /proc/43002/task/43011/cwd (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43011 service ceph rtd unknown /proc/43002/task/43011/root (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43011 service ceph txt unknown /proc/43002/task/43011/exe (readlink: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43011 service ceph NOFD /proc/43002/task/43011/fd (opendir: Permission denied) 2024-04-22T13:30:08.569 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43012 admin_soc ceph cwd unknown /proc/43002/task/43012/cwd (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43012 admin_soc ceph rtd unknown /proc/43002/task/43012/root (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43012 admin_soc ceph txt unknown /proc/43002/task/43012/exe (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43012 admin_soc ceph NOFD /proc/43002/task/43012/fd (opendir: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43156 signal_ha ceph cwd unknown /proc/43002/task/43156/cwd (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43156 signal_ha ceph rtd unknown /proc/43002/task/43156/root (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43156 signal_ha ceph txt unknown /proc/43002/task/43156/exe (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43156 signal_ha ceph NOFD /proc/43002/task/43156/fd (opendir: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43157 io_contex ceph cwd unknown /proc/43002/task/43157/cwd (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43157 io_contex ceph rtd unknown /proc/43002/task/43157/root (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43157 io_contex ceph txt unknown /proc/43002/task/43157/exe (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43157 io_contex ceph NOFD /proc/43002/task/43157/fd (opendir: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43158 io_contex ceph cwd unknown /proc/43002/task/43158/cwd (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43158 io_contex ceph rtd unknown /proc/43002/task/43158/root (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43158 io_contex ceph txt unknown /proc/43002/task/43158/exe (readlink: Permission denied) 2024-04-22T13:30:08.570 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43158 io_contex ceph NOFD /proc/43002/task/43158/fd (opendir: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43273 OpHistory ceph cwd unknown /proc/43002/task/43273/cwd (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43273 OpHistory ceph rtd unknown /proc/43002/task/43273/root (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43273 OpHistory ceph txt unknown /proc/43002/task/43273/exe (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43273 OpHistory ceph NOFD /proc/43002/task/43273/fd (opendir: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43274 ceph_time ceph cwd unknown /proc/43002/task/43274/cwd (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43274 ceph_time ceph rtd unknown /proc/43002/task/43274/root (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43274 ceph_time ceph txt unknown /proc/43002/task/43274/exe (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43274 ceph_time ceph NOFD /proc/43002/task/43274/fd (opendir: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43472 ceph-osd ceph cwd unknown /proc/43002/task/43472/cwd (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43472 ceph-osd ceph rtd unknown /proc/43002/task/43472/root (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43472 ceph-osd ceph txt unknown /proc/43002/task/43472/exe (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43472 ceph-osd ceph NOFD /proc/43002/task/43472/fd (opendir: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43473 safe_time ceph cwd unknown /proc/43002/task/43473/cwd (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43473 safe_time ceph rtd unknown /proc/43002/task/43473/root (readlink: Permission denied) 2024-04-22T13:30:08.571 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43473 safe_time ceph txt unknown /proc/43002/task/43473/exe (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43473 safe_time ceph NOFD /proc/43002/task/43473/fd (opendir: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43474 safe_time ceph cwd unknown /proc/43002/task/43474/cwd (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43474 safe_time ceph rtd unknown /proc/43002/task/43474/root (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43474 safe_time ceph txt unknown /proc/43002/task/43474/exe (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43474 safe_time ceph NOFD /proc/43002/task/43474/fd (opendir: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43475 safe_time ceph cwd unknown /proc/43002/task/43475/cwd (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43475 safe_time ceph rtd unknown /proc/43002/task/43475/root (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43475 safe_time ceph txt unknown /proc/43002/task/43475/exe (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43475 safe_time ceph NOFD /proc/43002/task/43475/fd (opendir: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43476 safe_time ceph cwd unknown /proc/43002/task/43476/cwd (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43476 safe_time ceph rtd unknown /proc/43002/task/43476/root (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43476 safe_time ceph txt unknown /proc/43002/task/43476/exe (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43476 safe_time ceph NOFD /proc/43002/task/43476/fd (opendir: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43477 fn_anonym ceph cwd unknown /proc/43002/task/43477/cwd (readlink: Permission denied) 2024-04-22T13:30:08.572 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43477 fn_anonym ceph rtd unknown /proc/43002/task/43477/root (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43477 fn_anonym ceph txt unknown /proc/43002/task/43477/exe (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43477 fn_anonym ceph NOFD /proc/43002/task/43477/fd (opendir: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43485 rocksdb:l ceph cwd unknown /proc/43002/task/43485/cwd (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43485 rocksdb:l ceph rtd unknown /proc/43002/task/43485/root (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43485 rocksdb:l ceph txt unknown /proc/43002/task/43485/exe (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43485 rocksdb:l ceph NOFD /proc/43002/task/43485/fd (opendir: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43486 rocksdb:l ceph cwd unknown /proc/43002/task/43486/cwd (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43486 rocksdb:l ceph rtd unknown /proc/43002/task/43486/root (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43486 rocksdb:l ceph txt unknown /proc/43002/task/43486/exe (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43486 rocksdb:l ceph NOFD /proc/43002/task/43486/fd (opendir: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43487 rocksdb:l ceph cwd unknown /proc/43002/task/43487/cwd (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43487 rocksdb:l ceph rtd unknown /proc/43002/task/43487/root (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43487 rocksdb:l ceph txt unknown /proc/43002/task/43487/exe (readlink: Permission denied) 2024-04-22T13:30:08.573 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43487 rocksdb:l ceph NOFD /proc/43002/task/43487/fd (opendir: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43489 rocksdb:h ceph cwd unknown /proc/43002/task/43489/cwd (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43489 rocksdb:h ceph rtd unknown /proc/43002/task/43489/root (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43489 rocksdb:h ceph txt unknown /proc/43002/task/43489/exe (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43489 rocksdb:h ceph NOFD /proc/43002/task/43489/fd (opendir: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43976 bstore_ai ceph cwd unknown /proc/43002/task/43976/cwd (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43976 bstore_ai ceph rtd unknown /proc/43002/task/43976/root (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43976 bstore_ai ceph txt unknown /proc/43002/task/43976/exe (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 43976 bstore_ai ceph NOFD /proc/43002/task/43976/fd (opendir: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44169 bstore_ai ceph cwd unknown /proc/43002/task/44169/cwd (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44169 bstore_ai ceph rtd unknown /proc/43002/task/44169/root (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44169 bstore_ai ceph txt unknown /proc/43002/task/44169/exe (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44169 bstore_ai ceph NOFD /proc/43002/task/44169/fd (opendir: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44352 ceph-osd ceph cwd unknown /proc/43002/task/44352/cwd (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44352 ceph-osd ceph rtd unknown /proc/43002/task/44352/root (readlink: Permission denied) 2024-04-22T13:30:08.574 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44352 ceph-osd ceph txt unknown /proc/43002/task/44352/exe (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44352 ceph-osd ceph NOFD /proc/43002/task/44352/fd (opendir: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44353 cfin ceph cwd unknown /proc/43002/task/44353/cwd (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44353 cfin ceph rtd unknown /proc/43002/task/44353/root (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44353 cfin ceph txt unknown /proc/43002/task/44353/exe (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44353 cfin ceph NOFD /proc/43002/task/44353/fd (opendir: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44354 bstore_kv ceph cwd unknown /proc/43002/task/44354/cwd (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44354 bstore_kv ceph rtd unknown /proc/43002/task/44354/root (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44354 bstore_kv ceph txt unknown /proc/43002/task/44354/exe (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44354 bstore_kv ceph NOFD /proc/43002/task/44354/fd (opendir: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44355 bstore_kv ceph cwd unknown /proc/43002/task/44355/cwd (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44355 bstore_kv ceph rtd unknown /proc/43002/task/44355/root (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44355 bstore_kv ceph txt unknown /proc/43002/task/44355/exe (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44355 bstore_kv ceph NOFD /proc/43002/task/44355/fd (opendir: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44356 bstore_me ceph cwd unknown /proc/43002/task/44356/cwd (readlink: Permission denied) 2024-04-22T13:30:08.575 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44356 bstore_me ceph rtd unknown /proc/43002/task/44356/root (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44356 bstore_me ceph txt unknown /proc/43002/task/44356/exe (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44356 bstore_me ceph NOFD /proc/43002/task/44356/fd (opendir: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44357 ms_dispat ceph cwd unknown /proc/43002/task/44357/cwd (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44357 ms_dispat ceph rtd unknown /proc/43002/task/44357/root (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44357 ms_dispat ceph txt unknown /proc/43002/task/44357/exe (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44357 ms_dispat ceph NOFD /proc/43002/task/44357/fd (opendir: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44358 ms_local ceph cwd unknown /proc/43002/task/44358/cwd (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44358 ms_local ceph rtd unknown /proc/43002/task/44358/root (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44358 ms_local ceph txt unknown /proc/43002/task/44358/exe (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44358 ms_local ceph NOFD /proc/43002/task/44358/fd (opendir: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44359 safe_time ceph cwd unknown /proc/43002/task/44359/cwd (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44359 safe_time ceph rtd unknown /proc/43002/task/44359/root (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44359 safe_time ceph txt unknown /proc/43002/task/44359/exe (readlink: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44359 safe_time ceph NOFD /proc/43002/task/44359/fd (opendir: Permission denied) 2024-04-22T13:30:08.576 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44360 safe_time ceph cwd unknown /proc/43002/task/44360/cwd (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44360 safe_time ceph rtd unknown /proc/43002/task/44360/root (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44360 safe_time ceph txt unknown /proc/43002/task/44360/exe (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44360 safe_time ceph NOFD /proc/43002/task/44360/fd (opendir: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44361 ms_dispat ceph cwd unknown /proc/43002/task/44361/cwd (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44361 ms_dispat ceph rtd unknown /proc/43002/task/44361/root (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44361 ms_dispat ceph txt unknown /proc/43002/task/44361/exe (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44361 ms_dispat ceph NOFD /proc/43002/task/44361/fd (opendir: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44362 ms_local ceph cwd unknown /proc/43002/task/44362/cwd (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44362 ms_local ceph rtd unknown /proc/43002/task/44362/root (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44362 ms_local ceph txt unknown /proc/43002/task/44362/exe (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44362 ms_local ceph NOFD /proc/43002/task/44362/fd (opendir: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44363 ms_dispat ceph cwd unknown /proc/43002/task/44363/cwd (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44363 ms_dispat ceph rtd unknown /proc/43002/task/44363/root (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44363 ms_dispat ceph txt unknown /proc/43002/task/44363/exe (readlink: Permission denied) 2024-04-22T13:30:08.577 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44363 ms_dispat ceph NOFD /proc/43002/task/44363/fd (opendir: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44364 ms_local ceph cwd unknown /proc/43002/task/44364/cwd (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44364 ms_local ceph rtd unknown /proc/43002/task/44364/root (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44364 ms_local ceph txt unknown /proc/43002/task/44364/exe (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44364 ms_local ceph NOFD /proc/43002/task/44364/fd (opendir: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44365 ms_dispat ceph cwd unknown /proc/43002/task/44365/cwd (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44365 ms_dispat ceph rtd unknown /proc/43002/task/44365/root (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44365 ms_dispat ceph txt unknown /proc/43002/task/44365/exe (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44365 ms_dispat ceph NOFD /proc/43002/task/44365/fd (opendir: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44366 ms_local ceph cwd unknown /proc/43002/task/44366/cwd (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44366 ms_local ceph rtd unknown /proc/43002/task/44366/root (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44366 ms_local ceph txt unknown /proc/43002/task/44366/exe (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44366 ms_local ceph NOFD /proc/43002/task/44366/fd (opendir: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44367 ms_dispat ceph cwd unknown /proc/43002/task/44367/cwd (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44367 ms_dispat ceph rtd unknown /proc/43002/task/44367/root (readlink: Permission denied) 2024-04-22T13:30:08.578 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44367 ms_dispat ceph txt unknown /proc/43002/task/44367/exe (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44367 ms_dispat ceph NOFD /proc/43002/task/44367/fd (opendir: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44368 ms_local ceph cwd unknown /proc/43002/task/44368/cwd (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44368 ms_local ceph rtd unknown /proc/43002/task/44368/root (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44368 ms_local ceph txt unknown /proc/43002/task/44368/exe (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44368 ms_local ceph NOFD /proc/43002/task/44368/fd (opendir: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44369 ms_dispat ceph cwd unknown /proc/43002/task/44369/cwd (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44369 ms_dispat ceph rtd unknown /proc/43002/task/44369/root (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44369 ms_dispat ceph txt unknown /proc/43002/task/44369/exe (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44369 ms_dispat ceph NOFD /proc/43002/task/44369/fd (opendir: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44370 ms_local ceph cwd unknown /proc/43002/task/44370/cwd (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44370 ms_local ceph rtd unknown /proc/43002/task/44370/root (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44370 ms_local ceph txt unknown /proc/43002/task/44370/exe (readlink: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44370 ms_local ceph NOFD /proc/43002/task/44370/fd (opendir: Permission denied) 2024-04-22T13:30:08.579 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44371 ms_dispat ceph cwd unknown /proc/43002/task/44371/cwd (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44371 ms_dispat ceph rtd unknown /proc/43002/task/44371/root (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44371 ms_dispat ceph txt unknown /proc/43002/task/44371/exe (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44371 ms_dispat ceph NOFD /proc/43002/task/44371/fd (opendir: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44372 ms_local ceph cwd unknown /proc/43002/task/44372/cwd (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44372 ms_local ceph rtd unknown /proc/43002/task/44372/root (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44372 ms_local ceph txt unknown /proc/43002/task/44372/exe (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44372 ms_local ceph NOFD /proc/43002/task/44372/fd (opendir: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44373 fn_anonym ceph cwd unknown /proc/43002/task/44373/cwd (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44373 fn_anonym ceph rtd unknown /proc/43002/task/44373/root (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44373 fn_anonym ceph txt unknown /proc/43002/task/44373/exe (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44373 fn_anonym ceph NOFD /proc/43002/task/44373/fd (opendir: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44374 finisher ceph cwd unknown /proc/43002/task/44374/cwd (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44374 finisher ceph rtd unknown /proc/43002/task/44374/root (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44374 finisher ceph txt unknown /proc/43002/task/44374/exe (readlink: Permission denied) 2024-04-22T13:30:08.580 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44374 finisher ceph NOFD /proc/43002/task/44374/fd (opendir: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44375 safe_time ceph cwd unknown /proc/43002/task/44375/cwd (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44375 safe_time ceph rtd unknown /proc/43002/task/44375/root (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44375 safe_time ceph txt unknown /proc/43002/task/44375/exe (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44375 safe_time ceph NOFD /proc/43002/task/44375/fd (opendir: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44376 safe_time ceph cwd unknown /proc/43002/task/44376/cwd (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44376 safe_time ceph rtd unknown /proc/43002/task/44376/root (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44376 safe_time ceph txt unknown /proc/43002/task/44376/exe (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44376 safe_time ceph NOFD /proc/43002/task/44376/fd (opendir: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44377 ceph-osd ceph cwd unknown /proc/43002/task/44377/cwd (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44377 ceph-osd ceph rtd unknown /proc/43002/task/44377/root (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44377 ceph-osd ceph txt unknown /proc/43002/task/44377/exe (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44377 ceph-osd ceph NOFD /proc/43002/task/44377/fd (opendir: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44378 osd_srv_a ceph cwd unknown /proc/43002/task/44378/cwd (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44378 osd_srv_a ceph rtd unknown /proc/43002/task/44378/root (readlink: Permission denied) 2024-04-22T13:30:08.581 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44378 osd_srv_a ceph txt unknown /proc/43002/task/44378/exe (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44378 osd_srv_a ceph NOFD /proc/43002/task/44378/fd (opendir: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44379 tp_osd_tp ceph cwd unknown /proc/43002/task/44379/cwd (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44379 tp_osd_tp ceph rtd unknown /proc/43002/task/44379/root (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44379 tp_osd_tp ceph txt unknown /proc/43002/task/44379/exe (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44379 tp_osd_tp ceph NOFD /proc/43002/task/44379/fd (opendir: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44380 tp_osd_tp ceph cwd unknown /proc/43002/task/44380/cwd (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44380 tp_osd_tp ceph rtd unknown /proc/43002/task/44380/root (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44380 tp_osd_tp ceph txt unknown /proc/43002/task/44380/exe (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44380 tp_osd_tp ceph NOFD /proc/43002/task/44380/fd (opendir: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44381 tp_osd_tp ceph cwd unknown /proc/43002/task/44381/cwd (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44381 tp_osd_tp ceph rtd unknown /proc/43002/task/44381/root (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44381 tp_osd_tp ceph txt unknown /proc/43002/task/44381/exe (readlink: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44381 tp_osd_tp ceph NOFD /proc/43002/task/44381/fd (opendir: Permission denied) 2024-04-22T13:30:08.582 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44382 tp_osd_tp ceph cwd unknown /proc/43002/task/44382/cwd (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44382 tp_osd_tp ceph rtd unknown /proc/43002/task/44382/root (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44382 tp_osd_tp ceph txt unknown /proc/43002/task/44382/exe (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44382 tp_osd_tp ceph NOFD /proc/43002/task/44382/fd (opendir: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44383 tp_osd_tp ceph cwd unknown /proc/43002/task/44383/cwd (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44383 tp_osd_tp ceph rtd unknown /proc/43002/task/44383/root (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44383 tp_osd_tp ceph txt unknown /proc/43002/task/44383/exe (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44383 tp_osd_tp ceph NOFD /proc/43002/task/44383/fd (opendir: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44384 tp_osd_tp ceph cwd unknown /proc/43002/task/44384/cwd (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44384 tp_osd_tp ceph rtd unknown /proc/43002/task/44384/root (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44384 tp_osd_tp ceph txt unknown /proc/43002/task/44384/exe (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44384 tp_osd_tp ceph NOFD /proc/43002/task/44384/fd (opendir: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44385 tp_osd_tp ceph cwd unknown /proc/43002/task/44385/cwd (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44385 tp_osd_tp ceph rtd unknown /proc/43002/task/44385/root (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44385 tp_osd_tp ceph txt unknown /proc/43002/task/44385/exe (readlink: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44385 tp_osd_tp ceph NOFD /proc/43002/task/44385/fd (opendir: Permission denied) 2024-04-22T13:30:08.583 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44386 tp_osd_tp ceph cwd unknown /proc/43002/task/44386/cwd (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44386 tp_osd_tp ceph rtd unknown /proc/43002/task/44386/root (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44386 tp_osd_tp ceph txt unknown /proc/43002/task/44386/exe (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44386 tp_osd_tp ceph NOFD /proc/43002/task/44386/fd (opendir: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44387 tp_osd_tp ceph cwd unknown /proc/43002/task/44387/cwd (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44387 tp_osd_tp ceph rtd unknown /proc/43002/task/44387/root (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44387 tp_osd_tp ceph txt unknown /proc/43002/task/44387/exe (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44387 tp_osd_tp ceph NOFD /proc/43002/task/44387/fd (opendir: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44388 tp_osd_tp ceph cwd unknown /proc/43002/task/44388/cwd (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44388 tp_osd_tp ceph rtd unknown /proc/43002/task/44388/root (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44388 tp_osd_tp ceph txt unknown /proc/43002/task/44388/exe (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44388 tp_osd_tp ceph NOFD /proc/43002/task/44388/fd (opendir: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44389 tp_osd_tp ceph cwd unknown /proc/43002/task/44389/cwd (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44389 tp_osd_tp ceph rtd unknown /proc/43002/task/44389/root (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44389 tp_osd_tp ceph txt unknown /proc/43002/task/44389/exe (readlink: Permission denied) 2024-04-22T13:30:08.584 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44389 tp_osd_tp ceph NOFD /proc/43002/task/44389/fd (opendir: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44390 tp_osd_tp ceph cwd unknown /proc/43002/task/44390/cwd (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44390 tp_osd_tp ceph rtd unknown /proc/43002/task/44390/root (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44390 tp_osd_tp ceph txt unknown /proc/43002/task/44390/exe (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44390 tp_osd_tp ceph NOFD /proc/43002/task/44390/fd (opendir: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44391 tp_osd_tp ceph cwd unknown /proc/43002/task/44391/cwd (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44391 tp_osd_tp ceph rtd unknown /proc/43002/task/44391/root (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44391 tp_osd_tp ceph txt unknown /proc/43002/task/44391/exe (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44391 tp_osd_tp ceph NOFD /proc/43002/task/44391/fd (opendir: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44392 tp_osd_tp ceph cwd unknown /proc/43002/task/44392/cwd (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44392 tp_osd_tp ceph rtd unknown /proc/43002/task/44392/root (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44392 tp_osd_tp ceph txt unknown /proc/43002/task/44392/exe (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44392 tp_osd_tp ceph NOFD /proc/43002/task/44392/fd (opendir: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44393 tp_osd_tp ceph cwd unknown /proc/43002/task/44393/cwd (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44393 tp_osd_tp ceph rtd unknown /proc/43002/task/44393/root (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44393 tp_osd_tp ceph txt unknown /proc/43002/task/44393/exe (readlink: Permission denied) 2024-04-22T13:30:08.585 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44393 tp_osd_tp ceph NOFD /proc/43002/task/44393/fd (opendir: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44394 tp_osd_tp ceph cwd unknown /proc/43002/task/44394/cwd (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44394 tp_osd_tp ceph rtd unknown /proc/43002/task/44394/root (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44394 tp_osd_tp ceph txt unknown /proc/43002/task/44394/exe (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44394 tp_osd_tp ceph NOFD /proc/43002/task/44394/fd (opendir: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44395 osd_srv_h ceph cwd unknown /proc/43002/task/44395/cwd (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44395 osd_srv_h ceph rtd unknown /proc/43002/task/44395/root (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44395 osd_srv_h ceph txt unknown /proc/43002/task/44395/exe (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 43002 44395 osd_srv_h ceph NOFD /proc/43002/task/44395/fd (opendir: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 44407 root cwd unknown /proc/44407/cwd (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 44407 root rtd unknown /proc/44407/root (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 44407 root txt unknown /proc/44407/exe (readlink: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 44407 root NOFD /proc/44407/fd (opendir: Permission denied) 2024-04-22T13:30:08.586 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 44428 root cwd unknown /proc/44428/cwd (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 44428 root rtd unknown /proc/44428/root (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 44428 root txt unknown /proc/44428/exe (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 44428 root NOFD /proc/44428/fd (opendir: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 46616 root cwd unknown /proc/46616/cwd (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 46616 root rtd unknown /proc/46616/root (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 46616 root txt unknown /proc/46616/exe (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 46616 root NOFD /proc/46616/fd (opendir: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 49507 root cwd unknown /proc/49507/cwd (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 49507 root rtd unknown /proc/49507/root (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 49507 root txt unknown /proc/49507/exe (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 49507 root NOFD /proc/49507/fd (opendir: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 49509 root cwd unknown /proc/49509/cwd (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 49509 root rtd unknown /proc/49509/root (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 49509 root txt unknown /proc/49509/exe (readlink: Permission denied) 2024-04-22T13:30:08.587 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 49509 root NOFD /proc/49509/fd (opendir: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 ceph cwd unknown /proc/49511/cwd (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 ceph rtd unknown /proc/49511/root (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 ceph txt unknown /proc/49511/exe (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 ceph NOFD /proc/49511/fd (opendir: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49512 log ceph cwd unknown /proc/49511/task/49512/cwd (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49512 log ceph rtd unknown /proc/49511/task/49512/root (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49512 log ceph txt unknown /proc/49511/task/49512/exe (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49512 log ceph NOFD /proc/49511/task/49512/fd (opendir: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49514 msgr-work ceph cwd unknown /proc/49511/task/49514/cwd (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49514 msgr-work ceph rtd unknown /proc/49511/task/49514/root (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49514 msgr-work ceph txt unknown /proc/49511/task/49514/exe (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49514 msgr-work ceph NOFD /proc/49511/task/49514/fd (opendir: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49515 msgr-work ceph cwd unknown /proc/49511/task/49515/cwd (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49515 msgr-work ceph rtd unknown /proc/49511/task/49515/root (readlink: Permission denied) 2024-04-22T13:30:08.588 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49515 msgr-work ceph txt unknown /proc/49511/task/49515/exe (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49515 msgr-work ceph NOFD /proc/49511/task/49515/fd (opendir: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49516 msgr-work ceph cwd unknown /proc/49511/task/49516/cwd (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49516 msgr-work ceph rtd unknown /proc/49511/task/49516/root (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49516 msgr-work ceph txt unknown /proc/49511/task/49516/exe (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49516 msgr-work ceph NOFD /proc/49511/task/49516/fd (opendir: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49520 service ceph cwd unknown /proc/49511/task/49520/cwd (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49520 service ceph rtd unknown /proc/49511/task/49520/root (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49520 service ceph txt unknown /proc/49511/task/49520/exe (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49520 service ceph NOFD /proc/49511/task/49520/fd (opendir: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49521 admin_soc ceph cwd unknown /proc/49511/task/49521/cwd (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49521 admin_soc ceph rtd unknown /proc/49511/task/49521/root (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49521 admin_soc ceph txt unknown /proc/49511/task/49521/exe (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49521 admin_soc ceph NOFD /proc/49511/task/49521/fd (opendir: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49663 signal_ha ceph cwd unknown /proc/49511/task/49663/cwd (readlink: Permission denied) 2024-04-22T13:30:08.589 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49663 signal_ha ceph rtd unknown /proc/49511/task/49663/root (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49663 signal_ha ceph txt unknown /proc/49511/task/49663/exe (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49663 signal_ha ceph NOFD /proc/49511/task/49663/fd (opendir: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49664 io_contex ceph cwd unknown /proc/49511/task/49664/cwd (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49664 io_contex ceph rtd unknown /proc/49511/task/49664/root (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49664 io_contex ceph txt unknown /proc/49511/task/49664/exe (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49664 io_contex ceph NOFD /proc/49511/task/49664/fd (opendir: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49665 io_contex ceph cwd unknown /proc/49511/task/49665/cwd (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49665 io_contex ceph rtd unknown /proc/49511/task/49665/root (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49665 io_contex ceph txt unknown /proc/49511/task/49665/exe (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49665 io_contex ceph NOFD /proc/49511/task/49665/fd (opendir: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49783 OpHistory ceph cwd unknown /proc/49511/task/49783/cwd (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49783 OpHistory ceph rtd unknown /proc/49511/task/49783/root (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49783 OpHistory ceph txt unknown /proc/49511/task/49783/exe (readlink: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49783 OpHistory ceph NOFD /proc/49511/task/49783/fd (opendir: Permission denied) 2024-04-22T13:30:08.590 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49784 ceph_time ceph cwd unknown /proc/49511/task/49784/cwd (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49784 ceph_time ceph rtd unknown /proc/49511/task/49784/root (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49784 ceph_time ceph txt unknown /proc/49511/task/49784/exe (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49784 ceph_time ceph NOFD /proc/49511/task/49784/fd (opendir: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49807 ceph-osd ceph cwd unknown /proc/49511/task/49807/cwd (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49807 ceph-osd ceph rtd unknown /proc/49511/task/49807/root (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49807 ceph-osd ceph txt unknown /proc/49511/task/49807/exe (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49807 ceph-osd ceph NOFD /proc/49511/task/49807/fd (opendir: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49811 ceph-osd ceph cwd unknown /proc/49511/task/49811/cwd (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49811 ceph-osd ceph rtd unknown /proc/49511/task/49811/root (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49811 ceph-osd ceph txt unknown /proc/49511/task/49811/exe (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49811 ceph-osd ceph NOFD /proc/49511/task/49811/fd (opendir: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49832 ceph-osd ceph cwd unknown /proc/49511/task/49832/cwd (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49832 ceph-osd ceph rtd unknown /proc/49511/task/49832/root (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49832 ceph-osd ceph txt unknown /proc/49511/task/49832/exe (readlink: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49832 ceph-osd ceph NOFD /proc/49511/task/49832/fd (opendir: Permission denied) 2024-04-22T13:30:08.591 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49869 ceph-osd ceph cwd unknown /proc/49511/task/49869/cwd (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49869 ceph-osd ceph rtd unknown /proc/49511/task/49869/root (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49869 ceph-osd ceph txt unknown /proc/49511/task/49869/exe (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49869 ceph-osd ceph NOFD /proc/49511/task/49869/fd (opendir: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49880 ceph-osd ceph cwd unknown /proc/49511/task/49880/cwd (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49880 ceph-osd ceph rtd unknown /proc/49511/task/49880/root (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49880 ceph-osd ceph txt unknown /proc/49511/task/49880/exe (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49880 ceph-osd ceph NOFD /proc/49511/task/49880/fd (opendir: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49901 ceph-osd ceph cwd unknown /proc/49511/task/49901/cwd (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49901 ceph-osd ceph rtd unknown /proc/49511/task/49901/root (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49901 ceph-osd ceph txt unknown /proc/49511/task/49901/exe (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49901 ceph-osd ceph NOFD /proc/49511/task/49901/fd (opendir: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49923 ceph-osd ceph cwd unknown /proc/49511/task/49923/cwd (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49923 ceph-osd ceph rtd unknown /proc/49511/task/49923/root (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49923 ceph-osd ceph txt unknown /proc/49511/task/49923/exe (readlink: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49923 ceph-osd ceph NOFD /proc/49511/task/49923/fd (opendir: Permission denied) 2024-04-22T13:30:08.592 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49964 ceph-osd ceph cwd unknown /proc/49511/task/49964/cwd (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49964 ceph-osd ceph rtd unknown /proc/49511/task/49964/root (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49964 ceph-osd ceph txt unknown /proc/49511/task/49964/exe (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49964 ceph-osd ceph NOFD /proc/49511/task/49964/fd (opendir: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49966 ceph-osd ceph cwd unknown /proc/49511/task/49966/cwd (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49966 ceph-osd ceph rtd unknown /proc/49511/task/49966/root (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49966 ceph-osd ceph txt unknown /proc/49511/task/49966/exe (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49966 ceph-osd ceph NOFD /proc/49511/task/49966/fd (opendir: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49967 safe_time ceph cwd unknown /proc/49511/task/49967/cwd (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49967 safe_time ceph rtd unknown /proc/49511/task/49967/root (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49967 safe_time ceph txt unknown /proc/49511/task/49967/exe (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49967 safe_time ceph NOFD /proc/49511/task/49967/fd (opendir: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49968 safe_time ceph cwd unknown /proc/49511/task/49968/cwd (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49968 safe_time ceph rtd unknown /proc/49511/task/49968/root (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49968 safe_time ceph txt unknown /proc/49511/task/49968/exe (readlink: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49968 safe_time ceph NOFD /proc/49511/task/49968/fd (opendir: Permission denied) 2024-04-22T13:30:08.593 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49969 safe_time ceph cwd unknown /proc/49511/task/49969/cwd (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49969 safe_time ceph rtd unknown /proc/49511/task/49969/root (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49969 safe_time ceph txt unknown /proc/49511/task/49969/exe (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49969 safe_time ceph NOFD /proc/49511/task/49969/fd (opendir: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49970 safe_time ceph cwd unknown /proc/49511/task/49970/cwd (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49970 safe_time ceph rtd unknown /proc/49511/task/49970/root (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49970 safe_time ceph txt unknown /proc/49511/task/49970/exe (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49970 safe_time ceph NOFD /proc/49511/task/49970/fd (opendir: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49971 fn_anonym ceph cwd unknown /proc/49511/task/49971/cwd (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49971 fn_anonym ceph rtd unknown /proc/49511/task/49971/root (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49971 fn_anonym ceph txt unknown /proc/49511/task/49971/exe (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49971 fn_anonym ceph NOFD /proc/49511/task/49971/fd (opendir: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49977 rocksdb:l ceph cwd unknown /proc/49511/task/49977/cwd (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49977 rocksdb:l ceph rtd unknown /proc/49511/task/49977/root (readlink: Permission denied) 2024-04-22T13:30:08.594 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49977 rocksdb:l ceph txt unknown /proc/49511/task/49977/exe (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49977 rocksdb:l ceph NOFD /proc/49511/task/49977/fd (opendir: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49978 rocksdb:l ceph cwd unknown /proc/49511/task/49978/cwd (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49978 rocksdb:l ceph rtd unknown /proc/49511/task/49978/root (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49978 rocksdb:l ceph txt unknown /proc/49511/task/49978/exe (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49978 rocksdb:l ceph NOFD /proc/49511/task/49978/fd (opendir: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49979 rocksdb:l ceph cwd unknown /proc/49511/task/49979/cwd (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49979 rocksdb:l ceph rtd unknown /proc/49511/task/49979/root (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49979 rocksdb:l ceph txt unknown /proc/49511/task/49979/exe (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49979 rocksdb:l ceph NOFD /proc/49511/task/49979/fd (opendir: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49980 rocksdb:h ceph cwd unknown /proc/49511/task/49980/cwd (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49980 rocksdb:h ceph rtd unknown /proc/49511/task/49980/root (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49980 rocksdb:h ceph txt unknown /proc/49511/task/49980/exe (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 49980 rocksdb:h ceph NOFD /proc/49511/task/49980/fd (opendir: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50419 bstore_ai ceph cwd unknown /proc/49511/task/50419/cwd (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50419 bstore_ai ceph rtd unknown /proc/49511/task/50419/root (readlink: Permission denied) 2024-04-22T13:30:08.595 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50419 bstore_ai ceph txt unknown /proc/49511/task/50419/exe (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50419 bstore_ai ceph NOFD /proc/49511/task/50419/fd (opendir: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50682 bstore_ai ceph cwd unknown /proc/49511/task/50682/cwd (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50682 bstore_ai ceph rtd unknown /proc/49511/task/50682/root (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50682 bstore_ai ceph txt unknown /proc/49511/task/50682/exe (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50682 bstore_ai ceph NOFD /proc/49511/task/50682/fd (opendir: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50864 ceph-osd ceph cwd unknown /proc/49511/task/50864/cwd (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50864 ceph-osd ceph rtd unknown /proc/49511/task/50864/root (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50864 ceph-osd ceph txt unknown /proc/49511/task/50864/exe (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50864 ceph-osd ceph NOFD /proc/49511/task/50864/fd (opendir: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50865 cfin ceph cwd unknown /proc/49511/task/50865/cwd (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50865 cfin ceph rtd unknown /proc/49511/task/50865/root (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50865 cfin ceph txt unknown /proc/49511/task/50865/exe (readlink: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50865 cfin ceph NOFD /proc/49511/task/50865/fd (opendir: Permission denied) 2024-04-22T13:30:08.596 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50866 bstore_kv ceph cwd unknown /proc/49511/task/50866/cwd (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50866 bstore_kv ceph rtd unknown /proc/49511/task/50866/root (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50866 bstore_kv ceph txt unknown /proc/49511/task/50866/exe (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50866 bstore_kv ceph NOFD /proc/49511/task/50866/fd (opendir: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50867 bstore_kv ceph cwd unknown /proc/49511/task/50867/cwd (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50867 bstore_kv ceph rtd unknown /proc/49511/task/50867/root (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50867 bstore_kv ceph txt unknown /proc/49511/task/50867/exe (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50867 bstore_kv ceph NOFD /proc/49511/task/50867/fd (opendir: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50868 bstore_me ceph cwd unknown /proc/49511/task/50868/cwd (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50868 bstore_me ceph rtd unknown /proc/49511/task/50868/root (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50868 bstore_me ceph txt unknown /proc/49511/task/50868/exe (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50868 bstore_me ceph NOFD /proc/49511/task/50868/fd (opendir: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50869 ms_dispat ceph cwd unknown /proc/49511/task/50869/cwd (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50869 ms_dispat ceph rtd unknown /proc/49511/task/50869/root (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50869 ms_dispat ceph txt unknown /proc/49511/task/50869/exe (readlink: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50869 ms_dispat ceph NOFD /proc/49511/task/50869/fd (opendir: Permission denied) 2024-04-22T13:30:08.597 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50870 ms_local ceph cwd unknown /proc/49511/task/50870/cwd (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50870 ms_local ceph rtd unknown /proc/49511/task/50870/root (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50870 ms_local ceph txt unknown /proc/49511/task/50870/exe (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50870 ms_local ceph NOFD /proc/49511/task/50870/fd (opendir: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50871 safe_time ceph cwd unknown /proc/49511/task/50871/cwd (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50871 safe_time ceph rtd unknown /proc/49511/task/50871/root (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50871 safe_time ceph txt unknown /proc/49511/task/50871/exe (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50871 safe_time ceph NOFD /proc/49511/task/50871/fd (opendir: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50872 safe_time ceph cwd unknown /proc/49511/task/50872/cwd (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50872 safe_time ceph rtd unknown /proc/49511/task/50872/root (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50872 safe_time ceph txt unknown /proc/49511/task/50872/exe (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50872 safe_time ceph NOFD /proc/49511/task/50872/fd (opendir: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50873 ms_dispat ceph cwd unknown /proc/49511/task/50873/cwd (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50873 ms_dispat ceph rtd unknown /proc/49511/task/50873/root (readlink: Permission denied) 2024-04-22T13:30:08.598 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50873 ms_dispat ceph txt unknown /proc/49511/task/50873/exe (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50873 ms_dispat ceph NOFD /proc/49511/task/50873/fd (opendir: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50874 ms_local ceph cwd unknown /proc/49511/task/50874/cwd (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50874 ms_local ceph rtd unknown /proc/49511/task/50874/root (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50874 ms_local ceph txt unknown /proc/49511/task/50874/exe (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50874 ms_local ceph NOFD /proc/49511/task/50874/fd (opendir: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50875 ms_dispat ceph cwd unknown /proc/49511/task/50875/cwd (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50875 ms_dispat ceph rtd unknown /proc/49511/task/50875/root (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50875 ms_dispat ceph txt unknown /proc/49511/task/50875/exe (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50875 ms_dispat ceph NOFD /proc/49511/task/50875/fd (opendir: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50876 ms_local ceph cwd unknown /proc/49511/task/50876/cwd (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50876 ms_local ceph rtd unknown /proc/49511/task/50876/root (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50876 ms_local ceph txt unknown /proc/49511/task/50876/exe (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50876 ms_local ceph NOFD /proc/49511/task/50876/fd (opendir: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50877 ms_dispat ceph cwd unknown /proc/49511/task/50877/cwd (readlink: Permission denied) 2024-04-22T13:30:08.599 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50877 ms_dispat ceph rtd unknown /proc/49511/task/50877/root (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50877 ms_dispat ceph txt unknown /proc/49511/task/50877/exe (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50877 ms_dispat ceph NOFD /proc/49511/task/50877/fd (opendir: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50878 ms_local ceph cwd unknown /proc/49511/task/50878/cwd (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50878 ms_local ceph rtd unknown /proc/49511/task/50878/root (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50878 ms_local ceph txt unknown /proc/49511/task/50878/exe (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50878 ms_local ceph NOFD /proc/49511/task/50878/fd (opendir: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50879 ms_dispat ceph cwd unknown /proc/49511/task/50879/cwd (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50879 ms_dispat ceph rtd unknown /proc/49511/task/50879/root (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50879 ms_dispat ceph txt unknown /proc/49511/task/50879/exe (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50879 ms_dispat ceph NOFD /proc/49511/task/50879/fd (opendir: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50880 ms_local ceph cwd unknown /proc/49511/task/50880/cwd (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50880 ms_local ceph rtd unknown /proc/49511/task/50880/root (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50880 ms_local ceph txt unknown /proc/49511/task/50880/exe (readlink: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50880 ms_local ceph NOFD /proc/49511/task/50880/fd (opendir: Permission denied) 2024-04-22T13:30:08.600 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50881 ms_dispat ceph cwd unknown /proc/49511/task/50881/cwd (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50881 ms_dispat ceph rtd unknown /proc/49511/task/50881/root (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50881 ms_dispat ceph txt unknown /proc/49511/task/50881/exe (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50881 ms_dispat ceph NOFD /proc/49511/task/50881/fd (opendir: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50882 ms_local ceph cwd unknown /proc/49511/task/50882/cwd (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50882 ms_local ceph rtd unknown /proc/49511/task/50882/root (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50882 ms_local ceph txt unknown /proc/49511/task/50882/exe (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50882 ms_local ceph NOFD /proc/49511/task/50882/fd (opendir: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50883 ms_dispat ceph cwd unknown /proc/49511/task/50883/cwd (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50883 ms_dispat ceph rtd unknown /proc/49511/task/50883/root (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50883 ms_dispat ceph txt unknown /proc/49511/task/50883/exe (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50883 ms_dispat ceph NOFD /proc/49511/task/50883/fd (opendir: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50884 ms_local ceph cwd unknown /proc/49511/task/50884/cwd (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50884 ms_local ceph rtd unknown /proc/49511/task/50884/root (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50884 ms_local ceph txt unknown /proc/49511/task/50884/exe (readlink: Permission denied) 2024-04-22T13:30:08.601 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50884 ms_local ceph NOFD /proc/49511/task/50884/fd (opendir: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50885 fn_anonym ceph cwd unknown /proc/49511/task/50885/cwd (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50885 fn_anonym ceph rtd unknown /proc/49511/task/50885/root (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50885 fn_anonym ceph txt unknown /proc/49511/task/50885/exe (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50885 fn_anonym ceph NOFD /proc/49511/task/50885/fd (opendir: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50886 finisher ceph cwd unknown /proc/49511/task/50886/cwd (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50886 finisher ceph rtd unknown /proc/49511/task/50886/root (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50886 finisher ceph txt unknown /proc/49511/task/50886/exe (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50886 finisher ceph NOFD /proc/49511/task/50886/fd (opendir: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50887 safe_time ceph cwd unknown /proc/49511/task/50887/cwd (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50887 safe_time ceph rtd unknown /proc/49511/task/50887/root (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50887 safe_time ceph txt unknown /proc/49511/task/50887/exe (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50887 safe_time ceph NOFD /proc/49511/task/50887/fd (opendir: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50888 safe_time ceph cwd unknown /proc/49511/task/50888/cwd (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50888 safe_time ceph rtd unknown /proc/49511/task/50888/root (readlink: Permission denied) 2024-04-22T13:30:08.602 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50888 safe_time ceph txt unknown /proc/49511/task/50888/exe (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50888 safe_time ceph NOFD /proc/49511/task/50888/fd (opendir: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50889 ceph-osd ceph cwd unknown /proc/49511/task/50889/cwd (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50889 ceph-osd ceph rtd unknown /proc/49511/task/50889/root (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50889 ceph-osd ceph txt unknown /proc/49511/task/50889/exe (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50889 ceph-osd ceph NOFD /proc/49511/task/50889/fd (opendir: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50890 osd_srv_a ceph cwd unknown /proc/49511/task/50890/cwd (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50890 osd_srv_a ceph rtd unknown /proc/49511/task/50890/root (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50890 osd_srv_a ceph txt unknown /proc/49511/task/50890/exe (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50890 osd_srv_a ceph NOFD /proc/49511/task/50890/fd (opendir: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50891 tp_osd_tp ceph cwd unknown /proc/49511/task/50891/cwd (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50891 tp_osd_tp ceph rtd unknown /proc/49511/task/50891/root (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50891 tp_osd_tp ceph txt unknown /proc/49511/task/50891/exe (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50891 tp_osd_tp ceph NOFD /proc/49511/task/50891/fd (opendir: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50892 tp_osd_tp ceph cwd unknown /proc/49511/task/50892/cwd (readlink: Permission denied) 2024-04-22T13:30:08.603 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50892 tp_osd_tp ceph rtd unknown /proc/49511/task/50892/root (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50892 tp_osd_tp ceph txt unknown /proc/49511/task/50892/exe (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50892 tp_osd_tp ceph NOFD /proc/49511/task/50892/fd (opendir: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50893 tp_osd_tp ceph cwd unknown /proc/49511/task/50893/cwd (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50893 tp_osd_tp ceph rtd unknown /proc/49511/task/50893/root (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50893 tp_osd_tp ceph txt unknown /proc/49511/task/50893/exe (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50893 tp_osd_tp ceph NOFD /proc/49511/task/50893/fd (opendir: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50894 tp_osd_tp ceph cwd unknown /proc/49511/task/50894/cwd (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50894 tp_osd_tp ceph rtd unknown /proc/49511/task/50894/root (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50894 tp_osd_tp ceph txt unknown /proc/49511/task/50894/exe (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50894 tp_osd_tp ceph NOFD /proc/49511/task/50894/fd (opendir: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50895 tp_osd_tp ceph cwd unknown /proc/49511/task/50895/cwd (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50895 tp_osd_tp ceph rtd unknown /proc/49511/task/50895/root (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50895 tp_osd_tp ceph txt unknown /proc/49511/task/50895/exe (readlink: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50895 tp_osd_tp ceph NOFD /proc/49511/task/50895/fd (opendir: Permission denied) 2024-04-22T13:30:08.604 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50896 tp_osd_tp ceph cwd unknown /proc/49511/task/50896/cwd (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50896 tp_osd_tp ceph rtd unknown /proc/49511/task/50896/root (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50896 tp_osd_tp ceph txt unknown /proc/49511/task/50896/exe (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50896 tp_osd_tp ceph NOFD /proc/49511/task/50896/fd (opendir: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50897 tp_osd_tp ceph cwd unknown /proc/49511/task/50897/cwd (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50897 tp_osd_tp ceph rtd unknown /proc/49511/task/50897/root (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50897 tp_osd_tp ceph txt unknown /proc/49511/task/50897/exe (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50897 tp_osd_tp ceph NOFD /proc/49511/task/50897/fd (opendir: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50898 tp_osd_tp ceph cwd unknown /proc/49511/task/50898/cwd (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50898 tp_osd_tp ceph rtd unknown /proc/49511/task/50898/root (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50898 tp_osd_tp ceph txt unknown /proc/49511/task/50898/exe (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50898 tp_osd_tp ceph NOFD /proc/49511/task/50898/fd (opendir: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50899 tp_osd_tp ceph cwd unknown /proc/49511/task/50899/cwd (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50899 tp_osd_tp ceph rtd unknown /proc/49511/task/50899/root (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50899 tp_osd_tp ceph txt unknown /proc/49511/task/50899/exe (readlink: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50899 tp_osd_tp ceph NOFD /proc/49511/task/50899/fd (opendir: Permission denied) 2024-04-22T13:30:08.605 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50900 tp_osd_tp ceph cwd unknown /proc/49511/task/50900/cwd (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50900 tp_osd_tp ceph rtd unknown /proc/49511/task/50900/root (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50900 tp_osd_tp ceph txt unknown /proc/49511/task/50900/exe (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50900 tp_osd_tp ceph NOFD /proc/49511/task/50900/fd (opendir: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50901 tp_osd_tp ceph cwd unknown /proc/49511/task/50901/cwd (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50901 tp_osd_tp ceph rtd unknown /proc/49511/task/50901/root (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50901 tp_osd_tp ceph txt unknown /proc/49511/task/50901/exe (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50901 tp_osd_tp ceph NOFD /proc/49511/task/50901/fd (opendir: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50902 tp_osd_tp ceph cwd unknown /proc/49511/task/50902/cwd (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50902 tp_osd_tp ceph rtd unknown /proc/49511/task/50902/root (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50902 tp_osd_tp ceph txt unknown /proc/49511/task/50902/exe (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50902 tp_osd_tp ceph NOFD /proc/49511/task/50902/fd (opendir: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50903 tp_osd_tp ceph cwd unknown /proc/49511/task/50903/cwd (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50903 tp_osd_tp ceph rtd unknown /proc/49511/task/50903/root (readlink: Permission denied) 2024-04-22T13:30:08.606 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50903 tp_osd_tp ceph txt unknown /proc/49511/task/50903/exe (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50903 tp_osd_tp ceph NOFD /proc/49511/task/50903/fd (opendir: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50904 tp_osd_tp ceph cwd unknown /proc/49511/task/50904/cwd (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50904 tp_osd_tp ceph rtd unknown /proc/49511/task/50904/root (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50904 tp_osd_tp ceph txt unknown /proc/49511/task/50904/exe (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50904 tp_osd_tp ceph NOFD /proc/49511/task/50904/fd (opendir: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50905 tp_osd_tp ceph cwd unknown /proc/49511/task/50905/cwd (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50905 tp_osd_tp ceph rtd unknown /proc/49511/task/50905/root (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50905 tp_osd_tp ceph txt unknown /proc/49511/task/50905/exe (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50905 tp_osd_tp ceph NOFD /proc/49511/task/50905/fd (opendir: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50906 tp_osd_tp ceph cwd unknown /proc/49511/task/50906/cwd (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50906 tp_osd_tp ceph rtd unknown /proc/49511/task/50906/root (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50906 tp_osd_tp ceph txt unknown /proc/49511/task/50906/exe (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50906 tp_osd_tp ceph NOFD /proc/49511/task/50906/fd (opendir: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50907 osd_srv_h ceph cwd unknown /proc/49511/task/50907/cwd (readlink: Permission denied) 2024-04-22T13:30:08.607 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50907 osd_srv_h ceph rtd unknown /proc/49511/task/50907/root (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50907 osd_srv_h ceph txt unknown /proc/49511/task/50907/exe (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 49511 50907 osd_srv_h ceph NOFD /proc/49511/task/50907/fd (opendir: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 50939 root cwd unknown /proc/50939/cwd (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 50939 root rtd unknown /proc/50939/root (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 50939 root txt unknown /proc/50939/exe (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 50939 root NOFD /proc/50939/fd (opendir: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 50970 root cwd unknown /proc/50970/cwd (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 50970 root rtd unknown /proc/50970/root (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 50970 root txt unknown /proc/50970/exe (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 50970 root NOFD /proc/50970/fd (opendir: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 55952 root cwd unknown /proc/55952/cwd (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 55952 root rtd unknown /proc/55952/root (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 55952 root txt unknown /proc/55952/exe (readlink: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/2 55952 root NOFD /proc/55952/fd (opendir: Permission denied) 2024-04-22T13:30:08.608 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 56160 root cwd unknown /proc/56160/cwd (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 56160 root rtd unknown /proc/56160/root (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 56160 root txt unknown /proc/56160/exe (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 56160 root NOFD /proc/56160/fd (opendir: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 56527 root cwd unknown /proc/56527/cwd (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 56527 root rtd unknown /proc/56527/root (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 56527 root txt unknown /proc/56527/exe (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 56527 root NOFD /proc/56527/fd (opendir: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 56529 root cwd unknown /proc/56529/cwd (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 56529 root rtd unknown /proc/56529/root (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 56529 root txt unknown /proc/56529/exe (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 56529 root NOFD /proc/56529/fd (opendir: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 ceph cwd unknown /proc/56531/cwd (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 ceph rtd unknown /proc/56531/root (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 ceph txt unknown /proc/56531/exe (readlink: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 ceph NOFD /proc/56531/fd (opendir: Permission denied) 2024-04-22T13:30:08.609 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56532 log ceph cwd unknown /proc/56531/task/56532/cwd (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56532 log ceph rtd unknown /proc/56531/task/56532/root (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56532 log ceph txt unknown /proc/56531/task/56532/exe (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56532 log ceph NOFD /proc/56531/task/56532/fd (opendir: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56534 msgr-work ceph cwd unknown /proc/56531/task/56534/cwd (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56534 msgr-work ceph rtd unknown /proc/56531/task/56534/root (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56534 msgr-work ceph txt unknown /proc/56531/task/56534/exe (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56534 msgr-work ceph NOFD /proc/56531/task/56534/fd (opendir: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56535 msgr-work ceph cwd unknown /proc/56531/task/56535/cwd (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56535 msgr-work ceph rtd unknown /proc/56531/task/56535/root (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56535 msgr-work ceph txt unknown /proc/56531/task/56535/exe (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56535 msgr-work ceph NOFD /proc/56531/task/56535/fd (opendir: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56536 msgr-work ceph cwd unknown /proc/56531/task/56536/cwd (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56536 msgr-work ceph rtd unknown /proc/56531/task/56536/root (readlink: Permission denied) 2024-04-22T13:30:08.610 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56536 msgr-work ceph txt unknown /proc/56531/task/56536/exe (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56536 msgr-work ceph NOFD /proc/56531/task/56536/fd (opendir: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56540 service ceph cwd unknown /proc/56531/task/56540/cwd (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56540 service ceph rtd unknown /proc/56531/task/56540/root (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56540 service ceph txt unknown /proc/56531/task/56540/exe (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56540 service ceph NOFD /proc/56531/task/56540/fd (opendir: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56541 admin_soc ceph cwd unknown /proc/56531/task/56541/cwd (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56541 admin_soc ceph rtd unknown /proc/56531/task/56541/root (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56541 admin_soc ceph txt unknown /proc/56531/task/56541/exe (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56541 admin_soc ceph NOFD /proc/56531/task/56541/fd (opendir: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56683 signal_ha ceph cwd unknown /proc/56531/task/56683/cwd (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56683 signal_ha ceph rtd unknown /proc/56531/task/56683/root (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56683 signal_ha ceph txt unknown /proc/56531/task/56683/exe (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56683 signal_ha ceph NOFD /proc/56531/task/56683/fd (opendir: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56684 io_contex ceph cwd unknown /proc/56531/task/56684/cwd (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56684 io_contex ceph rtd unknown /proc/56531/task/56684/root (readlink: Permission denied) 2024-04-22T13:30:08.611 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56684 io_contex ceph txt unknown /proc/56531/task/56684/exe (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56684 io_contex ceph NOFD /proc/56531/task/56684/fd (opendir: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56685 io_contex ceph cwd unknown /proc/56531/task/56685/cwd (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56685 io_contex ceph rtd unknown /proc/56531/task/56685/root (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56685 io_contex ceph txt unknown /proc/56531/task/56685/exe (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56685 io_contex ceph NOFD /proc/56531/task/56685/fd (opendir: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56794 OpHistory ceph cwd unknown /proc/56531/task/56794/cwd (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56794 OpHistory ceph rtd unknown /proc/56531/task/56794/root (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56794 OpHistory ceph txt unknown /proc/56531/task/56794/exe (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56794 OpHistory ceph NOFD /proc/56531/task/56794/fd (opendir: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56795 ceph_time ceph cwd unknown /proc/56531/task/56795/cwd (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56795 ceph_time ceph rtd unknown /proc/56531/task/56795/root (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56795 ceph_time ceph txt unknown /proc/56531/task/56795/exe (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56795 ceph_time ceph NOFD /proc/56531/task/56795/fd (opendir: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56956 ceph-osd ceph cwd unknown /proc/56531/task/56956/cwd (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56956 ceph-osd ceph rtd unknown /proc/56531/task/56956/root (readlink: Permission denied) 2024-04-22T13:30:08.612 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56956 ceph-osd ceph txt unknown /proc/56531/task/56956/exe (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56956 ceph-osd ceph NOFD /proc/56531/task/56956/fd (opendir: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56957 safe_time ceph cwd unknown /proc/56531/task/56957/cwd (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56957 safe_time ceph rtd unknown /proc/56531/task/56957/root (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56957 safe_time ceph txt unknown /proc/56531/task/56957/exe (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56957 safe_time ceph NOFD /proc/56531/task/56957/fd (opendir: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56958 safe_time ceph cwd unknown /proc/56531/task/56958/cwd (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56958 safe_time ceph rtd unknown /proc/56531/task/56958/root (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56958 safe_time ceph txt unknown /proc/56531/task/56958/exe (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56958 safe_time ceph NOFD /proc/56531/task/56958/fd (opendir: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56959 safe_time ceph cwd unknown /proc/56531/task/56959/cwd (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56959 safe_time ceph rtd unknown /proc/56531/task/56959/root (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56959 safe_time ceph txt unknown /proc/56531/task/56959/exe (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56959 safe_time ceph NOFD /proc/56531/task/56959/fd (opendir: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56960 safe_time ceph cwd unknown /proc/56531/task/56960/cwd (readlink: Permission denied) 2024-04-22T13:30:08.613 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56960 safe_time ceph rtd unknown /proc/56531/task/56960/root (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56960 safe_time ceph txt unknown /proc/56531/task/56960/exe (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56960 safe_time ceph NOFD /proc/56531/task/56960/fd (opendir: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56961 fn_anonym ceph cwd unknown /proc/56531/task/56961/cwd (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56961 fn_anonym ceph rtd unknown /proc/56531/task/56961/root (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56961 fn_anonym ceph txt unknown /proc/56531/task/56961/exe (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56961 fn_anonym ceph NOFD /proc/56531/task/56961/fd (opendir: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56966 rocksdb:l ceph cwd unknown /proc/56531/task/56966/cwd (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56966 rocksdb:l ceph rtd unknown /proc/56531/task/56966/root (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56966 rocksdb:l ceph txt unknown /proc/56531/task/56966/exe (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56966 rocksdb:l ceph NOFD /proc/56531/task/56966/fd (opendir: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56967 rocksdb:l ceph cwd unknown /proc/56531/task/56967/cwd (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56967 rocksdb:l ceph rtd unknown /proc/56531/task/56967/root (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56967 rocksdb:l ceph txt unknown /proc/56531/task/56967/exe (readlink: Permission denied) 2024-04-22T13:30:08.614 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56967 rocksdb:l ceph NOFD /proc/56531/task/56967/fd (opendir: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56968 rocksdb:l ceph cwd unknown /proc/56531/task/56968/cwd (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56968 rocksdb:l ceph rtd unknown /proc/56531/task/56968/root (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56968 rocksdb:l ceph txt unknown /proc/56531/task/56968/exe (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56968 rocksdb:l ceph NOFD /proc/56531/task/56968/fd (opendir: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56969 rocksdb:h ceph cwd unknown /proc/56531/task/56969/cwd (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56969 rocksdb:h ceph rtd unknown /proc/56531/task/56969/root (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56969 rocksdb:h ceph txt unknown /proc/56531/task/56969/exe (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 56969 rocksdb:h ceph NOFD /proc/56531/task/56969/fd (opendir: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57403 bstore_ai ceph cwd unknown /proc/56531/task/57403/cwd (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57403 bstore_ai ceph rtd unknown /proc/56531/task/57403/root (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57403 bstore_ai ceph txt unknown /proc/56531/task/57403/exe (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57403 bstore_ai ceph NOFD /proc/56531/task/57403/fd (opendir: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57606 bstore_ai ceph cwd unknown /proc/56531/task/57606/cwd (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57606 bstore_ai ceph rtd unknown /proc/56531/task/57606/root (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57606 bstore_ai ceph txt unknown /proc/56531/task/57606/exe (readlink: Permission denied) 2024-04-22T13:30:08.615 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57606 bstore_ai ceph NOFD /proc/56531/task/57606/fd (opendir: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57788 ceph-osd ceph cwd unknown /proc/56531/task/57788/cwd (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57788 ceph-osd ceph rtd unknown /proc/56531/task/57788/root (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57788 ceph-osd ceph txt unknown /proc/56531/task/57788/exe (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57788 ceph-osd ceph NOFD /proc/56531/task/57788/fd (opendir: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57789 cfin ceph cwd unknown /proc/56531/task/57789/cwd (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57789 cfin ceph rtd unknown /proc/56531/task/57789/root (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57789 cfin ceph txt unknown /proc/56531/task/57789/exe (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57789 cfin ceph NOFD /proc/56531/task/57789/fd (opendir: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57790 bstore_kv ceph cwd unknown /proc/56531/task/57790/cwd (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57790 bstore_kv ceph rtd unknown /proc/56531/task/57790/root (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57790 bstore_kv ceph txt unknown /proc/56531/task/57790/exe (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57790 bstore_kv ceph NOFD /proc/56531/task/57790/fd (opendir: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57791 bstore_kv ceph cwd unknown /proc/56531/task/57791/cwd (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57791 bstore_kv ceph rtd unknown /proc/56531/task/57791/root (readlink: Permission denied) 2024-04-22T13:30:08.616 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57791 bstore_kv ceph txt unknown /proc/56531/task/57791/exe (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57791 bstore_kv ceph NOFD /proc/56531/task/57791/fd (opendir: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57792 bstore_me ceph cwd unknown /proc/56531/task/57792/cwd (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57792 bstore_me ceph rtd unknown /proc/56531/task/57792/root (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57792 bstore_me ceph txt unknown /proc/56531/task/57792/exe (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57792 bstore_me ceph NOFD /proc/56531/task/57792/fd (opendir: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57793 ms_dispat ceph cwd unknown /proc/56531/task/57793/cwd (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57793 ms_dispat ceph rtd unknown /proc/56531/task/57793/root (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57793 ms_dispat ceph txt unknown /proc/56531/task/57793/exe (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57793 ms_dispat ceph NOFD /proc/56531/task/57793/fd (opendir: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57794 ms_local ceph cwd unknown /proc/56531/task/57794/cwd (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57794 ms_local ceph rtd unknown /proc/56531/task/57794/root (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57794 ms_local ceph txt unknown /proc/56531/task/57794/exe (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57794 ms_local ceph NOFD /proc/56531/task/57794/fd (opendir: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57795 safe_time ceph cwd unknown /proc/56531/task/57795/cwd (readlink: Permission denied) 2024-04-22T13:30:08.617 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57795 safe_time ceph rtd unknown /proc/56531/task/57795/root (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57795 safe_time ceph txt unknown /proc/56531/task/57795/exe (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57795 safe_time ceph NOFD /proc/56531/task/57795/fd (opendir: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57796 safe_time ceph cwd unknown /proc/56531/task/57796/cwd (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57796 safe_time ceph rtd unknown /proc/56531/task/57796/root (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57796 safe_time ceph txt unknown /proc/56531/task/57796/exe (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57796 safe_time ceph NOFD /proc/56531/task/57796/fd (opendir: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57797 ms_dispat ceph cwd unknown /proc/56531/task/57797/cwd (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57797 ms_dispat ceph rtd unknown /proc/56531/task/57797/root (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57797 ms_dispat ceph txt unknown /proc/56531/task/57797/exe (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57797 ms_dispat ceph NOFD /proc/56531/task/57797/fd (opendir: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57798 ms_local ceph cwd unknown /proc/56531/task/57798/cwd (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57798 ms_local ceph rtd unknown /proc/56531/task/57798/root (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57798 ms_local ceph txt unknown /proc/56531/task/57798/exe (readlink: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57798 ms_local ceph NOFD /proc/56531/task/57798/fd (opendir: Permission denied) 2024-04-22T13:30:08.618 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57799 ms_dispat ceph cwd unknown /proc/56531/task/57799/cwd (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57799 ms_dispat ceph rtd unknown /proc/56531/task/57799/root (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57799 ms_dispat ceph txt unknown /proc/56531/task/57799/exe (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57799 ms_dispat ceph NOFD /proc/56531/task/57799/fd (opendir: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57800 ms_local ceph cwd unknown /proc/56531/task/57800/cwd (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57800 ms_local ceph rtd unknown /proc/56531/task/57800/root (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57800 ms_local ceph txt unknown /proc/56531/task/57800/exe (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57800 ms_local ceph NOFD /proc/56531/task/57800/fd (opendir: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57801 ms_dispat ceph cwd unknown /proc/56531/task/57801/cwd (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57801 ms_dispat ceph rtd unknown /proc/56531/task/57801/root (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57801 ms_dispat ceph txt unknown /proc/56531/task/57801/exe (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57801 ms_dispat ceph NOFD /proc/56531/task/57801/fd (opendir: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57802 ms_local ceph cwd unknown /proc/56531/task/57802/cwd (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57802 ms_local ceph rtd unknown /proc/56531/task/57802/root (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57802 ms_local ceph txt unknown /proc/56531/task/57802/exe (readlink: Permission denied) 2024-04-22T13:30:08.619 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57802 ms_local ceph NOFD /proc/56531/task/57802/fd (opendir: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57803 ms_dispat ceph cwd unknown /proc/56531/task/57803/cwd (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57803 ms_dispat ceph rtd unknown /proc/56531/task/57803/root (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57803 ms_dispat ceph txt unknown /proc/56531/task/57803/exe (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57803 ms_dispat ceph NOFD /proc/56531/task/57803/fd (opendir: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57804 ms_local ceph cwd unknown /proc/56531/task/57804/cwd (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57804 ms_local ceph rtd unknown /proc/56531/task/57804/root (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57804 ms_local ceph txt unknown /proc/56531/task/57804/exe (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57804 ms_local ceph NOFD /proc/56531/task/57804/fd (opendir: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57805 ms_dispat ceph cwd unknown /proc/56531/task/57805/cwd (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57805 ms_dispat ceph rtd unknown /proc/56531/task/57805/root (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57805 ms_dispat ceph txt unknown /proc/56531/task/57805/exe (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57805 ms_dispat ceph NOFD /proc/56531/task/57805/fd (opendir: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57806 ms_local ceph cwd unknown /proc/56531/task/57806/cwd (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57806 ms_local ceph rtd unknown /proc/56531/task/57806/root (readlink: Permission denied) 2024-04-22T13:30:08.620 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57806 ms_local ceph txt unknown /proc/56531/task/57806/exe (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57806 ms_local ceph NOFD /proc/56531/task/57806/fd (opendir: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57807 ms_dispat ceph cwd unknown /proc/56531/task/57807/cwd (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57807 ms_dispat ceph rtd unknown /proc/56531/task/57807/root (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57807 ms_dispat ceph txt unknown /proc/56531/task/57807/exe (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57807 ms_dispat ceph NOFD /proc/56531/task/57807/fd (opendir: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57808 ms_local ceph cwd unknown /proc/56531/task/57808/cwd (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57808 ms_local ceph rtd unknown /proc/56531/task/57808/root (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57808 ms_local ceph txt unknown /proc/56531/task/57808/exe (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57808 ms_local ceph NOFD /proc/56531/task/57808/fd (opendir: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57809 fn_anonym ceph cwd unknown /proc/56531/task/57809/cwd (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57809 fn_anonym ceph rtd unknown /proc/56531/task/57809/root (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57809 fn_anonym ceph txt unknown /proc/56531/task/57809/exe (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57809 fn_anonym ceph NOFD /proc/56531/task/57809/fd (opendir: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57810 finisher ceph cwd unknown /proc/56531/task/57810/cwd (readlink: Permission denied) 2024-04-22T13:30:08.621 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57810 finisher ceph rtd unknown /proc/56531/task/57810/root (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57810 finisher ceph txt unknown /proc/56531/task/57810/exe (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57810 finisher ceph NOFD /proc/56531/task/57810/fd (opendir: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57811 safe_time ceph cwd unknown /proc/56531/task/57811/cwd (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57811 safe_time ceph rtd unknown /proc/56531/task/57811/root (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57811 safe_time ceph txt unknown /proc/56531/task/57811/exe (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57811 safe_time ceph NOFD /proc/56531/task/57811/fd (opendir: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57812 safe_time ceph cwd unknown /proc/56531/task/57812/cwd (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57812 safe_time ceph rtd unknown /proc/56531/task/57812/root (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57812 safe_time ceph txt unknown /proc/56531/task/57812/exe (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57812 safe_time ceph NOFD /proc/56531/task/57812/fd (opendir: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57813 ceph-osd ceph cwd unknown /proc/56531/task/57813/cwd (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57813 ceph-osd ceph rtd unknown /proc/56531/task/57813/root (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57813 ceph-osd ceph txt unknown /proc/56531/task/57813/exe (readlink: Permission denied) 2024-04-22T13:30:08.622 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57813 ceph-osd ceph NOFD /proc/56531/task/57813/fd (opendir: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57814 osd_srv_a ceph cwd unknown /proc/56531/task/57814/cwd (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57814 osd_srv_a ceph rtd unknown /proc/56531/task/57814/root (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57814 osd_srv_a ceph txt unknown /proc/56531/task/57814/exe (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57814 osd_srv_a ceph NOFD /proc/56531/task/57814/fd (opendir: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57815 tp_osd_tp ceph cwd unknown /proc/56531/task/57815/cwd (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57815 tp_osd_tp ceph rtd unknown /proc/56531/task/57815/root (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57815 tp_osd_tp ceph txt unknown /proc/56531/task/57815/exe (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57815 tp_osd_tp ceph NOFD /proc/56531/task/57815/fd (opendir: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57816 tp_osd_tp ceph cwd unknown /proc/56531/task/57816/cwd (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57816 tp_osd_tp ceph rtd unknown /proc/56531/task/57816/root (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57816 tp_osd_tp ceph txt unknown /proc/56531/task/57816/exe (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57816 tp_osd_tp ceph NOFD /proc/56531/task/57816/fd (opendir: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57817 tp_osd_tp ceph cwd unknown /proc/56531/task/57817/cwd (readlink: Permission denied) 2024-04-22T13:30:08.623 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57817 tp_osd_tp ceph rtd unknown /proc/56531/task/57817/root (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57817 tp_osd_tp ceph txt unknown /proc/56531/task/57817/exe (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57817 tp_osd_tp ceph NOFD /proc/56531/task/57817/fd (opendir: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57818 tp_osd_tp ceph cwd unknown /proc/56531/task/57818/cwd (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57818 tp_osd_tp ceph rtd unknown /proc/56531/task/57818/root (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57818 tp_osd_tp ceph txt unknown /proc/56531/task/57818/exe (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57818 tp_osd_tp ceph NOFD /proc/56531/task/57818/fd (opendir: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57819 tp_osd_tp ceph cwd unknown /proc/56531/task/57819/cwd (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57819 tp_osd_tp ceph rtd unknown /proc/56531/task/57819/root (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57819 tp_osd_tp ceph txt unknown /proc/56531/task/57819/exe (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57819 tp_osd_tp ceph NOFD /proc/56531/task/57819/fd (opendir: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57820 tp_osd_tp ceph cwd unknown /proc/56531/task/57820/cwd (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57820 tp_osd_tp ceph rtd unknown /proc/56531/task/57820/root (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57820 tp_osd_tp ceph txt unknown /proc/56531/task/57820/exe (readlink: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57820 tp_osd_tp ceph NOFD /proc/56531/task/57820/fd (opendir: Permission denied) 2024-04-22T13:30:08.624 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57821 tp_osd_tp ceph cwd unknown /proc/56531/task/57821/cwd (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57821 tp_osd_tp ceph rtd unknown /proc/56531/task/57821/root (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57821 tp_osd_tp ceph txt unknown /proc/56531/task/57821/exe (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57821 tp_osd_tp ceph NOFD /proc/56531/task/57821/fd (opendir: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57822 tp_osd_tp ceph cwd unknown /proc/56531/task/57822/cwd (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57822 tp_osd_tp ceph rtd unknown /proc/56531/task/57822/root (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57822 tp_osd_tp ceph txt unknown /proc/56531/task/57822/exe (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57822 tp_osd_tp ceph NOFD /proc/56531/task/57822/fd (opendir: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57823 tp_osd_tp ceph cwd unknown /proc/56531/task/57823/cwd (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57823 tp_osd_tp ceph rtd unknown /proc/56531/task/57823/root (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57823 tp_osd_tp ceph txt unknown /proc/56531/task/57823/exe (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57823 tp_osd_tp ceph NOFD /proc/56531/task/57823/fd (opendir: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57824 tp_osd_tp ceph cwd unknown /proc/56531/task/57824/cwd (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57824 tp_osd_tp ceph rtd unknown /proc/56531/task/57824/root (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57824 tp_osd_tp ceph txt unknown /proc/56531/task/57824/exe (readlink: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57824 tp_osd_tp ceph NOFD /proc/56531/task/57824/fd (opendir: Permission denied) 2024-04-22T13:30:08.625 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57825 tp_osd_tp ceph cwd unknown /proc/56531/task/57825/cwd (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57825 tp_osd_tp ceph rtd unknown /proc/56531/task/57825/root (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57825 tp_osd_tp ceph txt unknown /proc/56531/task/57825/exe (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57825 tp_osd_tp ceph NOFD /proc/56531/task/57825/fd (opendir: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57826 tp_osd_tp ceph cwd unknown /proc/56531/task/57826/cwd (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57826 tp_osd_tp ceph rtd unknown /proc/56531/task/57826/root (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57826 tp_osd_tp ceph txt unknown /proc/56531/task/57826/exe (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57826 tp_osd_tp ceph NOFD /proc/56531/task/57826/fd (opendir: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57827 tp_osd_tp ceph cwd unknown /proc/56531/task/57827/cwd (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57827 tp_osd_tp ceph rtd unknown /proc/56531/task/57827/root (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57827 tp_osd_tp ceph txt unknown /proc/56531/task/57827/exe (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57827 tp_osd_tp ceph NOFD /proc/56531/task/57827/fd (opendir: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57828 tp_osd_tp ceph cwd unknown /proc/56531/task/57828/cwd (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57828 tp_osd_tp ceph rtd unknown /proc/56531/task/57828/root (readlink: Permission denied) 2024-04-22T13:30:08.626 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57828 tp_osd_tp ceph txt unknown /proc/56531/task/57828/exe (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57828 tp_osd_tp ceph NOFD /proc/56531/task/57828/fd (opendir: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57829 tp_osd_tp ceph cwd unknown /proc/56531/task/57829/cwd (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57829 tp_osd_tp ceph rtd unknown /proc/56531/task/57829/root (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57829 tp_osd_tp ceph txt unknown /proc/56531/task/57829/exe (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57829 tp_osd_tp ceph NOFD /proc/56531/task/57829/fd (opendir: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57830 tp_osd_tp ceph cwd unknown /proc/56531/task/57830/cwd (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57830 tp_osd_tp ceph rtd unknown /proc/56531/task/57830/root (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57830 tp_osd_tp ceph txt unknown /proc/56531/task/57830/exe (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57830 tp_osd_tp ceph NOFD /proc/56531/task/57830/fd (opendir: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57831 osd_srv_h ceph cwd unknown /proc/56531/task/57831/cwd (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57831 osd_srv_h ceph rtd unknown /proc/56531/task/57831/root (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57831 osd_srv_h ceph txt unknown /proc/56531/task/57831/exe (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 56531 57831 osd_srv_h ceph NOFD /proc/56531/task/57831/fd (opendir: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 57832 root cwd unknown /proc/57832/cwd (readlink: Permission denied) 2024-04-22T13:30:08.627 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 57832 root rtd unknown /proc/57832/root (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 57832 root txt unknown /proc/57832/exe (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 57832 root NOFD /proc/57832/fd (opendir: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 57849 root cwd unknown /proc/57849/cwd (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 57849 root rtd unknown /proc/57849/root (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 57849 root txt unknown /proc/57849/exe (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 57849 root NOFD /proc/57849/fd (opendir: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 62957 root cwd unknown /proc/62957/cwd (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 62957 root rtd unknown /proc/62957/root (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 62957 root txt unknown /proc/62957/exe (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 62957 root NOFD /proc/62957/fd (opendir: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 63570 root cwd unknown /proc/63570/cwd (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 63570 root rtd unknown /proc/63570/root (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 63570 root txt unknown /proc/63570/exe (readlink: Permission denied) 2024-04-22T13:30:08.628 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 63570 root NOFD /proc/63570/fd (opendir: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 63572 root cwd unknown /proc/63572/cwd (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 63572 root rtd unknown /proc/63572/root (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 63572 root txt unknown /proc/63572/exe (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 63572 root NOFD /proc/63572/fd (opendir: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 ceph cwd unknown /proc/63574/cwd (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 ceph rtd unknown /proc/63574/root (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 ceph txt unknown /proc/63574/exe (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 ceph NOFD /proc/63574/fd (opendir: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63575 log ceph cwd unknown /proc/63574/task/63575/cwd (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63575 log ceph rtd unknown /proc/63574/task/63575/root (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63575 log ceph txt unknown /proc/63574/task/63575/exe (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63575 log ceph NOFD /proc/63574/task/63575/fd (opendir: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63577 msgr-work ceph cwd unknown /proc/63574/task/63577/cwd (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63577 msgr-work ceph rtd unknown /proc/63574/task/63577/root (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63577 msgr-work ceph txt unknown /proc/63574/task/63577/exe (readlink: Permission denied) 2024-04-22T13:30:08.629 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63577 msgr-work ceph NOFD /proc/63574/task/63577/fd (opendir: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63578 msgr-work ceph cwd unknown /proc/63574/task/63578/cwd (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63578 msgr-work ceph rtd unknown /proc/63574/task/63578/root (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63578 msgr-work ceph txt unknown /proc/63574/task/63578/exe (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63578 msgr-work ceph NOFD /proc/63574/task/63578/fd (opendir: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63579 msgr-work ceph cwd unknown /proc/63574/task/63579/cwd (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63579 msgr-work ceph rtd unknown /proc/63574/task/63579/root (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63579 msgr-work ceph txt unknown /proc/63574/task/63579/exe (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63579 msgr-work ceph NOFD /proc/63574/task/63579/fd (opendir: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63583 service ceph cwd unknown /proc/63574/task/63583/cwd (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63583 service ceph rtd unknown /proc/63574/task/63583/root (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63583 service ceph txt unknown /proc/63574/task/63583/exe (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63583 service ceph NOFD /proc/63574/task/63583/fd (opendir: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63584 admin_soc ceph cwd unknown /proc/63574/task/63584/cwd (readlink: Permission denied) 2024-04-22T13:30:08.630 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63584 admin_soc ceph rtd unknown /proc/63574/task/63584/root (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63584 admin_soc ceph txt unknown /proc/63574/task/63584/exe (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63584 admin_soc ceph NOFD /proc/63574/task/63584/fd (opendir: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63726 signal_ha ceph cwd unknown /proc/63574/task/63726/cwd (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63726 signal_ha ceph rtd unknown /proc/63574/task/63726/root (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63726 signal_ha ceph txt unknown /proc/63574/task/63726/exe (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63726 signal_ha ceph NOFD /proc/63574/task/63726/fd (opendir: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63727 io_contex ceph cwd unknown /proc/63574/task/63727/cwd (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63727 io_contex ceph rtd unknown /proc/63574/task/63727/root (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63727 io_contex ceph txt unknown /proc/63574/task/63727/exe (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63727 io_contex ceph NOFD /proc/63574/task/63727/fd (opendir: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63728 io_contex ceph cwd unknown /proc/63574/task/63728/cwd (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63728 io_contex ceph rtd unknown /proc/63574/task/63728/root (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63728 io_contex ceph txt unknown /proc/63574/task/63728/exe (readlink: Permission denied) 2024-04-22T13:30:08.631 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63728 io_contex ceph NOFD /proc/63574/task/63728/fd (opendir: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63840 OpHistory ceph cwd unknown /proc/63574/task/63840/cwd (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63840 OpHistory ceph rtd unknown /proc/63574/task/63840/root (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63840 OpHistory ceph txt unknown /proc/63574/task/63840/exe (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63840 OpHistory ceph NOFD /proc/63574/task/63840/fd (opendir: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63841 ceph_time ceph cwd unknown /proc/63574/task/63841/cwd (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63841 ceph_time ceph rtd unknown /proc/63574/task/63841/root (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63841 ceph_time ceph txt unknown /proc/63574/task/63841/exe (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63841 ceph_time ceph NOFD /proc/63574/task/63841/fd (opendir: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63877 ceph-osd ceph cwd unknown /proc/63574/task/63877/cwd (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63877 ceph-osd ceph rtd unknown /proc/63574/task/63877/root (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63877 ceph-osd ceph txt unknown /proc/63574/task/63877/exe (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63877 ceph-osd ceph NOFD /proc/63574/task/63877/fd (opendir: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63882 ceph-osd ceph cwd unknown /proc/63574/task/63882/cwd (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63882 ceph-osd ceph rtd unknown /proc/63574/task/63882/root (readlink: Permission denied) 2024-04-22T13:30:08.632 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63882 ceph-osd ceph txt unknown /proc/63574/task/63882/exe (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63882 ceph-osd ceph NOFD /proc/63574/task/63882/fd (opendir: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63891 ceph-osd ceph cwd unknown /proc/63574/task/63891/cwd (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63891 ceph-osd ceph rtd unknown /proc/63574/task/63891/root (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63891 ceph-osd ceph txt unknown /proc/63574/task/63891/exe (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63891 ceph-osd ceph NOFD /proc/63574/task/63891/fd (opendir: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63896 ceph-osd ceph cwd unknown /proc/63574/task/63896/cwd (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63896 ceph-osd ceph rtd unknown /proc/63574/task/63896/root (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63896 ceph-osd ceph txt unknown /proc/63574/task/63896/exe (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63896 ceph-osd ceph NOFD /proc/63574/task/63896/fd (opendir: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63934 ceph-osd ceph cwd unknown /proc/63574/task/63934/cwd (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63934 ceph-osd ceph rtd unknown /proc/63574/task/63934/root (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63934 ceph-osd ceph txt unknown /proc/63574/task/63934/exe (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63934 ceph-osd ceph NOFD /proc/63574/task/63934/fd (opendir: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63958 ceph-osd ceph cwd unknown /proc/63574/task/63958/cwd (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63958 ceph-osd ceph rtd unknown /proc/63574/task/63958/root (readlink: Permission denied) 2024-04-22T13:30:08.633 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63958 ceph-osd ceph txt unknown /proc/63574/task/63958/exe (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63958 ceph-osd ceph NOFD /proc/63574/task/63958/fd (opendir: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63982 ceph-osd ceph cwd unknown /proc/63574/task/63982/cwd (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63982 ceph-osd ceph rtd unknown /proc/63574/task/63982/root (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63982 ceph-osd ceph txt unknown /proc/63574/task/63982/exe (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 63982 ceph-osd ceph NOFD /proc/63574/task/63982/fd (opendir: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64022 ceph-osd ceph cwd unknown /proc/63574/task/64022/cwd (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64022 ceph-osd ceph rtd unknown /proc/63574/task/64022/root (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64022 ceph-osd ceph txt unknown /proc/63574/task/64022/exe (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64022 ceph-osd ceph NOFD /proc/63574/task/64022/fd (opendir: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64023 ceph-osd ceph cwd unknown /proc/63574/task/64023/cwd (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64023 ceph-osd ceph rtd unknown /proc/63574/task/64023/root (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64023 ceph-osd ceph txt unknown /proc/63574/task/64023/exe (readlink: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64023 ceph-osd ceph NOFD /proc/63574/task/64023/fd (opendir: Permission denied) 2024-04-22T13:30:08.634 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64024 safe_time ceph cwd unknown /proc/63574/task/64024/cwd (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64024 safe_time ceph rtd unknown /proc/63574/task/64024/root (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64024 safe_time ceph txt unknown /proc/63574/task/64024/exe (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64024 safe_time ceph NOFD /proc/63574/task/64024/fd (opendir: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64025 safe_time ceph cwd unknown /proc/63574/task/64025/cwd (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64025 safe_time ceph rtd unknown /proc/63574/task/64025/root (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64025 safe_time ceph txt unknown /proc/63574/task/64025/exe (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64025 safe_time ceph NOFD /proc/63574/task/64025/fd (opendir: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64026 safe_time ceph cwd unknown /proc/63574/task/64026/cwd (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64026 safe_time ceph rtd unknown /proc/63574/task/64026/root (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64026 safe_time ceph txt unknown /proc/63574/task/64026/exe (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64026 safe_time ceph NOFD /proc/63574/task/64026/fd (opendir: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64027 safe_time ceph cwd unknown /proc/63574/task/64027/cwd (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64027 safe_time ceph rtd unknown /proc/63574/task/64027/root (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64027 safe_time ceph txt unknown /proc/63574/task/64027/exe (readlink: Permission denied) 2024-04-22T13:30:08.635 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64027 safe_time ceph NOFD /proc/63574/task/64027/fd (opendir: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64028 fn_anonym ceph cwd unknown /proc/63574/task/64028/cwd (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64028 fn_anonym ceph rtd unknown /proc/63574/task/64028/root (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64028 fn_anonym ceph txt unknown /proc/63574/task/64028/exe (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64028 fn_anonym ceph NOFD /proc/63574/task/64028/fd (opendir: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64032 rocksdb:l ceph cwd unknown /proc/63574/task/64032/cwd (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64032 rocksdb:l ceph rtd unknown /proc/63574/task/64032/root (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64032 rocksdb:l ceph txt unknown /proc/63574/task/64032/exe (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64032 rocksdb:l ceph NOFD /proc/63574/task/64032/fd (opendir: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64033 rocksdb:l ceph cwd unknown /proc/63574/task/64033/cwd (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64033 rocksdb:l ceph rtd unknown /proc/63574/task/64033/root (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64033 rocksdb:l ceph txt unknown /proc/63574/task/64033/exe (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64033 rocksdb:l ceph NOFD /proc/63574/task/64033/fd (opendir: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64034 rocksdb:l ceph cwd unknown /proc/63574/task/64034/cwd (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64034 rocksdb:l ceph rtd unknown /proc/63574/task/64034/root (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64034 rocksdb:l ceph txt unknown /proc/63574/task/64034/exe (readlink: Permission denied) 2024-04-22T13:30:08.636 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64034 rocksdb:l ceph NOFD /proc/63574/task/64034/fd (opendir: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64035 rocksdb:h ceph cwd unknown /proc/63574/task/64035/cwd (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64035 rocksdb:h ceph rtd unknown /proc/63574/task/64035/root (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64035 rocksdb:h ceph txt unknown /proc/63574/task/64035/exe (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64035 rocksdb:h ceph NOFD /proc/63574/task/64035/fd (opendir: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64516 bstore_ai ceph cwd unknown /proc/63574/task/64516/cwd (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64516 bstore_ai ceph rtd unknown /proc/63574/task/64516/root (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64516 bstore_ai ceph txt unknown /proc/63574/task/64516/exe (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64516 bstore_ai ceph NOFD /proc/63574/task/64516/fd (opendir: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64699 bstore_ai ceph cwd unknown /proc/63574/task/64699/cwd (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64699 bstore_ai ceph rtd unknown /proc/63574/task/64699/root (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64699 bstore_ai ceph txt unknown /proc/63574/task/64699/exe (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64699 bstore_ai ceph NOFD /proc/63574/task/64699/fd (opendir: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64881 ceph-osd ceph cwd unknown /proc/63574/task/64881/cwd (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64881 ceph-osd ceph rtd unknown /proc/63574/task/64881/root (readlink: Permission denied) 2024-04-22T13:30:08.637 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64881 ceph-osd ceph txt unknown /proc/63574/task/64881/exe (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64881 ceph-osd ceph NOFD /proc/63574/task/64881/fd (opendir: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64882 cfin ceph cwd unknown /proc/63574/task/64882/cwd (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64882 cfin ceph rtd unknown /proc/63574/task/64882/root (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64882 cfin ceph txt unknown /proc/63574/task/64882/exe (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64882 cfin ceph NOFD /proc/63574/task/64882/fd (opendir: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64883 bstore_kv ceph cwd unknown /proc/63574/task/64883/cwd (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64883 bstore_kv ceph rtd unknown /proc/63574/task/64883/root (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64883 bstore_kv ceph txt unknown /proc/63574/task/64883/exe (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64883 bstore_kv ceph NOFD /proc/63574/task/64883/fd (opendir: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64884 bstore_kv ceph cwd unknown /proc/63574/task/64884/cwd (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64884 bstore_kv ceph rtd unknown /proc/63574/task/64884/root (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64884 bstore_kv ceph txt unknown /proc/63574/task/64884/exe (readlink: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64884 bstore_kv ceph NOFD /proc/63574/task/64884/fd (opendir: Permission denied) 2024-04-22T13:30:08.638 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64885 bstore_me ceph cwd unknown /proc/63574/task/64885/cwd (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64885 bstore_me ceph rtd unknown /proc/63574/task/64885/root (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64885 bstore_me ceph txt unknown /proc/63574/task/64885/exe (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64885 bstore_me ceph NOFD /proc/63574/task/64885/fd (opendir: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64886 ms_dispat ceph cwd unknown /proc/63574/task/64886/cwd (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64886 ms_dispat ceph rtd unknown /proc/63574/task/64886/root (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64886 ms_dispat ceph txt unknown /proc/63574/task/64886/exe (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64886 ms_dispat ceph NOFD /proc/63574/task/64886/fd (opendir: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64887 ms_local ceph cwd unknown /proc/63574/task/64887/cwd (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64887 ms_local ceph rtd unknown /proc/63574/task/64887/root (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64887 ms_local ceph txt unknown /proc/63574/task/64887/exe (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64887 ms_local ceph NOFD /proc/63574/task/64887/fd (opendir: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64888 safe_time ceph cwd unknown /proc/63574/task/64888/cwd (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64888 safe_time ceph rtd unknown /proc/63574/task/64888/root (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64888 safe_time ceph txt unknown /proc/63574/task/64888/exe (readlink: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64888 safe_time ceph NOFD /proc/63574/task/64888/fd (opendir: Permission denied) 2024-04-22T13:30:08.639 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64889 safe_time ceph cwd unknown /proc/63574/task/64889/cwd (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64889 safe_time ceph rtd unknown /proc/63574/task/64889/root (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64889 safe_time ceph txt unknown /proc/63574/task/64889/exe (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64889 safe_time ceph NOFD /proc/63574/task/64889/fd (opendir: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64890 ms_dispat ceph cwd unknown /proc/63574/task/64890/cwd (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64890 ms_dispat ceph rtd unknown /proc/63574/task/64890/root (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64890 ms_dispat ceph txt unknown /proc/63574/task/64890/exe (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64890 ms_dispat ceph NOFD /proc/63574/task/64890/fd (opendir: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64891 ms_local ceph cwd unknown /proc/63574/task/64891/cwd (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64891 ms_local ceph rtd unknown /proc/63574/task/64891/root (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64891 ms_local ceph txt unknown /proc/63574/task/64891/exe (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64891 ms_local ceph NOFD /proc/63574/task/64891/fd (opendir: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64892 ms_dispat ceph cwd unknown /proc/63574/task/64892/cwd (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64892 ms_dispat ceph rtd unknown /proc/63574/task/64892/root (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64892 ms_dispat ceph txt unknown /proc/63574/task/64892/exe (readlink: Permission denied) 2024-04-22T13:30:08.640 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64892 ms_dispat ceph NOFD /proc/63574/task/64892/fd (opendir: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64893 ms_local ceph cwd unknown /proc/63574/task/64893/cwd (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64893 ms_local ceph rtd unknown /proc/63574/task/64893/root (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64893 ms_local ceph txt unknown /proc/63574/task/64893/exe (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64893 ms_local ceph NOFD /proc/63574/task/64893/fd (opendir: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64894 ms_dispat ceph cwd unknown /proc/63574/task/64894/cwd (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64894 ms_dispat ceph rtd unknown /proc/63574/task/64894/root (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64894 ms_dispat ceph txt unknown /proc/63574/task/64894/exe (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64894 ms_dispat ceph NOFD /proc/63574/task/64894/fd (opendir: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64895 ms_local ceph cwd unknown /proc/63574/task/64895/cwd (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64895 ms_local ceph rtd unknown /proc/63574/task/64895/root (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64895 ms_local ceph txt unknown /proc/63574/task/64895/exe (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64895 ms_local ceph NOFD /proc/63574/task/64895/fd (opendir: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64896 ms_dispat ceph cwd unknown /proc/63574/task/64896/cwd (readlink: Permission denied) 2024-04-22T13:30:08.641 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64896 ms_dispat ceph rtd unknown /proc/63574/task/64896/root (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64896 ms_dispat ceph txt unknown /proc/63574/task/64896/exe (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64896 ms_dispat ceph NOFD /proc/63574/task/64896/fd (opendir: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64897 ms_local ceph cwd unknown /proc/63574/task/64897/cwd (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64897 ms_local ceph rtd unknown /proc/63574/task/64897/root (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64897 ms_local ceph txt unknown /proc/63574/task/64897/exe (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64897 ms_local ceph NOFD /proc/63574/task/64897/fd (opendir: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64898 ms_dispat ceph cwd unknown /proc/63574/task/64898/cwd (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64898 ms_dispat ceph rtd unknown /proc/63574/task/64898/root (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64898 ms_dispat ceph txt unknown /proc/63574/task/64898/exe (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64898 ms_dispat ceph NOFD /proc/63574/task/64898/fd (opendir: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64899 ms_local ceph cwd unknown /proc/63574/task/64899/cwd (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64899 ms_local ceph rtd unknown /proc/63574/task/64899/root (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64899 ms_local ceph txt unknown /proc/63574/task/64899/exe (readlink: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64899 ms_local ceph NOFD /proc/63574/task/64899/fd (opendir: Permission denied) 2024-04-22T13:30:08.642 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64900 ms_dispat ceph cwd unknown /proc/63574/task/64900/cwd (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64900 ms_dispat ceph rtd unknown /proc/63574/task/64900/root (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64900 ms_dispat ceph txt unknown /proc/63574/task/64900/exe (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64900 ms_dispat ceph NOFD /proc/63574/task/64900/fd (opendir: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64901 ms_local ceph cwd unknown /proc/63574/task/64901/cwd (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64901 ms_local ceph rtd unknown /proc/63574/task/64901/root (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64901 ms_local ceph txt unknown /proc/63574/task/64901/exe (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64901 ms_local ceph NOFD /proc/63574/task/64901/fd (opendir: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64902 fn_anonym ceph cwd unknown /proc/63574/task/64902/cwd (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64902 fn_anonym ceph rtd unknown /proc/63574/task/64902/root (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64902 fn_anonym ceph txt unknown /proc/63574/task/64902/exe (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64902 fn_anonym ceph NOFD /proc/63574/task/64902/fd (opendir: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64903 finisher ceph cwd unknown /proc/63574/task/64903/cwd (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64903 finisher ceph rtd unknown /proc/63574/task/64903/root (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64903 finisher ceph txt unknown /proc/63574/task/64903/exe (readlink: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64903 finisher ceph NOFD /proc/63574/task/64903/fd (opendir: Permission denied) 2024-04-22T13:30:08.643 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64904 safe_time ceph cwd unknown /proc/63574/task/64904/cwd (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64904 safe_time ceph rtd unknown /proc/63574/task/64904/root (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64904 safe_time ceph txt unknown /proc/63574/task/64904/exe (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64904 safe_time ceph NOFD /proc/63574/task/64904/fd (opendir: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64905 safe_time ceph cwd unknown /proc/63574/task/64905/cwd (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64905 safe_time ceph rtd unknown /proc/63574/task/64905/root (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64905 safe_time ceph txt unknown /proc/63574/task/64905/exe (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64905 safe_time ceph NOFD /proc/63574/task/64905/fd (opendir: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64906 ceph-osd ceph cwd unknown /proc/63574/task/64906/cwd (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64906 ceph-osd ceph rtd unknown /proc/63574/task/64906/root (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64906 ceph-osd ceph txt unknown /proc/63574/task/64906/exe (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64906 ceph-osd ceph NOFD /proc/63574/task/64906/fd (opendir: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64907 osd_srv_a ceph cwd unknown /proc/63574/task/64907/cwd (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64907 osd_srv_a ceph rtd unknown /proc/63574/task/64907/root (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64907 osd_srv_a ceph txt unknown /proc/63574/task/64907/exe (readlink: Permission denied) 2024-04-22T13:30:08.644 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64907 osd_srv_a ceph NOFD /proc/63574/task/64907/fd (opendir: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64908 tp_osd_tp ceph cwd unknown /proc/63574/task/64908/cwd (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64908 tp_osd_tp ceph rtd unknown /proc/63574/task/64908/root (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64908 tp_osd_tp ceph txt unknown /proc/63574/task/64908/exe (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64908 tp_osd_tp ceph NOFD /proc/63574/task/64908/fd (opendir: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64909 tp_osd_tp ceph cwd unknown /proc/63574/task/64909/cwd (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64909 tp_osd_tp ceph rtd unknown /proc/63574/task/64909/root (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64909 tp_osd_tp ceph txt unknown /proc/63574/task/64909/exe (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64909 tp_osd_tp ceph NOFD /proc/63574/task/64909/fd (opendir: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64910 tp_osd_tp ceph cwd unknown /proc/63574/task/64910/cwd (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64910 tp_osd_tp ceph rtd unknown /proc/63574/task/64910/root (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64910 tp_osd_tp ceph txt unknown /proc/63574/task/64910/exe (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64910 tp_osd_tp ceph NOFD /proc/63574/task/64910/fd (opendir: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64911 tp_osd_tp ceph cwd unknown /proc/63574/task/64911/cwd (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64911 tp_osd_tp ceph rtd unknown /proc/63574/task/64911/root (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64911 tp_osd_tp ceph txt unknown /proc/63574/task/64911/exe (readlink: Permission denied) 2024-04-22T13:30:08.645 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64911 tp_osd_tp ceph NOFD /proc/63574/task/64911/fd (opendir: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64912 tp_osd_tp ceph cwd unknown /proc/63574/task/64912/cwd (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64912 tp_osd_tp ceph rtd unknown /proc/63574/task/64912/root (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64912 tp_osd_tp ceph txt unknown /proc/63574/task/64912/exe (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64912 tp_osd_tp ceph NOFD /proc/63574/task/64912/fd (opendir: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64913 tp_osd_tp ceph cwd unknown /proc/63574/task/64913/cwd (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64913 tp_osd_tp ceph rtd unknown /proc/63574/task/64913/root (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64913 tp_osd_tp ceph txt unknown /proc/63574/task/64913/exe (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64913 tp_osd_tp ceph NOFD /proc/63574/task/64913/fd (opendir: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64914 tp_osd_tp ceph cwd unknown /proc/63574/task/64914/cwd (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64914 tp_osd_tp ceph rtd unknown /proc/63574/task/64914/root (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64914 tp_osd_tp ceph txt unknown /proc/63574/task/64914/exe (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64914 tp_osd_tp ceph NOFD /proc/63574/task/64914/fd (opendir: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64915 tp_osd_tp ceph cwd unknown /proc/63574/task/64915/cwd (readlink: Permission denied) 2024-04-22T13:30:08.646 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64915 tp_osd_tp ceph rtd unknown /proc/63574/task/64915/root (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64915 tp_osd_tp ceph txt unknown /proc/63574/task/64915/exe (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64915 tp_osd_tp ceph NOFD /proc/63574/task/64915/fd (opendir: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64916 tp_osd_tp ceph cwd unknown /proc/63574/task/64916/cwd (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64916 tp_osd_tp ceph rtd unknown /proc/63574/task/64916/root (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64916 tp_osd_tp ceph txt unknown /proc/63574/task/64916/exe (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64916 tp_osd_tp ceph NOFD /proc/63574/task/64916/fd (opendir: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64917 tp_osd_tp ceph cwd unknown /proc/63574/task/64917/cwd (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64917 tp_osd_tp ceph rtd unknown /proc/63574/task/64917/root (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64917 tp_osd_tp ceph txt unknown /proc/63574/task/64917/exe (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64917 tp_osd_tp ceph NOFD /proc/63574/task/64917/fd (opendir: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64918 tp_osd_tp ceph cwd unknown /proc/63574/task/64918/cwd (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64918 tp_osd_tp ceph rtd unknown /proc/63574/task/64918/root (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64918 tp_osd_tp ceph txt unknown /proc/63574/task/64918/exe (readlink: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64918 tp_osd_tp ceph NOFD /proc/63574/task/64918/fd (opendir: Permission denied) 2024-04-22T13:30:08.647 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64919 tp_osd_tp ceph cwd unknown /proc/63574/task/64919/cwd (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64919 tp_osd_tp ceph rtd unknown /proc/63574/task/64919/root (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64919 tp_osd_tp ceph txt unknown /proc/63574/task/64919/exe (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64919 tp_osd_tp ceph NOFD /proc/63574/task/64919/fd (opendir: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64920 tp_osd_tp ceph cwd unknown /proc/63574/task/64920/cwd (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64920 tp_osd_tp ceph rtd unknown /proc/63574/task/64920/root (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64920 tp_osd_tp ceph txt unknown /proc/63574/task/64920/exe (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64920 tp_osd_tp ceph NOFD /proc/63574/task/64920/fd (opendir: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64921 tp_osd_tp ceph cwd unknown /proc/63574/task/64921/cwd (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64921 tp_osd_tp ceph rtd unknown /proc/63574/task/64921/root (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64921 tp_osd_tp ceph txt unknown /proc/63574/task/64921/exe (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64921 tp_osd_tp ceph NOFD /proc/63574/task/64921/fd (opendir: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64922 tp_osd_tp ceph cwd unknown /proc/63574/task/64922/cwd (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64922 tp_osd_tp ceph rtd unknown /proc/63574/task/64922/root (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64922 tp_osd_tp ceph txt unknown /proc/63574/task/64922/exe (readlink: Permission denied) 2024-04-22T13:30:08.648 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64922 tp_osd_tp ceph NOFD /proc/63574/task/64922/fd (opendir: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64923 tp_osd_tp ceph cwd unknown /proc/63574/task/64923/cwd (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64923 tp_osd_tp ceph rtd unknown /proc/63574/task/64923/root (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64923 tp_osd_tp ceph txt unknown /proc/63574/task/64923/exe (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64923 tp_osd_tp ceph NOFD /proc/63574/task/64923/fd (opendir: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64924 osd_srv_h ceph cwd unknown /proc/63574/task/64924/cwd (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64924 osd_srv_h ceph rtd unknown /proc/63574/task/64924/root (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64924 osd_srv_h ceph txt unknown /proc/63574/task/64924/exe (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-osd 63574 64924 osd_srv_h ceph NOFD /proc/63574/task/64924/fd (opendir: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 63881 root cwd unknown /proc/63881/cwd (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 63881 root rtd unknown /proc/63881/root (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 63881 root txt unknown /proc/63881/exe (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 63881 root NOFD /proc/63881/fd (opendir: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 65019 root cwd unknown /proc/65019/cwd (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 65019 root rtd unknown /proc/65019/root (readlink: Permission denied) 2024-04-22T13:30:08.649 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 65019 root txt unknown /proc/65019/exe (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 65019 root NOFD /proc/65019/fd (opendir: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 65036 root cwd unknown /proc/65036/cwd (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 65036 root rtd unknown /proc/65036/root (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 65036 root txt unknown /proc/65036/exe (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 65036 root NOFD /proc/65036/fd (opendir: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 67786 root cwd unknown /proc/67786/cwd (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 67786 root rtd unknown /proc/67786/root (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 67786 root txt unknown /proc/67786/exe (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 67786 root NOFD /proc/67786/fd (opendir: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 68399 root cwd unknown /proc/68399/cwd (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 68399 root rtd unknown /proc/68399/root (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 68399 root txt unknown /proc/68399/exe (readlink: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/3 68399 root NOFD /proc/68399/fd (opendir: Permission denied) 2024-04-22T13:30:08.650 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68465 root cwd unknown /proc/68465/cwd (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68465 root rtd unknown /proc/68465/root (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68465 root txt unknown /proc/68465/exe (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68465 root NOFD /proc/68465/fd (opendir: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68482 root cwd unknown /proc/68482/cwd (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68482 root rtd unknown /proc/68482/root (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68482 root txt unknown /proc/68482/exe (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68482 root NOFD /proc/68482/fd (opendir: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68483 root cwd unknown /proc/68483/cwd (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68483 root rtd unknown /proc/68483/root (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68483 root txt unknown /proc/68483/exe (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68483 root NOFD /proc/68483/fd (opendir: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68500 root cwd unknown /proc/68500/cwd (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68500 root rtd unknown /proc/68500/root (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68500 root txt unknown /proc/68500/exe (readlink: Permission denied) 2024-04-22T13:30:08.651 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68500 root NOFD /proc/68500/fd (opendir: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68501 root cwd unknown /proc/68501/cwd (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68501 root rtd unknown /proc/68501/root (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68501 root txt unknown /proc/68501/exe (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68501 root NOFD /proc/68501/fd (opendir: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68518 root cwd unknown /proc/68518/cwd (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68518 root rtd unknown /proc/68518/root (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68518 root txt unknown /proc/68518/exe (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68518 root NOFD /proc/68518/fd (opendir: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68519 root cwd unknown /proc/68519/cwd (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68519 root rtd unknown /proc/68519/root (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68519 root txt unknown /proc/68519/exe (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 68519 root NOFD /proc/68519/fd (opendir: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68536 root cwd unknown /proc/68536/cwd (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68536 root rtd unknown /proc/68536/root (readlink: Permission denied) 2024-04-22T13:30:08.652 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68536 root txt unknown /proc/68536/exe (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:journalct 68536 root NOFD /proc/68536/fd (opendir: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 68660 root cwd unknown /proc/68660/cwd (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 68660 root rtd unknown /proc/68660/root (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 68660 root txt unknown /proc/68660/exe (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/7 68660 root NOFD /proc/68660/fd (opendir: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69076 root cwd unknown /proc/69076/cwd (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69076 root rtd unknown /proc/69076/root (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69076 root txt unknown /proc/69076/exe (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69076 root NOFD /proc/69076/fd (opendir: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69078 root cwd unknown /proc/69078/cwd (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69078 root rtd unknown /proc/69078/root (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69078 root txt unknown /proc/69078/exe (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69078 root NOFD /proc/69078/fd (opendir: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 ceph cwd unknown /proc/69080/cwd (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 ceph rtd unknown /proc/69080/root (readlink: Permission denied) 2024-04-22T13:30:08.653 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 ceph txt unknown /proc/69080/exe (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 ceph NOFD /proc/69080/fd (opendir: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69081 log ceph cwd unknown /proc/69080/task/69081/cwd (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69081 log ceph rtd unknown /proc/69080/task/69081/root (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69081 log ceph txt unknown /proc/69080/task/69081/exe (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69081 log ceph NOFD /proc/69080/task/69081/fd (opendir: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69083 msgr-work ceph cwd unknown /proc/69080/task/69083/cwd (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69083 msgr-work ceph rtd unknown /proc/69080/task/69083/root (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69083 msgr-work ceph txt unknown /proc/69080/task/69083/exe (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69083 msgr-work ceph NOFD /proc/69080/task/69083/fd (opendir: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69084 msgr-work ceph cwd unknown /proc/69080/task/69084/cwd (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69084 msgr-work ceph rtd unknown /proc/69080/task/69084/root (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69084 msgr-work ceph txt unknown /proc/69080/task/69084/exe (readlink: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69084 msgr-work ceph NOFD /proc/69080/task/69084/fd (opendir: Permission denied) 2024-04-22T13:30:08.654 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69085 msgr-work ceph cwd unknown /proc/69080/task/69085/cwd (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69085 msgr-work ceph rtd unknown /proc/69080/task/69085/root (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69085 msgr-work ceph txt unknown /proc/69080/task/69085/exe (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69085 msgr-work ceph NOFD /proc/69080/task/69085/fd (opendir: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69089 service ceph cwd unknown /proc/69080/task/69089/cwd (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69089 service ceph rtd unknown /proc/69080/task/69089/root (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69089 service ceph txt unknown /proc/69080/task/69089/exe (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69089 service ceph NOFD /proc/69080/task/69089/fd (opendir: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69090 admin_soc ceph cwd unknown /proc/69080/task/69090/cwd (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69090 admin_soc ceph rtd unknown /proc/69080/task/69090/root (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69090 admin_soc ceph txt unknown /proc/69080/task/69090/exe (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69090 admin_soc ceph NOFD /proc/69080/task/69090/fd (opendir: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69091 signal_ha ceph cwd unknown /proc/69080/task/69091/cwd (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69091 signal_ha ceph rtd unknown /proc/69080/task/69091/root (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69091 signal_ha ceph txt unknown /proc/69080/task/69091/exe (readlink: Permission denied) 2024-04-22T13:30:08.655 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69091 signal_ha ceph NOFD /proc/69080/task/69091/fd (opendir: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69092 io_contex ceph cwd unknown /proc/69080/task/69092/cwd (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69092 io_contex ceph rtd unknown /proc/69080/task/69092/root (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69092 io_contex ceph txt unknown /proc/69080/task/69092/exe (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69092 io_contex ceph NOFD /proc/69080/task/69092/fd (opendir: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69093 io_contex ceph cwd unknown /proc/69080/task/69093/cwd (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69093 io_contex ceph rtd unknown /proc/69080/task/69093/root (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69093 io_contex ceph txt unknown /proc/69080/task/69093/exe (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69093 io_contex ceph NOFD /proc/69080/task/69093/fd (opendir: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69094 ms_dispat ceph cwd unknown /proc/69080/task/69094/cwd (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69094 ms_dispat ceph rtd unknown /proc/69080/task/69094/root (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69094 ms_dispat ceph txt unknown /proc/69080/task/69094/exe (readlink: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69094 ms_dispat ceph NOFD /proc/69080/task/69094/fd (opendir: Permission denied) 2024-04-22T13:30:08.656 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69095 ms_local ceph cwd unknown /proc/69080/task/69095/cwd (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69095 ms_local ceph rtd unknown /proc/69080/task/69095/root (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69095 ms_local ceph txt unknown /proc/69080/task/69095/exe (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69095 ms_local ceph NOFD /proc/69080/task/69095/fd (opendir: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69096 safe_time ceph cwd unknown /proc/69080/task/69096/cwd (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69096 safe_time ceph rtd unknown /proc/69080/task/69096/root (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69096 safe_time ceph txt unknown /proc/69080/task/69096/exe (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69096 safe_time ceph NOFD /proc/69080/task/69096/fd (opendir: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69097 safe_time ceph cwd unknown /proc/69080/task/69097/cwd (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69097 safe_time ceph rtd unknown /proc/69080/task/69097/root (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69097 safe_time ceph txt unknown /proc/69080/task/69097/exe (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69097 safe_time ceph NOFD /proc/69080/task/69097/fd (opendir: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69098 ceph-mds ceph cwd unknown /proc/69080/task/69098/cwd (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69098 ceph-mds ceph rtd unknown /proc/69080/task/69098/root (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69098 ceph-mds ceph txt unknown /proc/69080/task/69098/exe (readlink: Permission denied) 2024-04-22T13:30:08.657 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69098 ceph-mds ceph NOFD /proc/69080/task/69098/fd (opendir: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69171 safe_time ceph cwd unknown /proc/69080/task/69171/cwd (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69171 safe_time ceph rtd unknown /proc/69080/task/69171/root (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69171 safe_time ceph txt unknown /proc/69080/task/69171/exe (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69080 69171 safe_time ceph NOFD /proc/69080/task/69171/fd (opendir: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69425 root cwd unknown /proc/69425/cwd (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69425 root rtd unknown /proc/69425/root (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69425 root txt unknown /proc/69425/exe (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 69425 root NOFD /proc/69425/fd (opendir: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69427 root cwd unknown /proc/69427/cwd (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69427 root rtd unknown /proc/69427/root (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69427 root txt unknown /proc/69427/exe (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 69427 root NOFD /proc/69427/fd (opendir: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 ceph cwd unknown /proc/69429/cwd (readlink: Permission denied) 2024-04-22T13:30:08.658 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 ceph rtd unknown /proc/69429/root (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 ceph txt unknown /proc/69429/exe (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 ceph NOFD /proc/69429/fd (opendir: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69430 log ceph cwd unknown /proc/69429/task/69430/cwd (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69430 log ceph rtd unknown /proc/69429/task/69430/root (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69430 log ceph txt unknown /proc/69429/task/69430/exe (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69430 log ceph NOFD /proc/69429/task/69430/fd (opendir: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69432 msgr-work ceph cwd unknown /proc/69429/task/69432/cwd (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69432 msgr-work ceph rtd unknown /proc/69429/task/69432/root (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69432 msgr-work ceph txt unknown /proc/69429/task/69432/exe (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69432 msgr-work ceph NOFD /proc/69429/task/69432/fd (opendir: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69433 msgr-work ceph cwd unknown /proc/69429/task/69433/cwd (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69433 msgr-work ceph rtd unknown /proc/69429/task/69433/root (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69433 msgr-work ceph txt unknown /proc/69429/task/69433/exe (readlink: Permission denied) 2024-04-22T13:30:08.659 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69433 msgr-work ceph NOFD /proc/69429/task/69433/fd (opendir: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69434 msgr-work ceph cwd unknown /proc/69429/task/69434/cwd (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69434 msgr-work ceph rtd unknown /proc/69429/task/69434/root (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69434 msgr-work ceph txt unknown /proc/69429/task/69434/exe (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69434 msgr-work ceph NOFD /proc/69429/task/69434/fd (opendir: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69438 service ceph cwd unknown /proc/69429/task/69438/cwd (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69438 service ceph rtd unknown /proc/69429/task/69438/root (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69438 service ceph txt unknown /proc/69429/task/69438/exe (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69438 service ceph NOFD /proc/69429/task/69438/fd (opendir: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69439 admin_soc ceph cwd unknown /proc/69429/task/69439/cwd (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69439 admin_soc ceph rtd unknown /proc/69429/task/69439/root (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69439 admin_soc ceph txt unknown /proc/69429/task/69439/exe (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69439 admin_soc ceph NOFD /proc/69429/task/69439/fd (opendir: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69440 signal_ha ceph cwd unknown /proc/69429/task/69440/cwd (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69440 signal_ha ceph rtd unknown /proc/69429/task/69440/root (readlink: Permission denied) 2024-04-22T13:30:08.660 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69440 signal_ha ceph txt unknown /proc/69429/task/69440/exe (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69440 signal_ha ceph NOFD /proc/69429/task/69440/fd (opendir: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69441 io_contex ceph cwd unknown /proc/69429/task/69441/cwd (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69441 io_contex ceph rtd unknown /proc/69429/task/69441/root (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69441 io_contex ceph txt unknown /proc/69429/task/69441/exe (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69441 io_contex ceph NOFD /proc/69429/task/69441/fd (opendir: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69442 io_contex ceph cwd unknown /proc/69429/task/69442/cwd (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69442 io_contex ceph rtd unknown /proc/69429/task/69442/root (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69442 io_contex ceph txt unknown /proc/69429/task/69442/exe (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69442 io_contex ceph NOFD /proc/69429/task/69442/fd (opendir: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69443 ms_dispat ceph cwd unknown /proc/69429/task/69443/cwd (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69443 ms_dispat ceph rtd unknown /proc/69429/task/69443/root (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69443 ms_dispat ceph txt unknown /proc/69429/task/69443/exe (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69443 ms_dispat ceph NOFD /proc/69429/task/69443/fd (opendir: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69444 ms_local ceph cwd unknown /proc/69429/task/69444/cwd (readlink: Permission denied) 2024-04-22T13:30:08.661 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69444 ms_local ceph rtd unknown /proc/69429/task/69444/root (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69444 ms_local ceph txt unknown /proc/69429/task/69444/exe (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69444 ms_local ceph NOFD /proc/69429/task/69444/fd (opendir: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69445 safe_time ceph cwd unknown /proc/69429/task/69445/cwd (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69445 safe_time ceph rtd unknown /proc/69429/task/69445/root (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69445 safe_time ceph txt unknown /proc/69429/task/69445/exe (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69445 safe_time ceph NOFD /proc/69429/task/69445/fd (opendir: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69446 safe_time ceph cwd unknown /proc/69429/task/69446/cwd (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69446 safe_time ceph rtd unknown /proc/69429/task/69446/root (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69446 safe_time ceph txt unknown /proc/69429/task/69446/exe (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69446 safe_time ceph NOFD /proc/69429/task/69446/fd (opendir: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69447 ceph-mds ceph cwd unknown /proc/69429/task/69447/cwd (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69447 ceph-mds ceph rtd unknown /proc/69429/task/69447/root (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69447 ceph-mds ceph txt unknown /proc/69429/task/69447/exe (readlink: Permission denied) 2024-04-22T13:30:08.662 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69447 ceph-mds ceph NOFD /proc/69429/task/69447/fd (opendir: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69452 safe_time ceph cwd unknown /proc/69429/task/69452/cwd (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69452 safe_time ceph rtd unknown /proc/69429/task/69452/root (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69452 safe_time ceph txt unknown /proc/69429/task/69452/exe (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 69452 safe_time ceph NOFD /proc/69429/task/69452/fd (opendir: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71849 ceph_time ceph cwd unknown /proc/69429/task/71849/cwd (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71849 ceph_time ceph rtd unknown /proc/69429/task/71849/root (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71849 ceph_time ceph txt unknown /proc/69429/task/71849/exe (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71849 ceph_time ceph NOFD /proc/69429/task/71849/fd (opendir: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71850 OpHistory ceph cwd unknown /proc/69429/task/71850/cwd (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71850 OpHistory ceph rtd unknown /proc/69429/task/71850/root (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71850 OpHistory ceph txt unknown /proc/69429/task/71850/exe (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71850 OpHistory ceph NOFD /proc/69429/task/71850/fd (opendir: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71854 ms_dispat ceph cwd unknown /proc/69429/task/71854/cwd (readlink: Permission denied) 2024-04-22T13:30:08.663 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71854 ms_dispat ceph rtd unknown /proc/69429/task/71854/root (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71854 ms_dispat ceph txt unknown /proc/69429/task/71854/exe (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71854 ms_dispat ceph NOFD /proc/69429/task/71854/fd (opendir: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71857 ms_dispat ceph cwd unknown /proc/69429/task/71857/cwd (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71857 ms_dispat ceph rtd unknown /proc/69429/task/71857/root (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71857 ms_dispat ceph txt unknown /proc/69429/task/71857/exe (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71857 ms_dispat ceph NOFD /proc/69429/task/71857/fd (opendir: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71861 mds_rank_ ceph cwd unknown /proc/69429/task/71861/cwd (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71861 mds_rank_ ceph rtd unknown /proc/69429/task/71861/root (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71861 mds_rank_ ceph txt unknown /proc/69429/task/71861/exe (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71861 mds_rank_ ceph NOFD /proc/69429/task/71861/fd (opendir: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71863 PQ_Finish ceph cwd unknown /proc/69429/task/71863/cwd (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71863 PQ_Finish ceph rtd unknown /proc/69429/task/71863/root (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71863 PQ_Finish ceph txt unknown /proc/69429/task/71863/exe (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71863 PQ_Finish ceph NOFD /proc/69429/task/71863/fd (opendir: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71865 safe_time ceph cwd unknown /proc/69429/task/71865/cwd (readlink: Permission denied) 2024-04-22T13:30:08.664 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71865 safe_time ceph rtd unknown /proc/69429/task/71865/root (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71865 safe_time ceph txt unknown /proc/69429/task/71865/exe (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71865 safe_time ceph NOFD /proc/69429/task/71865/fd (opendir: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71867 MR_Finish ceph cwd unknown /proc/69429/task/71867/cwd (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71867 MR_Finish ceph rtd unknown /proc/69429/task/71867/root (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71867 MR_Finish ceph txt unknown /proc/69429/task/71867/exe (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71867 MR_Finish ceph NOFD /proc/69429/task/71867/fd (opendir: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71875 md_submit ceph cwd unknown /proc/69429/task/71875/cwd (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71875 md_submit ceph rtd unknown /proc/69429/task/71875/root (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71875 md_submit ceph txt unknown /proc/69429/task/71875/exe (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 69429 71875 md_submit ceph NOFD /proc/69429/task/71875/fd (opendir: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 69535 root cwd unknown /proc/69535/cwd (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 69535 root rtd unknown /proc/69535/root (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 69535 root txt unknown /proc/69535/exe (readlink: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/u 69535 root NOFD /proc/69535/fd (opendir: Permission denied) 2024-04-22T13:30:08.665 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70022 root cwd unknown /proc/70022/cwd (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70022 root rtd unknown /proc/70022/root (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70022 root txt unknown /proc/70022/exe (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70022 root NOFD /proc/70022/fd (opendir: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70024 root cwd unknown /proc/70024/cwd (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70024 root rtd unknown /proc/70024/root (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70024 root txt unknown /proc/70024/exe (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70024 root NOFD /proc/70024/fd (opendir: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 ceph cwd unknown /proc/70026/cwd (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 ceph rtd unknown /proc/70026/root (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 ceph txt unknown /proc/70026/exe (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 ceph NOFD /proc/70026/fd (opendir: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70027 log ceph cwd unknown /proc/70026/task/70027/cwd (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70027 log ceph rtd unknown /proc/70026/task/70027/root (readlink: Permission denied) 2024-04-22T13:30:08.666 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70027 log ceph txt unknown /proc/70026/task/70027/exe (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70027 log ceph NOFD /proc/70026/task/70027/fd (opendir: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70029 msgr-work ceph cwd unknown /proc/70026/task/70029/cwd (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70029 msgr-work ceph rtd unknown /proc/70026/task/70029/root (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70029 msgr-work ceph txt unknown /proc/70026/task/70029/exe (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70029 msgr-work ceph NOFD /proc/70026/task/70029/fd (opendir: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70030 msgr-work ceph cwd unknown /proc/70026/task/70030/cwd (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70030 msgr-work ceph rtd unknown /proc/70026/task/70030/root (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70030 msgr-work ceph txt unknown /proc/70026/task/70030/exe (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70030 msgr-work ceph NOFD /proc/70026/task/70030/fd (opendir: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70031 msgr-work ceph cwd unknown /proc/70026/task/70031/cwd (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70031 msgr-work ceph rtd unknown /proc/70026/task/70031/root (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70031 msgr-work ceph txt unknown /proc/70026/task/70031/exe (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70031 msgr-work ceph NOFD /proc/70026/task/70031/fd (opendir: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70035 service ceph cwd unknown /proc/70026/task/70035/cwd (readlink: Permission denied) 2024-04-22T13:30:08.667 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70035 service ceph rtd unknown /proc/70026/task/70035/root (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70035 service ceph txt unknown /proc/70026/task/70035/exe (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70035 service ceph NOFD /proc/70026/task/70035/fd (opendir: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70036 admin_soc ceph cwd unknown /proc/70026/task/70036/cwd (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70036 admin_soc ceph rtd unknown /proc/70026/task/70036/root (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70036 admin_soc ceph txt unknown /proc/70026/task/70036/exe (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70036 admin_soc ceph NOFD /proc/70026/task/70036/fd (opendir: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70037 signal_ha ceph cwd unknown /proc/70026/task/70037/cwd (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70037 signal_ha ceph rtd unknown /proc/70026/task/70037/root (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70037 signal_ha ceph txt unknown /proc/70026/task/70037/exe (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70037 signal_ha ceph NOFD /proc/70026/task/70037/fd (opendir: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70038 io_contex ceph cwd unknown /proc/70026/task/70038/cwd (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70038 io_contex ceph rtd unknown /proc/70026/task/70038/root (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70038 io_contex ceph txt unknown /proc/70026/task/70038/exe (readlink: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70038 io_contex ceph NOFD /proc/70026/task/70038/fd (opendir: Permission denied) 2024-04-22T13:30:08.668 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70039 io_contex ceph cwd unknown /proc/70026/task/70039/cwd (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70039 io_contex ceph rtd unknown /proc/70026/task/70039/root (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70039 io_contex ceph txt unknown /proc/70026/task/70039/exe (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70039 io_contex ceph NOFD /proc/70026/task/70039/fd (opendir: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70040 ms_dispat ceph cwd unknown /proc/70026/task/70040/cwd (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70040 ms_dispat ceph rtd unknown /proc/70026/task/70040/root (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70040 ms_dispat ceph txt unknown /proc/70026/task/70040/exe (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70040 ms_dispat ceph NOFD /proc/70026/task/70040/fd (opendir: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70041 ms_local ceph cwd unknown /proc/70026/task/70041/cwd (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70041 ms_local ceph rtd unknown /proc/70026/task/70041/root (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70041 ms_local ceph txt unknown /proc/70026/task/70041/exe (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70041 ms_local ceph NOFD /proc/70026/task/70041/fd (opendir: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70042 safe_time ceph cwd unknown /proc/70026/task/70042/cwd (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70042 safe_time ceph rtd unknown /proc/70026/task/70042/root (readlink: Permission denied) 2024-04-22T13:30:08.669 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70042 safe_time ceph txt unknown /proc/70026/task/70042/exe (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70042 safe_time ceph NOFD /proc/70026/task/70042/fd (opendir: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70043 safe_time ceph cwd unknown /proc/70026/task/70043/cwd (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70043 safe_time ceph rtd unknown /proc/70026/task/70043/root (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70043 safe_time ceph txt unknown /proc/70026/task/70043/exe (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70043 safe_time ceph NOFD /proc/70026/task/70043/fd (opendir: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70044 ceph-mds ceph cwd unknown /proc/70026/task/70044/cwd (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70044 ceph-mds ceph rtd unknown /proc/70026/task/70044/root (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70044 ceph-mds ceph txt unknown /proc/70026/task/70044/exe (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70044 ceph-mds ceph NOFD /proc/70026/task/70044/fd (opendir: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70126 safe_time ceph cwd unknown /proc/70026/task/70126/cwd (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70126 safe_time ceph rtd unknown /proc/70026/task/70126/root (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70126 safe_time ceph txt unknown /proc/70026/task/70126/exe (readlink: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 70126 safe_time ceph NOFD /proc/70026/task/70126/fd (opendir: Permission denied) 2024-04-22T13:30:08.670 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71848 ceph_time ceph cwd unknown /proc/70026/task/71848/cwd (readlink: Permission denied) 2024-04-22T13:30:08.671 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71848 ceph_time ceph rtd unknown /proc/70026/task/71848/root (readlink: Permission denied) 2024-04-22T13:30:08.671 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71848 ceph_time ceph txt unknown /proc/70026/task/71848/exe (readlink: Permission denied) 2024-04-22T13:30:08.671 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71848 ceph_time ceph NOFD /proc/70026/task/71848/fd (opendir: Permission denied) 2024-04-22T13:30:08.671 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71851 OpHistory ceph cwd unknown /proc/70026/task/71851/cwd (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71851 OpHistory ceph rtd unknown /proc/70026/task/71851/root (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71851 OpHistory ceph txt unknown /proc/70026/task/71851/exe (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71851 OpHistory ceph NOFD /proc/70026/task/71851/fd (opendir: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71855 ms_dispat ceph cwd unknown /proc/70026/task/71855/cwd (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71855 ms_dispat ceph rtd unknown /proc/70026/task/71855/root (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71855 ms_dispat ceph txt unknown /proc/70026/task/71855/exe (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71855 ms_dispat ceph NOFD /proc/70026/task/71855/fd (opendir: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71856 ms_dispat ceph cwd unknown /proc/70026/task/71856/cwd (readlink: Permission denied) 2024-04-22T13:30:08.673 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71856 ms_dispat ceph rtd unknown /proc/70026/task/71856/root (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71856 ms_dispat ceph txt unknown /proc/70026/task/71856/exe (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71856 ms_dispat ceph NOFD /proc/70026/task/71856/fd (opendir: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71860 mds_rank_ ceph cwd unknown /proc/70026/task/71860/cwd (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71860 mds_rank_ ceph rtd unknown /proc/70026/task/71860/root (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71860 mds_rank_ ceph txt unknown /proc/70026/task/71860/exe (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71860 mds_rank_ ceph NOFD /proc/70026/task/71860/fd (opendir: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71862 PQ_Finish ceph cwd unknown /proc/70026/task/71862/cwd (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71862 PQ_Finish ceph rtd unknown /proc/70026/task/71862/root (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71862 PQ_Finish ceph txt unknown /proc/70026/task/71862/exe (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71862 PQ_Finish ceph NOFD /proc/70026/task/71862/fd (opendir: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71864 safe_time ceph cwd unknown /proc/70026/task/71864/cwd (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71864 safe_time ceph rtd unknown /proc/70026/task/71864/root (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71864 safe_time ceph txt unknown /proc/70026/task/71864/exe (readlink: Permission denied) 2024-04-22T13:30:08.674 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71864 safe_time ceph NOFD /proc/70026/task/71864/fd (opendir: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71866 MR_Finish ceph cwd unknown /proc/70026/task/71866/cwd (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71866 MR_Finish ceph rtd unknown /proc/70026/task/71866/root (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71866 MR_Finish ceph txt unknown /proc/70026/task/71866/exe (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71866 MR_Finish ceph NOFD /proc/70026/task/71866/fd (opendir: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71874 md_submit ceph cwd unknown /proc/70026/task/71874/cwd (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71874 md_submit ceph rtd unknown /proc/70026/task/71874/root (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71874 md_submit ceph txt unknown /proc/70026/task/71874/exe (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70026 71874 md_submit ceph NOFD /proc/70026/task/71874/fd (opendir: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70379 root cwd unknown /proc/70379/cwd (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70379 root rtd unknown /proc/70379/root (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70379 root txt unknown /proc/70379/exe (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:conmon 70379 root NOFD /proc/70379/fd (opendir: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70381 root cwd unknown /proc/70381/cwd (readlink: Permission denied) 2024-04-22T13:30:08.675 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70381 root rtd unknown /proc/70381/root (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70381 root txt unknown /proc/70381/exe (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:podman-in 70381 root NOFD /proc/70381/fd (opendir: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 ceph cwd unknown /proc/70383/cwd (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 ceph rtd unknown /proc/70383/root (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 ceph txt unknown /proc/70383/exe (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 ceph NOFD /proc/70383/fd (opendir: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70384 log ceph cwd unknown /proc/70383/task/70384/cwd (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70384 log ceph rtd unknown /proc/70383/task/70384/root (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70384 log ceph txt unknown /proc/70383/task/70384/exe (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70384 log ceph NOFD /proc/70383/task/70384/fd (opendir: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70386 msgr-work ceph cwd unknown /proc/70383/task/70386/cwd (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70386 msgr-work ceph rtd unknown /proc/70383/task/70386/root (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70386 msgr-work ceph txt unknown /proc/70383/task/70386/exe (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70386 msgr-work ceph NOFD /proc/70383/task/70386/fd (opendir: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70387 msgr-work ceph cwd unknown /proc/70383/task/70387/cwd (readlink: Permission denied) 2024-04-22T13:30:08.676 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70387 msgr-work ceph rtd unknown /proc/70383/task/70387/root (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70387 msgr-work ceph txt unknown /proc/70383/task/70387/exe (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70387 msgr-work ceph NOFD /proc/70383/task/70387/fd (opendir: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70388 msgr-work ceph cwd unknown /proc/70383/task/70388/cwd (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70388 msgr-work ceph rtd unknown /proc/70383/task/70388/root (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70388 msgr-work ceph txt unknown /proc/70383/task/70388/exe (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70388 msgr-work ceph NOFD /proc/70383/task/70388/fd (opendir: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70392 service ceph cwd unknown /proc/70383/task/70392/cwd (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70392 service ceph rtd unknown /proc/70383/task/70392/root (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70392 service ceph txt unknown /proc/70383/task/70392/exe (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70392 service ceph NOFD /proc/70383/task/70392/fd (opendir: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70393 admin_soc ceph cwd unknown /proc/70383/task/70393/cwd (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70393 admin_soc ceph rtd unknown /proc/70383/task/70393/root (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70393 admin_soc ceph txt unknown /proc/70383/task/70393/exe (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70393 admin_soc ceph NOFD /proc/70383/task/70393/fd (opendir: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70394 signal_ha ceph cwd unknown /proc/70383/task/70394/cwd (readlink: Permission denied) 2024-04-22T13:30:08.677 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70394 signal_ha ceph rtd unknown /proc/70383/task/70394/root (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70394 signal_ha ceph txt unknown /proc/70383/task/70394/exe (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70394 signal_ha ceph NOFD /proc/70383/task/70394/fd (opendir: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70395 io_contex ceph cwd unknown /proc/70383/task/70395/cwd (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70395 io_contex ceph rtd unknown /proc/70383/task/70395/root (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70395 io_contex ceph txt unknown /proc/70383/task/70395/exe (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70395 io_contex ceph NOFD /proc/70383/task/70395/fd (opendir: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70396 io_contex ceph cwd unknown /proc/70383/task/70396/cwd (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70396 io_contex ceph rtd unknown /proc/70383/task/70396/root (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70396 io_contex ceph txt unknown /proc/70383/task/70396/exe (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70396 io_contex ceph NOFD /proc/70383/task/70396/fd (opendir: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70397 ms_dispat ceph cwd unknown /proc/70383/task/70397/cwd (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70397 ms_dispat ceph rtd unknown /proc/70383/task/70397/root (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70397 ms_dispat ceph txt unknown /proc/70383/task/70397/exe (readlink: Permission denied) 2024-04-22T13:30:08.678 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70397 ms_dispat ceph NOFD /proc/70383/task/70397/fd (opendir: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70398 ms_local ceph cwd unknown /proc/70383/task/70398/cwd (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70398 ms_local ceph rtd unknown /proc/70383/task/70398/root (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70398 ms_local ceph txt unknown /proc/70383/task/70398/exe (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70398 ms_local ceph NOFD /proc/70383/task/70398/fd (opendir: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70399 safe_time ceph cwd unknown /proc/70383/task/70399/cwd (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70399 safe_time ceph rtd unknown /proc/70383/task/70399/root (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70399 safe_time ceph txt unknown /proc/70383/task/70399/exe (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70399 safe_time ceph NOFD /proc/70383/task/70399/fd (opendir: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70400 safe_time ceph cwd unknown /proc/70383/task/70400/cwd (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70400 safe_time ceph rtd unknown /proc/70383/task/70400/root (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70400 safe_time ceph txt unknown /proc/70383/task/70400/exe (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70400 safe_time ceph NOFD /proc/70383/task/70400/fd (opendir: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70401 ceph-mds ceph cwd unknown /proc/70383/task/70401/cwd (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70401 ceph-mds ceph rtd unknown /proc/70383/task/70401/root (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70401 ceph-mds ceph txt unknown /proc/70383/task/70401/exe (readlink: Permission denied) 2024-04-22T13:30:08.679 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70401 ceph-mds ceph NOFD /proc/70383/task/70401/fd (opendir: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70406 safe_time ceph cwd unknown /proc/70383/task/70406/cwd (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70406 safe_time ceph rtd unknown /proc/70383/task/70406/root (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70406 safe_time ceph txt unknown /proc/70383/task/70406/exe (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 70406 safe_time ceph NOFD /proc/70383/task/70406/fd (opendir: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71852 ceph_time ceph cwd unknown /proc/70383/task/71852/cwd (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71852 ceph_time ceph rtd unknown /proc/70383/task/71852/root (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71852 ceph_time ceph txt unknown /proc/70383/task/71852/exe (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71852 ceph_time ceph NOFD /proc/70383/task/71852/fd (opendir: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71853 OpHistory ceph cwd unknown /proc/70383/task/71853/cwd (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71853 OpHistory ceph rtd unknown /proc/70383/task/71853/root (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71853 OpHistory ceph txt unknown /proc/70383/task/71853/exe (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71853 OpHistory ceph NOFD /proc/70383/task/71853/fd (opendir: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71858 ms_dispat ceph cwd unknown /proc/70383/task/71858/cwd (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71858 ms_dispat ceph rtd unknown /proc/70383/task/71858/root (readlink: Permission denied) 2024-04-22T13:30:08.680 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71858 ms_dispat ceph txt unknown /proc/70383/task/71858/exe (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71858 ms_dispat ceph NOFD /proc/70383/task/71858/fd (opendir: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71859 ms_dispat ceph cwd unknown /proc/70383/task/71859/cwd (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71859 ms_dispat ceph rtd unknown /proc/70383/task/71859/root (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71859 ms_dispat ceph txt unknown /proc/70383/task/71859/exe (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71859 ms_dispat ceph NOFD /proc/70383/task/71859/fd (opendir: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71868 mds_rank_ ceph cwd unknown /proc/70383/task/71868/cwd (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71868 mds_rank_ ceph rtd unknown /proc/70383/task/71868/root (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71868 mds_rank_ ceph txt unknown /proc/70383/task/71868/exe (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71868 mds_rank_ ceph NOFD /proc/70383/task/71868/fd (opendir: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71869 PQ_Finish ceph cwd unknown /proc/70383/task/71869/cwd (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71869 PQ_Finish ceph rtd unknown /proc/70383/task/71869/root (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71869 PQ_Finish ceph txt unknown /proc/70383/task/71869/exe (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71869 PQ_Finish ceph NOFD /proc/70383/task/71869/fd (opendir: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71870 safe_time ceph cwd unknown /proc/70383/task/71870/cwd (readlink: Permission denied) 2024-04-22T13:30:08.681 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71870 safe_time ceph rtd unknown /proc/70383/task/71870/root (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71870 safe_time ceph txt unknown /proc/70383/task/71870/exe (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71870 safe_time ceph NOFD /proc/70383/task/71870/fd (opendir: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71871 MR_Finish ceph cwd unknown /proc/70383/task/71871/cwd (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71871 MR_Finish ceph rtd unknown /proc/70383/task/71871/root (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71871 MR_Finish ceph txt unknown /proc/70383/task/71871/exe (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71871 MR_Finish ceph NOFD /proc/70383/task/71871/fd (opendir: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71877 md_submit ceph cwd unknown /proc/70383/task/71877/cwd (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71877 md_submit ceph rtd unknown /proc/70383/task/71877/root (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71877 md_submit ceph txt unknown /proc/70383/task/71877/exe (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-mds 70383 71877 md_submit ceph NOFD /proc/70383/task/71877/fd (opendir: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 73901 root cwd unknown /proc/73901/cwd (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 73901 root rtd unknown /proc/73901/root (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 73901 root txt unknown /proc/73901/exe (readlink: Permission denied) 2024-04-22T13:30:08.682 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/6 73901 root NOFD /proc/73901/fd (opendir: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 76426 root cwd unknown /proc/76426/cwd (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 76426 root rtd unknown /proc/76426/root (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 76426 root txt unknown /proc/76426/exe (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/5 76426 root NOFD /proc/76426/fd (opendir: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-msgr 80900 root cwd unknown /proc/80900/cwd (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-msgr 80900 root rtd unknown /proc/80900/root (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-msgr 80900 root txt unknown /proc/80900/exe (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-msgr 80900 root NOFD /proc/80900/fd (opendir: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-watc 80901 root cwd unknown /proc/80901/cwd (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-watc 80901 root rtd unknown /proc/80901/root (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-watc 80901 root txt unknown /proc/80901/exe (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-watc 80901 root NOFD /proc/80901/fd (opendir: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-comp 80902 root cwd unknown /proc/80902/cwd (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-comp 80902 root rtd unknown /proc/80902/root (readlink: Permission denied) 2024-04-22T13:30:08.683 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-comp 80902 root txt unknown /proc/80902/exe (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:ceph-comp 80902 root NOFD /proc/80902/fd (opendir: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81213 root cwd unknown /proc/81213/cwd (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81213 root rtd unknown /proc/81213/root (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81213 root txt unknown /proc/81213/exe (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81213 root NOFD /proc/81213/fd (opendir: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81273 root cwd unknown /proc/81273/cwd (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81273 root rtd unknown /proc/81273/root (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81273 root txt unknown /proc/81273/exe (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sshd 81273 root NOFD /proc/81273/fd (opendir: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 83795 root cwd unknown /proc/83795/cwd (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 83795 root rtd unknown /proc/83795/root (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 83795 root txt unknown /proc/83795/exe (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:sudo 83795 root NOFD /proc/83795/fd (opendir: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:daemon-he 83820 root cwd unknown /proc/83820/cwd (readlink: Permission denied) 2024-04-22T13:30:08.684 INFO:tasks.workunit.client.0.smithi005.stdout:daemon-he 83820 root rtd unknown /proc/83820/root (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:daemon-he 83820 root txt unknown /proc/83820/exe (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:daemon-he 83820 root NOFD /proc/83820/fd (opendir: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:bash 83833 root cwd unknown /proc/83833/cwd (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:bash 83833 root rtd unknown /proc/83833/root (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:bash 83833 root txt unknown /proc/83833/exe (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:bash 83833 root NOFD /proc/83833/fd (opendir: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85446 root cwd unknown /proc/85446/cwd (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85446 root rtd unknown /proc/85446/root (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85446 root txt unknown /proc/85446/exe (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85446 root NOFD /proc/85446/fd (opendir: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85622 root cwd unknown /proc/85622/cwd (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85622 root rtd unknown /proc/85622/root (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85622 root txt unknown /proc/85622/exe (readlink: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/4 85622 root NOFD /proc/85622/fd (opendir: Permission denied) 2024-04-22T13:30:08.685 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-22T13:30:08.686 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu 0r FIFO 0,13 0t0 227224 pipe 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu 1w FIFO 0,13 0t0 227225 pipe 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:timeout 87317 ubuntu 2w FIFO 0,13 0t0 227226 pipe 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-22T13:30:08.687 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu 0r FIFO 0,13 0t0 227224 pipe 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu 1w FIFO 0,13 0t0 227225 pipe 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu 2w FIFO 0,13 0t0 227226 pipe 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:fsync-tes 88051 ubuntu 255r REG 8,1 207 530606 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-22T13:30:08.688 INFO:tasks.workunit.client.0.smithi005.stdout:sleep 89257 root cwd unknown /proc/89257/cwd (readlink: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:sleep 89257 root rtd unknown /proc/89257/root (readlink: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:sleep 89257 root txt unknown /proc/89257/exe (readlink: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:sleep 89257 root NOFD /proc/89257/fd (opendir: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 89606 root cwd unknown /proc/89606/cwd (readlink: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 89606 root rtd unknown /proc/89606/root (readlink: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 89606 root txt unknown /proc/89606/exe (readlink: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:kworker/1 89606 root NOFD /proc/89606/fd (opendir: Permission denied) 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-22T13:30:08.689 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 0r FIFO 0,13 0t0 227224 pipe 2024-04-22T13:30:08.690 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 1w FIFO 0,13 0t0 227225 pipe 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 2w FIFO 0,13 0t0 227226 pipe 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 3r DIR 0,20 0 1 /proc 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 4r DIR 0,20 0 238117 /proc/89719/fd 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 5w FIFO 0,13 0t0 238122 pipe 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89719 ubuntu 6r FIFO 0,13 0t0 238123 pipe 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-22T13:30:08.691 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu 4r FIFO 0,13 0t0 238122 pipe 2024-04-22T13:30:08.692 INFO:tasks.workunit.client.0.smithi005.stdout:lsof 89720 ubuntu 7w FIFO 0,13 0t0 238123 pipe 2024-04-22T13:30:08.692 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T13:30:08.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:30:08.737 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-04-22T13:30:08.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T13:30:09.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:08 smithi049 ceph-mon[31379]: pgmap v93: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-22T13:30:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:08 smithi107 ceph-mon[31859]: pgmap v93: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-22T13:30:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:08 smithi005 ceph-mon[28006]: pgmap v93: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-22T13:30:09.066 DEBUG:teuthology.parallel:result is None 2024-04-22T13:30:09.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T13:30:09.100 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T13:30:09.100 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-22T13:30:09.111 INFO:tasks.quiescer:joining Quiescers 2024-04-22T13:30:09.111 INFO:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-22T13:30:09.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:30:09.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 -- 172.21.15.5:0/2016316302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed006ab20 msgr2=0x7f6ed010edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/2016316302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed006ab20 0x7f6ed010edd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009ca0 tx=0x7f6ec802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 -- 172.21.15.5:0/2016316302 shutdown_connections 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/2016316302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed006ab20 0x7f6ed010edd0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/2016316302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed006a1a0 0x7f6ed006a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/2016316302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed0107fa0 0x7f6ed0108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 -- 172.21.15.5:0/2016316302 >> 172.21.15.5:0/2016316302 conn(0x7f6ed0077cd0 msgr2=0x7f6ed00780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 -- 172.21.15.5:0/2016316302 shutdown_connections 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 -- 172.21.15.5:0/2016316302 wait complete. 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 Processor -- start 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.250+0000 7f6ed75b2640 1 -- start start 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed006a1a0 0x7f6ed0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed006ab20 0x7f6ed0198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 0x7f6ed019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0112130 con 0x7f6ed0107fa0 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111fb0 con 0x7f6ed006a1a0 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01122b0 con 0x7f6ed006ab20 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 0x7f6ed019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 0x7f6ed019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57870/0 (socket says 172.21.15.5:57870) 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 -- 172.21.15.5:0/3716372205 learned_addr learned my addr 172.21.15.5:0/3716372205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed534e640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed006a1a0 0x7f6ed0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed4b4d640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed006ab20 0x7f6ed0198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 -- 172.21.15.5:0/3716372205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed006ab20 msgr2=0x7f6ed0198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed006ab20 0x7f6ed0198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 -- 172.21.15.5:0/3716372205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed006a1a0 msgr2=0x7f6ed0198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed006a1a0 0x7f6ed0198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 -- 172.21.15.5:0/3716372205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed019d620 con 0x7f6ed0107fa0 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed534e640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed006a1a0 0x7f6ed0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed5b4f640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 0x7f6ed019cef0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802c700 tx=0x7f6ec80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8041020 con 0x7f6ed0107fa0 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec8032960 con 0x7f6ed0107fa0 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed019d910 con 0x7f6ed0107fa0 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.251+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8039040 con 0x7f6ed0107fa0 2024-04-22T13:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.252+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01a51a0 con 0x7f6ed0107fa0 2024-04-22T13:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.252+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0068570 con 0x7f6ed0107fa0 2024-04-22T13:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.254+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6ec8044020 con 0x7f6ed0107fa0 2024-04-22T13:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.254+0000 7f6ec67fc640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.254+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f6ec80f28b0 con 0x7f6ed0107fa0 2024-04-22T13:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.254+0000 7f6ed534e640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.255+0000 7f6ed534e640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009ba0 tx=0x7f6ecc005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.258+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ec804b020 con 0x7f6ed0107fa0 2024-04-22T13:30:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.488+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ed0199980 con 0x7f6ed0107fa0 2024-04-22T13:30:09.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.489+0000 7f6ec67fc640 1 -- 172.21.15.5:0/3716372205 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f6ed0199980 con 0x7f6ed0107fa0 2024-04-22T13:30:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:30:09.486 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:30:09.486 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.491+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ebc0a3d20 msgr2=0x7f6ebc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.491+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009ba0 tx=0x7f6ecc005f90 comp rx=0 tx=0).stop 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.491+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 msgr2=0x7f6ed019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.491+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 0x7f6ed019cef0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802c700 tx=0x7f6ec80346d0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 shutdown_connections 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed0107fa0 0x7f6ed019cef0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed006ab20 0x7f6ed0198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 --2- 172.21.15.5:0/3716372205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed006a1a0 0x7f6ed0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 >> 172.21.15.5:0/3716372205 conn(0x7f6ed0077cd0 msgr2=0x7f6ed00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 shutdown_connections 2024-04-22T13:30:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.492+0000 7f6ed75b2640 1 -- 172.21.15.5:0/3716372205 wait complete. 2024-04-22T13:30:09.505 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24557 quiesce db --cancel --all' 2024-04-22T13:30:09.505 DEBUG:teuthology.orchestra.run.smithi005:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24557 quiesce db --cancel --all 2024-04-22T13:30:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 -- 172.21.15.5:0/362920360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec101090 msgr2=0x7f8bec101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 --2- 172.21.15.5:0/362920360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec101090 0x7f8bec101470 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009900 tx=0x7f8be402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 -- 172.21.15.5:0/362920360 shutdown_connections 2024-04-22T13:30:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 --2- 172.21.15.5:0/362920360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec106160 0x7f8bec10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 --2- 172.21.15.5:0/362920360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bec101a40 0x7f8bec105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 --2- 172.21.15.5:0/362920360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec101090 0x7f8bec101470 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 -- 172.21.15.5:0/362920360 >> 172.21.15.5:0/362920360 conn(0x7f8bec078000 msgr2=0x7f8bec078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 -- 172.21.15.5:0/362920360 shutdown_connections 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.619+0000 7f8bf3811640 1 -- 172.21.15.5:0/362920360 wait complete. 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 Processor -- start 2024-04-22T13:30:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 -- start start 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec101090 0x7f8bec19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bec106160 0x7f8bec19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 0x7f8bec198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1110a0 con 0x7f8bec198620 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec110f20 con 0x7f8bec101090 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf3811640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec111220 con 0x7f8bec106160 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf1dae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 0x7f8bec198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf1dae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 0x7f8bec198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57896/0 (socket says 172.21.15.5:57896) 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf1dae640 1 -- 172.21.15.5:0/3754852717 learned_addr learned my addr 172.21.15.5:0/3754852717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bec106160 0x7f8bec19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.620+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec101090 0x7f8bec19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf1dae640 1 -- 172.21.15.5:0/3754852717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bec106160 msgr2=0x7f8bec19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf1dae640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bec106160 0x7f8bec19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf1dae640 1 -- 172.21.15.5:0/3754852717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec101090 msgr2=0x7f8bec19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf1dae640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec101090 0x7f8bec19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf1dae640 1 -- 172.21.15.5:0/3754852717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec199140 con 0x7f8bec198620 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec101090 0x7f8bec19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf1dae640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 0x7f8bec198ad0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8be800cab0 tx=0x7f8be800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8be27fc640 1 -- 172.21.15.5:0/3754852717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be800adc0 con 0x7f8bec198620 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8be27fc640 1 -- 172.21.15.5:0/3754852717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be800f900 con 0x7f8bec198620 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec199430 con 0x7f8bec198620 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8be27fc640 1 -- 172.21.15.5:0/3754852717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be80189a0 con 0x7f8bec198620 2024-04-22T13:30:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.621+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1a5630 con 0x7f8bec198620 2024-04-22T13:30:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.623+0000 7f8be27fc640 1 -- 172.21.15.5:0/3754852717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8be800f450 con 0x7f8bec198620 2024-04-22T13:30:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 Processor -- start 2024-04-22T13:30:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 -- start start 2024-04-22T13:30:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 0x55a4a05ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0602690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4a055c000 con 0x55a4a05f76c0 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4a04f8b20 con 0x55a4a0600240 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a4a0557fc0 con 0x55a4a05f6d10 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8be27fc640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bd80a3d20 0x7f8bd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf15ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 0x55a4a05ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf15ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 0x55a4a05ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57910/0 (socket says 172.21.15.5:57910) 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf15ad640 1 -- 172.21.15.5:0/80884724 learned_addr learned my addr 172.21.15.5:0/80884724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0602690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8be27fc640 1 -- 172.21.15.5:0/3754852717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8be80d0a10 con 0x7f8bec198620 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.624+0000 7f8bf1dae640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bd80a3d20 0x7f8bd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf15ad640 1 -- 172.21.15.5:0/80884724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 msgr2=0x55a4a05f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf15ad640 1 -- 172.21.15.5:0/80884724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 msgr2=0x55a4a0602690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0602690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf15ad640 1 -- 172.21.15.5:0/80884724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a0512ea0 con 0x55a4a05f76c0 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0602690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 0x55a4a05ffc70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8be4000c00 tx=0x7f8be4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5fe5038feaa589 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bf1dae640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bd80a3d20 0x7f8bd80a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a6e0 tx=0x7f8be8018400 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bd77fe640 1 -- 172.21.15.5:0/80884724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4044020 con 0x55a4a05f76c0 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bd77fe640 1 -- 172.21.15.5:0/80884724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be402cc90 con 0x55a4a05f76c0 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.625+0000 7f8bd77fe640 1 -- 172.21.15.5:0/80884724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4004920 con 0x55a4a05f76c0 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 -- 172.21.15.5:0/80884724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 msgr2=0x55a4a05ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 0x55a4a05ffc70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8be4000c00 tx=0x7f8be4005e80 comp rx=0 tx=0).stop 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 -- 172.21.15.5:0/80884724 shutdown_connections 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f76c0 0x55a4a05ffc70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0602690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/80884724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 -- 172.21.15.5:0/80884724 >> 172.21.15.5:0/80884724 conn(0x55a4a05f13e0 msgr2=0x55a4a05f17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 -- 172.21.15.5:0/80884724 shutdown_connections 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 -- 172.21.15.5:0/80884724 wait complete. 2024-04-22T13:30:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 -- start start 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.626+0000 7f8bf63ff740 1 Processor -- start 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a05fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a05fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a0600240 0x55a4a05fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4a0603060 con 0x55a4a05f6d10 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a4a04f6510 con 0x55a4a05f76c0 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a4a06031e0 con 0x55a4a0600240 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf63ff740 10 client.0 _ll_register_callbacks cb 0x55a4a04f5f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a05fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a05fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57918/0 (socket says 172.21.15.5:57918) 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 -- 172.21.15.5:0/296843321 learned_addr learned my addr 172.21.15.5:0/296843321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a05fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a0600240 0x55a4a05fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 -- 172.21.15.5:0/296843321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a0600240 msgr2=0x55a4a05fdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a0600240 0x55a4a05fdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 -- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 msgr2=0x55a4a05fd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a05fd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.627+0000 7f8bf0dac640 1 -- 172.21.15.5:0/296843321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a033aa20 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a05fd1b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00bfc0 tx=0x7f8bdc00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779d0aaeefde6686 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/296843321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc013e60 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/296843321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bdc00ec60 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be402cc90 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/296843321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bdc00ee60 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 10 client.16986 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4a039b1b0 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/296843321 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be402cc90 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/296843321 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bdc003c90 con 0x55a4a05f6d10 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 10 client.16986 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 10 client.16986 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 20 client.16986 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06105e0 0x55a4a0612a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 4 client.16986 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.628+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4a0569a20 con 0x55a4a06105e0 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.629+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06105e0 0x55a4a0612a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.629+0000 7f8bd4ff9640 10 client.16986 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.629+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06105e0 0x55a4a0612a10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768802dddcca3000 server_cookie=8c2eab36671cbc97 in_seq=0 out_seq=0 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/296843321 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4a0569a20 con 0x55a4a06105e0 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bd4ff9640 10 client.16986 handle_command_reply: tid=0 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 client.16986 shutdown 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 msgr2=0x55a4a05fd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a05fd1b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00bfc0 tx=0x7f8bdc00bbb0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 shutdown_connections 2024-04-22T13:30:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06105e0 0x55a4a0612a10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a0600240 0x55a4a05fdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a05fd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/296843321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a05fd1b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bd4ff9640 0 client.16986 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 >> 172.21.15.5:0/296843321 conn(0x55a4a05f13e0 msgr2=0x55a4a05f0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 shutdown_connections 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 1 -- 172.21.15.5:0/296843321 wait complete. 2024-04-22T13:30:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.630+0000 7f8bf63ff740 20 client.16986 trim_cache size 0 max 16384 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 Processor -- start 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 -- start start 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a060bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a0600240 0x55a4a05fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1adbc0 con 0x55a4a0600240 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1af700 con 0x55a4a05f76c0 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ae290 con 0x55a4a05f6d10 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a0600240 0x55a4a05fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf15ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a060bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:47852/0 (socket says 172.21.15.5:47852) 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.664+0000 7f8bf0dac640 1 -- 172.21.15.5:0/2969496345 learned_addr learned my addr 172.21.15.5:0/2969496345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf15ad640 1 -- 172.21.15.5:0/2969496345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 msgr2=0x55a4a05f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf15ad640 1 -- 172.21.15.5:0/2969496345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a0600240 msgr2=0x55a4a05fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a0600240 0x55a4a05fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf15ad640 1 -- 172.21.15.5:0/2969496345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a022e310 con 0x55a4a05f76c0 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.665+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a060bfe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55a4a05f2450 tx=0x7f8be4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ec85fafeefdc59 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.666+0000 7f8bd6ffd640 1 -- 172.21.15.5:0/2969496345 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4044040 con 0x55a4a05f76c0 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.666+0000 7f8bd6ffd640 1 -- 172.21.15.5:0/2969496345 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be402cc90 con 0x55a4a05f76c0 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.666+0000 7f8bd6ffd640 1 -- 172.21.15.5:0/2969496345 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4003850 con 0x55a4a05f76c0 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 -- 172.21.15.5:0/2969496345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 msgr2=0x55a4a060bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a060bfe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55a4a05f2450 tx=0x7f8be4005d50 comp rx=0 tx=0).stop 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 -- 172.21.15.5:0/2969496345 shutdown_connections 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a0600240 0x55a4a05fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a05f76c0 0x55a4a060bfe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/2969496345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f6d10 0x55a4a05f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 -- 172.21.15.5:0/2969496345 >> 172.21.15.5:0/2969496345 conn(0x55a4a05f13e0 msgr2=0x55a4a05fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 -- 172.21.15.5:0/2969496345 shutdown_connections 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 -- 172.21.15.5:0/2969496345 wait complete. 2024-04-22T13:30:09.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.667+0000 7f8bf63ff740 1 -- start start 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 Processor -- start 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a0604700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f76c0 0x55a4a0604c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0675b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1af030 con 0x55a4a05f6d10 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1ae960 con 0x55a4a0600240 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1abbc0 con 0x55a4a05f76c0 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf63ff740 10 client.0 _ll_register_callbacks cb 0x7f8bdc013540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a0604700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a0604700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57928/0 (socket says 172.21.15.5:57928) 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 -- 172.21.15.5:0/400760634 learned_addr learned my addr 172.21.15.5:0/400760634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f76c0 0x55a4a0604c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 -- 172.21.15.5:0/400760634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f76c0 msgr2=0x55a4a0604c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f76c0 0x55a4a0604c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 -- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 msgr2=0x55a4a0675b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0675b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.668+0000 7f8bf0dac640 1 -- 172.21.15.5:0/400760634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a0676110 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf0dac640 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a0604700 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00b4e0 tx=0x7f8bdc00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a07e7247191d096 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/400760634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc013e60 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/400760634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bdc020a70 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be402cc90 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/400760634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bdc03c050 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 10 client.16992 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4a06773d0 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/400760634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be402cc90 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/400760634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bdc028400 con 0x55a4a05f6d10 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 10 client.16992 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 10 client.16992 resolve_mds: validated gid 24557 aka daemon mds.b 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 20 client.16992 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06779c0 0x55a4a0679de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 4 client.16992 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.669+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55a4a0563890 con 0x55a4a06779c0 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06779c0 0x55a4a0679de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf15ad640 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06779c0 0x55a4a0679de0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4be0c22ff29384c server_cookie=befc4495cc076674 in_seq=0 out_seq=0 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bd4ff9640 10 client.16992 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bd4ff9640 1 -- 172.21.15.5:0/400760634 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55a4a0563890 con 0x55a4a06779c0 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bd4ff9640 10 client.16992 handle_command_reply: tid=0 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 client.16992 shutdown 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 msgr2=0x55a4a0604700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a0604700 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00b4e0 tx=0x7f8bdc00b7e0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 shutdown_connections 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a4a06779c0 0x55a4a0679de0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4a0600240 0x55a4a0675b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4a05f76c0 0x55a4a0604c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bd4ff9640 0 client.16992 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 --2- 172.21.15.5:0/400760634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4a05f6d10 0x55a4a0604700 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.670+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 >> 172.21.15.5:0/400760634 conn(0x55a4a05f13e0 msgr2=0x55a4a05f2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 shutdown_connections 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf63ff740 1 -- 172.21.15.5:0/400760634 wait complete. 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf63ff740 20 client.16992 trim_cache size 0 max 16384 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-04-22T13:30:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24557, 2024-04-22T13:30:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 16, 2024-04-22T13:30:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": {} 2024-04-22T13:30:09.673 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:30:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bd80a3d20 msgr2=0x7f8bd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bd80a3d20 0x7f8bd80a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a6e0 tx=0x7f8be8018400 comp rx=0 tx=0).stop 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 msgr2=0x7f8bec198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 0x7f8bec198ad0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8be800cab0 tx=0x7f8be800cf70 comp rx=0 tx=0).stop 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 shutdown_connections 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec198620 0x7f8bec198ad0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bd80a3d20 0x7f8bd80a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bec106160 0x7f8bec19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 --2- 172.21.15.5:0/3754852717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bec101090 0x7f8bec19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 >> 172.21.15.5:0/3754852717 conn(0x7f8bec078000 msgr2=0x7f8bec069790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 shutdown_connections 2024-04-22T13:30:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:09.671+0000 7f8bf3811640 1 -- 172.21.15.5:0/3754852717 wait complete. 2024-04-22T13:30:09.679 INFO:tasks.quiescer:done joining Quiescers 2024-04-22T13:30:09.679 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-22T13:30:09.690 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:30:09.700 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-22T13:30:09.710 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-22T13:30:09.710 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:30:09.710 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:30:09.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-22T13:30:09.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:09 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3716372205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:09.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-22T13:30:10.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:09 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3716372205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:10.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:09 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3716372205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:10 smithi049 ceph-mon[31379]: pgmap v94: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-22T13:30:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:10 smithi107 ceph-mon[31859]: pgmap v94: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-22T13:30:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:10 smithi005 ceph-mon[28006]: pgmap v94: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-22T13:30:13.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:12 smithi049 ceph-mon[31379]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 752 op/s 2024-04-22T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:12 smithi107 ceph-mon[31859]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 752 op/s 2024-04-22T13:30:13.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:12 smithi005 ceph-mon[28006]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 752 op/s 2024-04-22T13:30:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:14 smithi049 ceph-mon[31379]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 743 op/s 2024-04-22T13:30:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:14 smithi107 ceph-mon[31859]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 743 op/s 2024-04-22T13:30:15.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:14 smithi005 ceph-mon[28006]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 743 op/s 2024-04-22T13:30:17.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:16 smithi049 ceph-mon[31379]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 912 op/s 2024-04-22T13:30:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:16 smithi107 ceph-mon[31859]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 912 op/s 2024-04-22T13:30:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:16 smithi005 ceph-mon[28006]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 912 op/s 2024-04-22T13:30:18.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:30:18.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:30:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:30:19.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:18 smithi049 ceph-mon[31379]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 568 op/s 2024-04-22T13:30:19.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:18 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:30:19.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:18 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:30:19.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:18 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:30:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:18 smithi107 ceph-mon[31859]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 568 op/s 2024-04-22T13:30:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:18 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:30:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:18 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:30:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:18 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:30:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:18 smithi005 ceph-mon[28006]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 568 op/s 2024-04-22T13:30:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:18 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:30:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:18 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:30:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:18 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:30:21.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:20 smithi049 ceph-mon[31379]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 568 op/s 2024-04-22T13:30:21.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:20 smithi107 ceph-mon[31859]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 568 op/s 2024-04-22T13:30:21.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:20 smithi005 ceph-mon[28006]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 568 op/s 2024-04-22T13:30:23.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:22 smithi049 ceph-mon[31379]: pgmap v100: 97 pgs: 97 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.23k op/s 2024-04-22T13:30:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:22 smithi107 ceph-mon[31859]: pgmap v100: 97 pgs: 97 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.23k op/s 2024-04-22T13:30:23.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:22 smithi005 ceph-mon[28006]: pgmap v100: 97 pgs: 97 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.23k op/s 2024-04-22T13:30:25.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:24 smithi049 ceph-mon[31379]: pgmap v101: 97 pgs: 97 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.6 MiB/s wr, 832 op/s 2024-04-22T13:30:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:24 smithi107 ceph-mon[31859]: pgmap v101: 97 pgs: 97 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.6 MiB/s wr, 832 op/s 2024-04-22T13:30:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:24 smithi005 ceph-mon[28006]: pgmap v101: 97 pgs: 97 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.6 MiB/s wr, 832 op/s 2024-04-22T13:30:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:26 smithi049 ceph-mon[31379]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 MiB/s wr, 842 op/s 2024-04-22T13:30:27.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:26 smithi107 ceph-mon[31859]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 MiB/s wr, 842 op/s 2024-04-22T13:30:27.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:26 smithi005 ceph-mon[28006]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 MiB/s wr, 842 op/s 2024-04-22T13:30:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:28 smithi049 ceph-mon[31379]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 672 op/s 2024-04-22T13:30:29.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:28 smithi107 ceph-mon[31859]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 672 op/s 2024-04-22T13:30:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:28 smithi005 ceph-mon[28006]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 672 op/s 2024-04-22T13:30:30.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:30.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:30.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 -- 172.21.15.5:0/2518069962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 msgr2=0x7f1a54102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 --2- 172.21.15.5:0/2518069962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54102560 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:30.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 -- 172.21.15.5:0/2518069962 shutdown_connections 2024-04-22T13:30:30.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 --2- 172.21.15.5:0/2518069962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a54107250 0x7f1a5410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 --2- 172.21.15.5:0/2518069962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a54102b30 0x7f1a54106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 --2- 172.21.15.5:0/2518069962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54102560 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 -- 172.21.15.5:0/2518069962 >> 172.21.15.5:0/2518069962 conn(0x7f1a540fda00 msgr2=0x7f1a540ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 -- 172.21.15.5:0/2518069962 shutdown_connections 2024-04-22T13:30:30.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.449+0000 7f1a59106640 1 -- 172.21.15.5:0/2518069962 wait complete. 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.450+0000 7f1a59106640 1 Processor -- start 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.450+0000 7f1a59106640 1 -- start start 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a59106640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a59106640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a54107250 0x7f1a54074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a59106640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a54074cd0 0x7f1a541a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a59106640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541121b0 con 0x7f1a54102180 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a59106640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54112030 con 0x7f1a54074cd0 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a59106640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54112330 con 0x7f1a54107250 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36102/0 (socket says 172.21.15.5:36102) 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 -- 172.21.15.5:0/429005942 learned_addr learned my addr 172.21.15.5:0/429005942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a5259c640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a54107250 0x7f1a54074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a5359e640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a54074cd0 0x7f1a541a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 -- 172.21.15.5:0/429005942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a54107250 msgr2=0x7f1a54074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a54107250 0x7f1a54074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 -- 172.21.15.5:0/429005942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a54074cd0 msgr2=0x7f1a541a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a54074cd0 0x7f1a541a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 -- 172.21.15.5:0/429005942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a541a7040 con 0x7f1a54102180 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54074250 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005940 tx=0x7f1a4402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.451+0000 7f1a3bfff640 1 -- 172.21.15.5:0/429005942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44043020 con 0x7f1a54102180 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.452+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a541a72d0 con 0x7f1a54102180 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.452+0000 7f1a3bfff640 1 -- 172.21.15.5:0/429005942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4402cbe0 con 0x7f1a54102180 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.452+0000 7f1a3bfff640 1 -- 172.21.15.5:0/429005942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44003bf0 con 0x7f1a54102180 2024-04-22T13:30:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.452+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a541a7820 con 0x7f1a54102180 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.454+0000 7f1a3bfff640 1 -- 172.21.15.5:0/429005942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1a44046020 con 0x7f1a54102180 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.454+0000 7f1a5bdcb740 1 Processor -- start 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.454+0000 7f1a5bdcb740 1 -- start start 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a85960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 0x561424a90e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614249ea1e0 con 0x561424a8ea80 2024-04-22T13:30:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561424987390 con 0x561424a85ea0 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5614249e61a0 con 0x561424a85580 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a3bfff640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a3c0a3d50 0x7f1a3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 0x561424a90e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 0x561424a90e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36104/0 (socket says 172.21.15.5:36104) 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 -- 172.21.15.5:0/4255446364 learned_addr learned my addr 172.21.15.5:0/4255446364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5359e640 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a85960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5359e640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a3c0a3d50 0x7f1a3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a3bfff640 1 -- 172.21.15.5:0/429005942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1a440f2840 con 0x7f1a54102180 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 -- 172.21.15.5:0/4255446364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 msgr2=0x561424a85960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a85960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 -- 172.21.15.5:0/4255446364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 msgr2=0x561424a8e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 -- 172.21.15.5:0/4255446364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614247c67d0 con 0x561424a8ea80 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.455+0000 7f1a5259c640 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 0x561424a90e70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b6c0 tx=0x7f1a4800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f065e8e323a54357 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5359e640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a3c0a3d50 0x7f1a3c0a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c009600 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a397fa640 1 -- 172.21.15.5:0/4255446364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48003bd0 con 0x561424a8ea80 2024-04-22T13:30:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a397fa640 1 -- 172.21.15.5:0/4255446364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a48015070 con 0x561424a8ea80 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a397fa640 1 -- 172.21.15.5:0/4255446364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004c30 con 0x561424a8ea80 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/4255446364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 msgr2=0x561424a90e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 0x561424a90e70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b6c0 tx=0x7f1a4800bb80 comp rx=0 tx=0).stop 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/4255446364 shutdown_connections 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a8ea80 0x561424a90e70 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/4255446364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a85960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/4255446364 >> 172.21.15.5:0/4255446364 conn(0x561424a7fcb0 msgr2=0x561424a80090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/4255446364 shutdown_connections 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.456+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/4255446364 wait complete. 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 -- start start 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 Processor -- start 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 0x561424a8baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a8c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561424a91870 con 0x561424a85580 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5614249a32f0 con 0x561424a85ea0 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561424a919f0 con 0x561424a8ea80 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5bdcb740 10 client.0 _ll_register_callbacks cb 0x5614249b0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 0x561424a8baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 0x561424a8baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36108/0 (socket says 172.21.15.5:36108) 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a52d9d640 1 -- 172.21.15.5:0/371238358 learned_addr learned my addr 172.21.15.5:0/371238358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.457+0000 7f1a5259c640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a52d9d640 1 -- 172.21.15.5:0/371238358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 msgr2=0x561424a8c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a5359e640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a8c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a8c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a52d9d640 1 -- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 msgr2=0x561424a8bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a52d9d640 1 -- 172.21.15.5:0/371238358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614246fc3f0 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a5259c640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a5359e640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a8c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 0x561424a8baa0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1a440be8b0 tx=0x7f1a4408e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ded4bda74939e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a36ff5640 1 -- 172.21.15.5:0/371238358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a440a7e60 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a36ff5640 1 -- 172.21.15.5:0/371238358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1a440baf90 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a48015070 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a36ff5640 1 -- 172.21.15.5:0/371238358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a440be280 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a5bdcb740 10 client.17004 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614246bc0f0 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.458+0000 7f1a36ff5640 1 -- 172.21.15.5:0/371238358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a48015070 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a36ff5640 1 -- 172.21.15.5:0/371238358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1a440a90b0 con 0x561424a85580 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5bdcb740 10 client.17004 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5bdcb740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5bdcb740 20 client.17004 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424a9e930 0x561424aa0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5bdcb740 4 client.17004 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614249f7c00 con 0x561424a9e930 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5259c640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424a9e930 0x561424aa0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a36ff5640 10 client.17004 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.459+0000 7f1a5259c640 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424a9e930 0x561424aa0d60 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b78816f3b180e9 server_cookie=3c52e08dd8de5439 in_seq=0 out_seq=0 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a36ff5640 1 -- 172.21.15.5:0/371238358 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614249f7c00 con 0x561424a9e930 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a36ff5640 10 client.17004 handle_command_reply: tid=0 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 client.17004 shutdown 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 msgr2=0x561424a8baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 0x561424a8baa0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1a440be8b0 tx=0x7f1a4408e200 comp rx=0 tx=0).stop 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 shutdown_connections 2024-04-22T13:30:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424a9e930 0x561424aa0d60 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a8c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a36ff5640 0 client.17004 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85ea0 0x561424a8bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/371238358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85580 0x561424a8baa0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 >> 172.21.15.5:0/371238358 conn(0x561424a7fcb0 msgr2=0x561424a81a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.460+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 shutdown_connections 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.461+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/371238358 wait complete. 2024-04-22T13:30:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.461+0000 7f1a5bdcb740 20 client.17004 trim_cache size 0 max 16384 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.478+0000 7f1a5bdcb740 1 Processor -- start 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.478+0000 7f1a5bdcb740 1 -- start start 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85580 0x561424a8e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a86280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541afcc0 con 0x561424a85ea0 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b1800 con 0x561424a85580 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b0390 con 0x561424a8ea80 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36118/0 (socket says 172.21.15.5:36118) 2024-04-22T13:30:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 -- 172.21.15.5:0/1609010039 learned_addr learned my addr 172.21.15.5:0/1609010039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5359e640 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a86280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85580 0x561424a8e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 -- 172.21.15.5:0/1609010039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 msgr2=0x561424a86280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a86280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 -- 172.21.15.5:0/1609010039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85580 msgr2=0x561424a8e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85580 0x561424a8e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 -- 172.21.15.5:0/1609010039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561424a8aa50 con 0x561424a85ea0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a5259c640 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a9a730 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1a480039c0 tx=0x7f1a4800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36d570ad1299603 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a38ff9640 1 -- 172.21.15.5:0/1609010039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48003c50 con 0x561424a85ea0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a38ff9640 1 -- 172.21.15.5:0/1609010039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a48015070 con 0x561424a85ea0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.479+0000 7f1a38ff9640 1 -- 172.21.15.5:0/1609010039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004a70 con 0x561424a85ea0 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/1609010039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 msgr2=0x561424a9a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a9a730 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1a480039c0 tx=0x7f1a4800b390 comp rx=0 tx=0).stop 2024-04-22T13:30:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/1609010039 shutdown_connections 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a9a730 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a8ea80 0x561424a86280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/1609010039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a85580 0x561424a8e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/1609010039 >> 172.21.15.5:0/1609010039 conn(0x561424a7fcb0 msgr2=0x561424a7e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/1609010039 shutdown_connections 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/1609010039 wait complete. 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.480+0000 7f1a5bdcb740 1 -- start start 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 Processor -- start 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a96c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a97180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a8ea80 0x561424a93c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b1130 con 0x561424a85ea0 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b0a60 con 0x561424a8ea80 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541adcc0 con 0x561424a85580 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5bdcb740 10 client.0 _ll_register_callbacks cb 0x7f1a440057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a97180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a97180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36120/0 (socket says 172.21.15.5:36120) 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 -- 172.21.15.5:0/3006490090 learned_addr learned my addr 172.21.15.5:0/3006490090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a96c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5359e640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a8ea80 0x561424a93c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 -- 172.21.15.5:0/3006490090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 msgr2=0x561424a96c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a96c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 -- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a8ea80 msgr2=0x561424a93c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a8ea80 0x561424a93c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.481+0000 7f1a5259c640 1 -- 172.21.15.5:0/3006490090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561424a94280 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5259c640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a97180 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1a480045c0 tx=0x7f1a48003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740061ffba58558 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a36ff5640 1 -- 172.21.15.5:0/3006490090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48005050 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a36ff5640 1 -- 172.21.15.5:0/3006490090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1a480051f0 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a48015070 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a36ff5640 1 -- 172.21.15.5:0/3006490090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a480053f0 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 10 client.17010 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561424b05810 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a36ff5640 1 -- 172.21.15.5:0/3006490090 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a48015070 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a36ff5640 1 -- 172.21.15.5:0/3006490090 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1a48028260 con 0x561424a85ea0 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 10 client.17010 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 20 client.17010 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.482+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424b05cf0 0x561424b08110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a5bdcb740 4 client.17010 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614249f1a70 con 0x561424b05cf0 2024-04-22T13:30:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424b05cf0 0x561424b08110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a36ff5640 10 client.17010 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a52d9d640 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424b05cf0 0x561424b08110 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c863574ac281acc server_cookie=24ee25940a1241f6 in_seq=0 out_seq=0 2024-04-22T13:30:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a36ff5640 1 -- 172.21.15.5:0/3006490090 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5614249f1a70 con 0x561424b05cf0 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.483+0000 7f1a36ff5640 10 client.17010 handle_command_reply: tid=0 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 client.17010 shutdown 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 msgr2=0x561424a97180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a97180 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1a480045c0 tx=0x7f1a48003b00 comp rx=0 tx=0).stop 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 shutdown_connections 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x561424b05cf0 0x561424b08110 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561424a85ea0 0x561424a97180 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561424a8ea80 0x561424a93c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a36ff5640 0 client.17010 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 --2- 172.21.15.5:0/3006490090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561424a85580 0x561424a96c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 >> 172.21.15.5:0/3006490090 conn(0x561424a7fcb0 msgr2=0x561424a8c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 shutdown_connections 2024-04-22T13:30:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 1 -- 172.21.15.5:0/3006490090 wait complete. 2024-04-22T13:30:30.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a5bdcb740 20 client.17010 trim_cache size 0 max 16384 2024-04-22T13:30:30.487 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:30:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:30:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:30:30.487 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a3c0a3d50 msgr2=0x7f1a3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a3c0a3d50 0x7f1a3c0a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c009600 comp rx=0 tx=0).stop 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 msgr2=0x7f1a54074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54074250 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005940 tx=0x7f1a4402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 shutdown_connections 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1a3c0a3d50 0x7f1a3c0a6210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a54074cd0 0x7f1a541a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a54107250 0x7f1a54074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 --2- 172.21.15.5:0/429005942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a54102180 0x7f1a54074250 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.484+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 >> 172.21.15.5:0/429005942 conn(0x7f1a540fda00 msgr2=0x7f1a540fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.485+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 shutdown_connections 2024-04-22T13:30:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.485+0000 7f1a59106640 1 -- 172.21.15.5:0/429005942 wait complete. 2024-04-22T13:30:30.491 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:30:30.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:30:30.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.598+0000 7f0983338640 1 -- 172.21.15.5:0/2514545472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101a20 msgr2=0x7f097c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.598+0000 7f0983338640 1 --2- 172.21.15.5:0/2514545472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101a20 0x7f097c105a60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f096c009900 tx=0x7f096c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- 172.21.15.5:0/2514545472 shutdown_connections 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 --2- 172.21.15.5:0/2514545472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101a20 0x7f097c105a60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 --2- 172.21.15.5:0/2514545472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c106140 0x7f097c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 --2- 172.21.15.5:0/2514545472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f097c101070 0x7f097c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- 172.21.15.5:0/2514545472 >> 172.21.15.5:0/2514545472 conn(0x7f097c0771d0 msgr2=0x7f097c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- 172.21.15.5:0/2514545472 shutdown_connections 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- 172.21.15.5:0/2514545472 wait complete. 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 Processor -- start 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- start start 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 0x7f097c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f097c106140 0x7f097c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c19cf90 0x7f097c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f097c111060 con 0x7f097c101070 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f097c110ee0 con 0x7f097c106140 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.599+0000 7f0983338640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1111e0 con 0x7f097c19cf90 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 0x7f097c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 0x7f097c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36130/0 (socket says 172.21.15.5:36130) 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 -- 172.21.15.5:0/2744067353 learned_addr learned my addr 172.21.15.5:0/2744067353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09808d3640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f097c106140 0x7f097c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09818d5640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c19cf90 0x7f097c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 -- 172.21.15.5:0/2744067353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c19cf90 msgr2=0x7f097c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c19cf90 0x7f097c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 -- 172.21.15.5:0/2744067353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f097c106140 msgr2=0x7f097c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f097c106140 0x7f097c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 -- 172.21.15.5:0/2744067353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c19da80 con 0x7f097c101070 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09818d5640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c19cf90 0x7f097c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09810d4640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 0x7f097c1986d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f097800d950 tx=0x7f097800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978017070 con 0x7f097c101070 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0978009c30 con 0x7f097c101070 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c19dd70 con 0x7f097c101070 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097800f620 con 0x7f097c101070 2024-04-22T13:30:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.600+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c1a5580 con 0x7f097c101070 2024-04-22T13:30:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.601+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c068e00 con 0x7f097c101070 2024-04-22T13:30:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.602+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0978010460 con 0x7f097c101070 2024-04-22T13:30:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.603+0000 7f09727fc640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f09680a3da0 0x7f09680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.603+0000 7f09808d3640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f09680a3da0 0x7f09680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.603+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f09780d03f0 con 0x7f097c101070 2024-04-22T13:30:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.604+0000 7f09808d3640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f09680a3da0 0x7f09680a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f096c009230 tx=0x7f096c005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.608+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f097809ca60 con 0x7f097c101070 2024-04-22T13:30:30.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.836+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f097c101bb0 con 0x7f097c101070 2024-04-22T13:30:30.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.837+0000 7f09727fc640 1 -- 172.21.15.5:0/2744067353 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f0978009de0 con 0x7f097c101070 2024-04-22T13:30:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:30:30.833 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:30:30.833 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f09680a3da0 msgr2=0x7f09680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f09680a3da0 0x7f09680a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f096c009230 tx=0x7f096c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 msgr2=0x7f097c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 0x7f097c1986d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f097800d950 tx=0x7f097800de10 comp rx=0 tx=0).stop 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 shutdown_connections 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f09680a3da0 0x7f09680a6260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f097c19cf90 0x7f097c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f097c106140 0x7f097c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 --2- 172.21.15.5:0/2744067353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f097c101070 0x7f097c1986d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 >> 172.21.15.5:0/2744067353 conn(0x7f097c0771d0 msgr2=0x7f097c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 shutdown_connections 2024-04-22T13:30:30.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.839+0000 7f0983338640 1 -- 172.21.15.5:0/2744067353 wait complete. 2024-04-22T13:30:30.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:30:30.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 -- 172.21.15.5:0/602164797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb020107250 msgr2=0x7fb02010ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 --2- 172.21.15.5:0/602164797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb020107250 0x7fb02010ee20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb008009900 tx=0x7fb00802bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:30.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 -- 172.21.15.5:0/602164797 shutdown_connections 2024-04-22T13:30:30.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 --2- 172.21.15.5:0/602164797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb020107250 0x7fb02010ee20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 --2- 172.21.15.5:0/602164797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb020102b30 0x7fb020106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 --2- 172.21.15.5:0/602164797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb020102180 0x7fb020102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 -- 172.21.15.5:0/602164797 >> 172.21.15.5:0/602164797 conn(0x7fb0200fda00 msgr2=0x7fb0200ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.958+0000 7fb026b2e640 1 -- 172.21.15.5:0/602164797 shutdown_connections 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 -- 172.21.15.5:0/602164797 wait complete. 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 Processor -- start 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 -- start start 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb020102180 0x7fb020198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb020107250 0x7fb020198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 0x7fb02019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201121a0 con 0x7fb02019cf90 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb020112020 con 0x7fb020102180 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb026b2e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb020112320 con 0x7fb020107250 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb0250cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 0x7fb02019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb0250cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 0x7fb02019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36144/0 (socket says 172.21.15.5:36144) 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb0250cb640 1 -- 172.21.15.5:0/1875219815 learned_addr learned my addr 172.21.15.5:0/1875219815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb0248ca640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb020102180 0x7fb020198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.959+0000 7fb01ffff640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb020107250 0x7fb020198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb0250cb640 1 -- 172.21.15.5:0/1875219815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb020107250 msgr2=0x7fb020198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb0250cb640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb020107250 0x7fb020198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb0250cb640 1 -- 172.21.15.5:0/1875219815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb020102180 msgr2=0x7fb020198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb0250cb640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb020102180 0x7fb020198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb0250cb640 1 -- 172.21.15.5:0/1875219815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02019da80 con 0x7fb02019cf90 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb0250cb640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 0x7fb02019d440 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb008009230 tx=0x7fb0080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb01dffb640 1 -- 172.21.15.5:0/1875219815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008043020 con 0x7fb02019cf90 2024-04-22T13:30:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb01dffb640 1 -- 172.21.15.5:0/1875219815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00802cb70 con 0x7fb02019cf90 2024-04-22T13:30:30.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02019dd10 con 0x7fb02019cf90 2024-04-22T13:30:30.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb01dffb640 1 -- 172.21.15.5:0/1875219815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008003a50 con 0x7fb02019cf90 2024-04-22T13:30:30.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.960+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201a5530 con 0x7fb02019cf90 2024-04-22T13:30:30.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb01dffb640 1 -- 172.21.15.5:0/1875219815 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb008046020 con 0x7fb02019cf90 2024-04-22T13:30:30.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 Processor -- start 2024-04-22T13:30:30.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 -- start start 2024-04-22T13:30:30.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b14d0 0x55a9587b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a9587161f0 con 0x55a9587baa00 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a9586b34b0 con 0x55a9587b14d0 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.963+0000 7fb0297f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a9587121b0 con 0x55a9587b1e80 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:37366/0 (socket says 172.21.15.5:37366) 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 -- 172.21.15.5:0/2347734754 learned_addr learned my addr 172.21.15.5:0/2347734754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb01ffff640 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb01ffff640 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b14d0 0x55a9587b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb01dffb640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb00c0a3d20 0x7fb00c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 -- 172.21.15.5:0/2347734754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b14d0 msgr2=0x55a9587b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b14d0 0x55a9587b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 -- 172.21.15.5:0/2347734754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 msgr2=0x55a9587bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 -- 172.21.15.5:0/2347734754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9586a51b0 con 0x55a9587b1e80 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb01dffb640 1 -- 172.21.15.5:0/1875219815 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb0080f28f0 con 0x7fb02019cf90 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0250cb640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb00c0a3d20 0x7fb00c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb0248ca640 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587ba430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb01800ad00 tx=0x7fb01800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f366d6507b68f093 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb016ffd640 1 -- 172.21.15.5:0/2347734754 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01801d070 con 0x55a9587b1e80 2024-04-22T13:30:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb016ffd640 1 -- 172.21.15.5:0/2347734754 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb018018af0 con 0x55a9587b1e80 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.964+0000 7fb016ffd640 1 -- 172.21.15.5:0/2347734754 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018010b40 con 0x55a9587b1e80 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 -- 172.21.15.5:0/2347734754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 msgr2=0x55a9587ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0250cb640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb00c0a3d20 0x7fb00c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb008005940 tx=0x7fb008005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587ba430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb01800ad00 tx=0x7fb01800f790 comp rx=0 tx=0).stop 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 -- 172.21.15.5:0/2347734754 shutdown_connections 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587ba430 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2347734754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b14d0 0x55a9587b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 -- 172.21.15.5:0/2347734754 >> 172.21.15.5:0/2347734754 conn(0x55a9587abba0 msgr2=0x55a9587abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 -- 172.21.15.5:0/2347734754 shutdown_connections 2024-04-22T13:30:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 -- 172.21.15.5:0/2347734754 wait complete. 2024-04-22T13:30:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.965+0000 7fb0297f5740 1 -- start start 2024-04-22T13:30:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 Processor -- start 2024-04-22T13:30:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 0x55a9587ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a9587bd970 con 0x55a9587b14d0 2024-04-22T13:30:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a9586cf500 con 0x55a9587baa00 2024-04-22T13:30:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a9587bdaf0 con 0x55a9587b1e80 2024-04-22T13:30:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0297f5740 10 client.0 _ll_register_callbacks cb 0x55a95869db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 0x55a9587ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 0x55a9587ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36168/0 (socket says 172.21.15.5:36168) 2024-04-22T13:30:30.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 -- 172.21.15.5:0/2008777302 learned_addr learned my addr 172.21.15.5:0/2008777302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb0248ca640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 -- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 msgr2=0x55a9587b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 -- 172.21.15.5:0/2008777302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 msgr2=0x55a9587b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.966+0000 7fb01ffff640 1 -- 172.21.15.5:0/2008777302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9584283f0 con 0x55a9587b14d0 2024-04-22T13:30:30.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb01ffff640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 0x55a9587ba310 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb0100042f0 tx=0x7fb010004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1fbf3ecc0df571 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:30.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7faffffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:30.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7faffffff640 1 -- 172.21.15.5:0/2008777302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010004b20 con 0x55a9587b14d0 2024-04-22T13:30:30.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7faffffff640 1 -- 172.21.15.5:0/2008777302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb010004cc0 con 0x55a9587b14d0 2024-04-22T13:30:30.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb018018af0 con 0x55a9587b14d0 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7faffffff640 1 -- 172.21.15.5:0/2008777302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb010012a80 con 0x55a9587b14d0 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 10 client.17028 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9584f27d0 con 0x55a9587b14d0 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7faffffff640 1 -- 172.21.15.5:0/2008777302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb018018af0 con 0x55a9587b14d0 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7faffffff640 1 -- 172.21.15.5:0/2008777302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb01000f040 con 0x55a9587b14d0 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 10 client.17028 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:30.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 10 client.17028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:30:30.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 20 client.17028 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:30.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a9587caca0 0x55a9587cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 4 client.17028 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:30.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9586b5930 con 0x55a9587caca0 2024-04-22T13:30:30.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.967+0000 7fb0248ca640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a9587caca0 0x55a9587cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.968+0000 7faffffff640 10 client.17028 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:30:30.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.968+0000 7fb0248ca640 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a9587caca0 0x55a9587cd090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1bc504b9d899202 server_cookie=c124f3d13d75c405 in_seq=0 out_seq=0 2024-04-22T13:30:30.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.968+0000 7faffffff640 1 -- 172.21.15.5:0/2008777302 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9586b5930 con 0x55a9587caca0 2024-04-22T13:30:30.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.968+0000 7faffffff640 10 client.17028 handle_command_reply: tid=0 2024-04-22T13:30:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 client.17028 shutdown 2024-04-22T13:30:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 msgr2=0x55a9587ba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 0x55a9587ba310 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb0100042f0 tx=0x7fb010004ab0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 shutdown_connections 2024-04-22T13:30:30.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b1e80 0x55a9587b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a9587caca0 0x55a9587cd090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 --2- 172.21.15.5:0/2008777302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b14d0 0x55a9587ba310 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7faffffff640 0 client.17028 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:30:30.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 >> 172.21.15.5:0/2008777302 conn(0x55a9587abba0 msgr2=0x55a9587ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:30.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 shutdown_connections 2024-04-22T13:30:30.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 1 -- 172.21.15.5:0/2008777302 wait complete. 2024-04-22T13:30:30.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.969+0000 7fb0297f5740 20 client.17028 trim_cache size 0 max 16384 2024-04-22T13:30:30.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.982+0000 7fb0297f5740 1 Processor -- start 2024-04-22T13:30:30.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.982+0000 7fb0297f5740 1 -- start start 2024-04-22T13:30:30.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.982+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a9587c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b1e80 0x55a9587c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0297f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201adb10 con 0x55a9587baa00 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0297f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201af650 con 0x55a9587b1e80 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0297f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201ae1e0 con 0x55a9587b14d0 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36172/0 (socket says 172.21.15.5:36172) 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 -- 172.21.15.5:0/4117381251 learned_addr learned my addr 172.21.15.5:0/4117381251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a9587c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0248ca640 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b1e80 0x55a9587c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 -- 172.21.15.5:0/4117381251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 msgr2=0x55a9587c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a9587c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 -- 172.21.15.5:0/4117381251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b1e80 msgr2=0x55a9587c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b1e80 0x55a9587c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:30.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 -- 172.21.15.5:0/4117381251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9587b1c20 con 0x55a9587baa00 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0248ca640 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b1e80 0x55a9587c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587d2940 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb010003c10 tx=0x7fb010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6424481a83641620 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0167fc640 1 -- 172.21.15.5:0/4117381251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01002d040 con 0x55a9587baa00 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0167fc640 1 -- 172.21.15.5:0/4117381251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0100120b0 con 0x55a9587baa00 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.983+0000 7fb0167fc640 1 -- 172.21.15.5:0/4117381251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010010900 con 0x55a9587baa00 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 -- 172.21.15.5:0/4117381251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 msgr2=0x55a9587d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587d2940 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb010003c10 tx=0x7fb010004a40 comp rx=0 tx=0).stop 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 -- 172.21.15.5:0/4117381251 shutdown_connections 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587baa00 0x55a9587d2940 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587b1e80 0x55a9587c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4117381251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a9587c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 -- 172.21.15.5:0/4117381251 >> 172.21.15.5:0/4117381251 conn(0x55a9587abba0 msgr2=0x55a9587b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 -- 172.21.15.5:0/4117381251 shutdown_connections 2024-04-22T13:30:31.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 -- 172.21.15.5:0/4117381251 wait complete. 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 -- start start 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.984+0000 7fb0297f5740 1 Processor -- start 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a95882dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 0x55a95882e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201aef80 con 0x55a9587b1e80 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201ae8b0 con 0x55a9587baa00 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201abb10 con 0x55a9587b14d0 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 10 client.0 _ll_register_callbacks cb 0x7fb010003eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 0x55a95882e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 0x55a95882e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36186/0 (socket says 172.21.15.5:36186) 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 -- 172.21.15.5:0/4014998987 learned_addr learned my addr 172.21.15.5:0/4014998987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a95882dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 -- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 msgr2=0x55a95882dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a95882dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 -- 172.21.15.5:0/4014998987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 msgr2=0x55a9587b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 -- 172.21.15.5:0/4014998987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9587b9de0 con 0x55a9587b1e80 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0248ca640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 0x55a95882e160 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb01800a5c0 tx=0x7fb01800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5dfbd4e90ca2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7faffffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7faffffff640 1 -- 172.21.15.5:0/4014998987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018032060 con 0x55a9587b1e80 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7faffffff640 1 -- 172.21.15.5:0/4014998987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb018018440 con 0x55a9587b1e80 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.985+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0100120b0 con 0x55a9587b1e80 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7faffffff640 1 -- 172.21.15.5:0/4014998987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb01801d040 con 0x55a9587b1e80 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 10 client.17034 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9588353e0 con 0x55a9587b1e80 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7faffffff640 1 -- 172.21.15.5:0/4014998987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0100120b0 con 0x55a9587b1e80 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7faffffff640 1 -- 172.21.15.5:0/4014998987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb01802c070 con 0x55a9587b1e80 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 10 client.17034 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 10 client.17034 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 20 client.17034 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a958835820 0x55a958837c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 4 client.17034 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a958723c10 con 0x55a958835820 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.986+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a958835820 0x55a958837c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb01ffff640 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a958835820 0x55a958837c10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=145c554ba256a0dc server_cookie=2c1156ec1c7e94b8 in_seq=0 out_seq=0 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7faffffff640 10 client.17034 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7faffffff640 1 -- 172.21.15.5:0/4014998987 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a958723c10 con 0x55a958835820 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7faffffff640 10 client.17034 handle_command_reply: tid=0 2024-04-22T13:30:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 client.17034 shutdown 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 msgr2=0x55a95882e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 0x55a95882e160 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb01800a5c0 tx=0x7fb01800a710 comp rx=0 tx=0).stop 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 shutdown_connections 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55a958835820 0x55a958837c10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9587b1e80 0x55a95882e160 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7faffffff640 0 client.17034 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9587baa00 0x55a9587b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 --2- 172.21.15.5:0/4014998987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a9587b14d0 0x55a95882dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 >> 172.21.15.5:0/4014998987 conn(0x55a9587abba0 msgr2=0x55a9587acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 shutdown_connections 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 1 -- 172.21.15.5:0/4014998987 wait complete. 2024-04-22T13:30:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.987+0000 7fb0297f5740 20 client.17034 trim_cache size 0 max 16384 2024-04-22T13:30:31.005 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb00c0a3d20 msgr2=0x7fb00c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb00c0a3d20 0x7fb00c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb008005940 tx=0x7fb008005a20 comp rx=0 tx=0).stop 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 msgr2=0x7fb02019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 0x7fb02019d440 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb008009230 tx=0x7fb0080045e0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 shutdown_connections 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb02019cf90 0x7fb02019d440 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb00c0a3d20 0x7fb00c0a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb020107250 0x7fb020198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 --2- 172.21.15.5:0/1875219815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb020102180 0x7fb020198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 >> 172.21.15.5:0/1875219815 conn(0x7fb0200fda00 msgr2=0x7fb0200ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 shutdown_connections 2024-04-22T13:30:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:30.988+0000 7fb026b2e640 1 -- 172.21.15.5:0/1875219815 wait complete. 2024-04-22T13:30:31.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:30 smithi005 ceph-mon[28006]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 672 op/s 2024-04-22T13:30:31.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:30:31.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:30 smithi049 ceph-mon[31379]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 672 op/s 2024-04-22T13:30:31.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:30 smithi107 ceph-mon[31859]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 MiB/s wr, 672 op/s 2024-04-22T13:30:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 -- 172.21.15.5:0/2517100775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66481011d0 msgr2=0x7f664810edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 --2- 172.21.15.5:0/2517100775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66481011d0 0x7f664810edd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6640009930 tx=0x7f664002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 -- 172.21.15.5:0/2517100775 shutdown_connections 2024-04-22T13:30:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 --2- 172.21.15.5:0/2517100775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66481011d0 0x7f664810edd0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 --2- 172.21.15.5:0/2517100775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648100810 0x7f6648100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 --2- 172.21.15.5:0/2517100775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6648108150 0x7f6648108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 -- 172.21.15.5:0/2517100775 >> 172.21.15.5:0/2517100775 conn(0x7f6648076170 msgr2=0x7f6648076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 -- 172.21.15.5:0/2517100775 shutdown_connections 2024-04-22T13:30:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.116+0000 7f664d552640 1 -- 172.21.15.5:0/2517100775 wait complete. 2024-04-22T13:30:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 Processor -- start 2024-04-22T13:30:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 -- start start 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648100810 0x7f6648198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66481011d0 0x7f6648198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 0x7f664819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6648112200 con 0x7f6648108150 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6648112080 con 0x7f66481011d0 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f664d552640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6648112380 con 0x7f6648100810 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 0x7f664819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 0x7f664819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36210/0 (socket says 172.21.15.5:36210) 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 -- 172.21.15.5:0/1625995172 learned_addr learned my addr 172.21.15.5:0/1625995172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648100810 0x7f6648198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 -- 172.21.15.5:0/1625995172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648100810 msgr2=0x7f6648198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648100810 0x7f6648198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 -- 172.21.15.5:0/1625995172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66481011d0 msgr2=0x7f6648198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66481011d0 0x7f6648198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.117+0000 7f66477fe640 1 -- 172.21.15.5:0/1625995172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664819d700 con 0x7f6648108150 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.118+0000 7f66477fe640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 0x7f664819cfd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6640005940 tx=0x7f6640004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.118+0000 7f662ffff640 1 -- 172.21.15.5:0/1625995172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640043020 con 0x7f6648108150 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.118+0000 7f662ffff640 1 -- 172.21.15.5:0/1625995172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6640004440 con 0x7f6648108150 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.118+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664819d990 con 0x7f6648108150 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.118+0000 7f662ffff640 1 -- 172.21.15.5:0/1625995172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640003bb0 con 0x7f6648108150 2024-04-22T13:30:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.118+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66481a5280 con 0x7f6648108150 2024-04-22T13:30:31.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f662ffff640 1 -- 172.21.15.5:0/1625995172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6640046020 con 0x7f6648108150 2024-04-22T13:30:31.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 Processor -- start 2024-04-22T13:30:31.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 -- start start 2024-04-22T13:30:31.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x56026674c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266755430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x560266757e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602666b11f0 con 0x56026674ce80 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56026664e4b0 con 0x56026674c4d0 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.120+0000 7f6650228740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5602666ad1b0 con 0x560266755a00 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266755430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266755430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36220/0 (socket says 172.21.15.5:36220) 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 -- 172.21.15.5:0/1656605383 learned_addr learned my addr 172.21.15.5:0/1656605383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f66467fc640 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x56026674c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f662ffff640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66300a3d20 0x7f66300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f66467fc640 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x560266757e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 -- 172.21.15.5:0/1656605383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 msgr2=0x560266757e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x560266757e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 -- 172.21.15.5:0/1656605383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 msgr2=0x56026674c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x56026674c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 -- 172.21.15.5:0/1656605383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602666401b0 con 0x56026674ce80 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f66467fc640 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x560266757e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f66477fe640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66300a3d20 0x7f66300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f662ffff640 1 -- 172.21.15.5:0/1625995172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f66400f29b0 con 0x7f6648108150 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266755430 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6638010370 tx=0x7f6638010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f075390d816a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f662d7fa640 1 -- 172.21.15.5:0/1656605383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638016070 con 0x56026674ce80 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f662d7fa640 1 -- 172.21.15.5:0/1656605383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638004d20 con 0x56026674ce80 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f662d7fa640 1 -- 172.21.15.5:0/1656605383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380053c0 con 0x56026674ce80 2024-04-22T13:30:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.121+0000 7f66477fe640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66300a3d20 0x7f66300a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66400a7fd0 tx=0x7f66400904d0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 -- 172.21.15.5:0/1656605383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 msgr2=0x560266755430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266755430 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6638010370 tx=0x7f6638010830 comp rx=0 tx=0).stop 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 -- 172.21.15.5:0/1656605383 shutdown_connections 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266755430 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x560266757e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 --2- 172.21.15.5:0/1656605383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x56026674c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 -- 172.21.15.5:0/1656605383 >> 172.21.15.5:0/1656605383 conn(0x560266746ba0 msgr2=0x560266746f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 -- 172.21.15.5:0/1656605383 shutdown_connections 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 -- 172.21.15.5:0/1656605383 wait complete. 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 -- start start 2024-04-22T13:30:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.122+0000 7f6650228740 1 Processor -- start 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x5602667552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x560266752680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 0x560266752c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560266758970 con 0x560266755a00 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56026666a500 con 0x56026674c4d0 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560266758af0 con 0x56026674ce80 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6650228740 10 client.0 _ll_register_callbacks cb 0x560266638b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 0x560266752c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x560266752680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x560266752680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:37432/0 (socket says 172.21.15.5:37432) 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 0x560266752c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36236/0 (socket says 172.21.15.5:36236) 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f6646ffd640 1 -- 172.21.15.5:0/1274031000 learned_addr learned my addr 172.21.15.5:0/1274031000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x5602667552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 -- 172.21.15.5:0/1274031000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 msgr2=0x560266752680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x560266752680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 -- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 msgr2=0x5602667552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x5602667552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 -- 172.21.15.5:0/1274031000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602663c33f0 con 0x560266755a00 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f66467fc640 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 0x560266752c00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f66481076c0 tx=0x7f663c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8239028550c71a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f662aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.123+0000 7f662aff5640 1 -- 172.21.15.5:0/1274031000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c010ce0 con 0x560266755a00 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f662aff5640 1 -- 172.21.15.5:0/1274031000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f663c004020 con 0x560266755a00 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6638004d20 con 0x560266755a00 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f662aff5640 1 -- 172.21.15.5:0/1274031000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f663c024050 con 0x560266755a00 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 10 client.17046 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56026648d7d0 con 0x560266755a00 2024-04-22T13:30:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f662aff5640 1 -- 172.21.15.5:0/1274031000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638004d20 con 0x560266755a00 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f662aff5640 1 -- 172.21.15.5:0/1274031000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f663c01c020 con 0x560266755a00 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 10 client.17046 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 10 client.17046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 20 client.17046 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560266765c40 0x560266768080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 4 client.17046 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560266650930 con 0x560266765c40 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.124+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560266765c40 0x560266768080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.125+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560266765c40 0x560266768080 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfd2b366c4b862ba server_cookie=628806c516ca7142 in_seq=0 out_seq=0 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.125+0000 7f662aff5640 10 client.17046 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f662aff5640 1 -- 172.21.15.5:0/1274031000 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560266650930 con 0x560266765c40 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f662aff5640 10 client.17046 handle_command_reply: tid=0 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 client.17046 shutdown 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 msgr2=0x560266752c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 0x560266752c00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f66481076c0 tx=0x7f663c005c70 comp rx=0 tx=0).stop 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 shutdown_connections 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560266765c40 0x560266768080 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560266755a00 0x560266752c00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f662aff5640 0 client.17046 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x560266752680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 --2- 172.21.15.5:0/1274031000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x5602667552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 >> 172.21.15.5:0/1274031000 conn(0x560266746ba0 msgr2=0x5602667488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 shutdown_connections 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 1 -- 172.21.15.5:0/1274031000 wait complete. 2024-04-22T13:30:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.126+0000 7f6650228740 20 client.17046 trim_cache size 0 max 16384 2024-04-22T13:30:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 Processor -- start 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 -- start start 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x560266761860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266754db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x5602667552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66481adab0 con 0x56026674ce80 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66481af5f0 con 0x56026674c4d0 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6650228740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66481ae180 con 0x560266755a00 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.139+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266754db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266754db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36238/0 (socket says 172.21.15.5:36238) 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 -- 172.21.15.5:0/1423751313 learned_addr learned my addr 172.21.15.5:0/1423751313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f66467fc640 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x560266761860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f66467fc640 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x5602667552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 -- 172.21.15.5:0/1423751313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 msgr2=0x5602667552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x5602667552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 -- 172.21.15.5:0/1423751313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 msgr2=0x560266761860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x560266761860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 -- 172.21.15.5:0/1423751313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026674cc20 con 0x56026674ce80 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f66467fc640 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x5602667552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f6646ffd640 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266754db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6638000f80 tx=0x7f6638004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe7a68f9b3547af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f662cff9640 1 -- 172.21.15.5:0/1423751313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638030070 con 0x56026674ce80 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f662cff9640 1 -- 172.21.15.5:0/1423751313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663802a900 con 0x56026674ce80 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.140+0000 7f662cff9640 1 -- 172.21.15.5:0/1423751313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638016040 con 0x56026674ce80 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- 172.21.15.5:0/1423751313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 msgr2=0x560266754db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266754db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6638000f80 tx=0x7f6638004d00 comp rx=0 tx=0).stop 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- 172.21.15.5:0/1423751313 shutdown_connections 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674ce80 0x560266754db0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560266755a00 0x5602667552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- 172.21.15.5:0/1423751313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56026674c4d0 0x560266761860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- 172.21.15.5:0/1423751313 >> 172.21.15.5:0/1423751313 conn(0x560266746ba0 msgr2=0x560266745c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- 172.21.15.5:0/1423751313 shutdown_connections 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- 172.21.15.5:0/1423751313 wait complete. 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- start start 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 Processor -- start 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 0x56026674f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x56026674f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560266755a00 0x56026674fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66481aef20 con 0x56026674c4d0 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f66481ae850 con 0x560266755a00 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66481aba90 con 0x56026674ce80 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.141+0000 7f6650228740 10 client.0 _ll_register_callbacks cb 0x7f663c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 0x56026674f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 0x56026674f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36242/0 (socket says 172.21.15.5:36242) 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 -- 172.21.15.5:0/110240358 learned_addr learned my addr 172.21.15.5:0/110240358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f6646ffd640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x56026674f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560266755a00 0x56026674fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 -- 172.21.15.5:0/110240358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 msgr2=0x56026674f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x56026674f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 -- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560266755a00 msgr2=0x56026674fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560266755a00 0x56026674fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 -- 172.21.15.5:0/110240358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602667cb210 con 0x56026674c4d0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f66467fc640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 0x56026674f390 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f663c003e70 tx=0x7f663c00ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162762098e8d8b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f662aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f662aff5640 1 -- 172.21.15.5:0/110240358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0037f0 con 0x56026674c4d0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f662aff5640 1 -- 172.21.15.5:0/110240358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f663c003990 con 0x56026674c4d0 2024-04-22T13:30:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663802a900 con 0x56026674c4d0 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f662aff5640 1 -- 172.21.15.5:0/110240358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f663c003ea0 con 0x56026674c4d0 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f6650228740 10 client.17052 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.142+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602667cc4f0 con 0x56026674c4d0 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f662aff5640 1 -- 172.21.15.5:0/110240358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663802a900 con 0x56026674c4d0 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f662aff5640 1 -- 172.21.15.5:0/110240358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f663c01c040 con 0x56026674c4d0 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6650228740 10 client.17052 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6650228740 10 client.17052 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6650228740 20 client.17052 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6650228740 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5602667ccc10 0x5602667cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6650228740 4 client.17052 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602666bec10 con 0x5602667ccc10 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6646ffd640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5602667ccc10 0x5602667cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f662aff5640 10 client.17052 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.143+0000 7f6646ffd640 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5602667ccc10 0x5602667cf030 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f479798b545351bb server_cookie=1cebb1b878132826 in_seq=0 out_seq=0 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f662aff5640 1 -- 172.21.15.5:0/110240358 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602666bec10 con 0x5602667ccc10 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f662aff5640 10 client.17052 handle_command_reply: tid=0 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 client.17052 shutdown 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 msgr2=0x56026674f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 0x56026674f390 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f663c003e70 tx=0x7f663c00ad50 comp rx=0 tx=0).stop 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 shutdown_connections 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5602667ccc10 0x5602667cf030 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560266755a00 0x56026674fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f662aff5640 0 client.17052 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56026674ce80 0x56026674f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 --2- 172.21.15.5:0/110240358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56026674c4d0 0x56026674f390 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 >> 172.21.15.5:0/110240358 conn(0x560266746ba0 msgr2=0x560266747ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 shutdown_connections 2024-04-22T13:30:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 1 -- 172.21.15.5:0/110240358 wait complete. 2024-04-22T13:30:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f6650228740 20 client.17052 trim_cache size 0 max 16384 2024-04-22T13:30:31.147 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66300a3d20 msgr2=0x7f66300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66300a3d20 0x7f66300a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66400a7fd0 tx=0x7f66400904d0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 msgr2=0x7f664819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 0x7f664819cfd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6640005940 tx=0x7f6640004040 comp rx=0 tx=0).stop 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 shutdown_connections 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6648108150 0x7f664819cfd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66300a3d20 0x7f66300a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66481011d0 0x7f6648198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 --2- 172.21.15.5:0/1625995172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648100810 0x7f6648198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.144+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 >> 172.21.15.5:0/1625995172 conn(0x7f6648076170 msgr2=0x7f66480ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.145+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 shutdown_connections 2024-04-22T13:30:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.145+0000 7f664d552640 1 -- 172.21.15.5:0/1625995172 wait complete. 2024-04-22T13:30:31.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/3833016279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c0103b00 msgr2=0x7fc1c010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/3833016279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c0103b00 0x7fc1c010bfe0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009860 tx=0x7fc1b402bca0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/3833016279 shutdown_connections 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/3833016279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c0103b00 0x7fc1c010bfe0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/3833016279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1c010c520 0x7fc1c010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/3833016279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1c01031e0 0x7fc1c01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/3833016279 >> 172.21.15.5:0/3833016279 conn(0x7fc1c0077f40 msgr2=0x7fc1c0078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/3833016279 shutdown_connections 2024-04-22T13:30:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.260+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/3833016279 wait complete. 2024-04-22T13:30:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 Processor -- start 2024-04-22T13:30:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 -- start start 2024-04-22T13:30:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 0x7fc1c019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1c0103b00 0x7fc1c019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1c010c520 0x7fc1c0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01121b0 con 0x7fc1c01031e0 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0112030 con 0x7fc1c0103b00 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1c5ad2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0112330 con 0x7fc1c010c520 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 0x7fc1c019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 0x7fc1c019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36258/0 (socket says 172.21.15.5:36258) 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/918607179 learned_addr learned my addr 172.21.15.5:0/918607179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1beffd640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1c0103b00 0x7fc1c019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bffff640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1c010c520 0x7fc1c0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/918607179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1c010c520 msgr2=0x7fc1c0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1c010c520 0x7fc1c0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/918607179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1c0103b00 msgr2=0x7fc1c019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1c0103b00 0x7fc1c019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.261+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/918607179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c0198d10 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.262+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 0x7fc1c019e3e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000e9f0 tx=0x7fc1b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.262+0000 7fc1bcff9640 1 -- 172.21.15.5:0/918607179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b000cd80 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.262+0000 7fc1bcff9640 1 -- 172.21.15.5:0/918607179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b00108e0 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.262+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c0199000 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.262+0000 7fc1bcff9640 1 -- 172.21.15.5:0/918607179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0018980 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.262+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01a52a0 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 Processor -- start 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 -- start start 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1bcff9640 1 -- 172.21.15.5:0/918607179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc1b0010430 con 0x7fc1c01031e0 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3eda7a00 0x563e3eda9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e3ed031f0 con 0x563e3ed9ee80 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563e3eca04b0 con 0x563e3eda7a00 2024-04-22T13:30:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563e3ecff1b0 con 0x563e3ed9e4d0 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36264/0 (socket says 172.21.15.5:36264) 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/3883213066 learned_addr learned my addr 172.21.15.5:0/3883213066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.265+0000 7fc1beffd640 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3eda7a00 0x563e3eda9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/3883213066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 msgr2=0x563e3ed9e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/3883213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3eda7a00 msgr2=0x563e3eda9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3eda7a00 0x563e3eda9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bcff9640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1a80a3d20 0x7fc1a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bf7fe640 1 -- 172.21.15.5:0/3883213066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3ec921b0 con 0x563e3ed9ee80 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bffff640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1a80a3d20 0x7fc1a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda7430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00ce1a0 tx=0x7fc1b0098e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e40f03739b3a04b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1bcff9640 1 -- 172.21.15.5:0/918607179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc1b00d0ab0 con 0x7fc1c01031e0 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1a5ffb640 1 -- 172.21.15.5:0/3883213066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b009c900 con 0x563e3ed9ee80 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1a5ffb640 1 -- 172.21.15.5:0/3883213066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b0010720 con 0x563e3ed9ee80 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.266+0000 7fc1a5ffb640 1 -- 172.21.15.5:0/3883213066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b009c490 con 0x563e3ed9ee80 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 -- 172.21.15.5:0/3883213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 msgr2=0x563e3eda7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1bffff640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1a80a3d20 0x7fc1a80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8005fd0 tx=0x7fc1b8010040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda7430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00ce1a0 tx=0x7fc1b0098e90 comp rx=0 tx=0).stop 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 -- 172.21.15.5:0/3883213066 shutdown_connections 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda7430 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3eda7a00 0x563e3eda9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/3883213066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 -- 172.21.15.5:0/3883213066 >> 172.21.15.5:0/3883213066 conn(0x563e3ed98ba0 msgr2=0x563e3ed98f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 -- 172.21.15.5:0/3883213066 shutdown_connections 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 -- 172.21.15.5:0/3883213066 wait complete. 2024-04-22T13:30:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 -- start start 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.267+0000 7fc1c87a4740 1 Processor -- start 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9e4d0 0x563e3eda7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3eda7a00 0x563e3eda4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e3edaa8b0 con 0x563e3ed9ee80 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563e3ecbc500 con 0x563e3ed9e4d0 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563e3edaaa30 con 0x563e3eda7a00 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 10 client.0 _ll_register_callbacks cb 0x563e3ec8ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36266/0 (socket says 172.21.15.5:36266) 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 -- 172.21.15.5:0/1235306225 learned_addr learned my addr 172.21.15.5:0/1235306225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1bffff640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3eda7a00 0x563e3eda4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9e4d0 0x563e3eda7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 -- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3eda7a00 msgr2=0x563e3eda4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3eda7a00 0x563e3eda4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 -- 172.21.15.5:0/1235306225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9e4d0 msgr2=0x563e3eda7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9e4d0 0x563e3eda7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 -- 172.21.15.5:0/1235306225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3ea153f0 con 0x563e3ed9ee80 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1beffd640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda45e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009780 tx=0x7fc1b4034f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba35b741507785e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc19effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc19effd640 1 -- 172.21.15.5:0/1235306225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4039040 con 0x563e3ed9ee80 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc19effd640 1 -- 172.21.15.5:0/1235306225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc1b4004450 con 0x563e3ed9ee80 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.268+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e3eca2930 con 0x563e3ed9ee80 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc19effd640 1 -- 172.21.15.5:0/1235306225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1b404f050 con 0x563e3ed9ee80 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 10 client.17064 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3eadf7d0 con 0x563e3ed9ee80 2024-04-22T13:30:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc19effd640 1 -- 172.21.15.5:0/1235306225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563e3eca2930 con 0x563e3ed9ee80 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc19effd640 1 -- 172.21.15.5:0/1235306225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc1b40035f0 con 0x563e3ed9ee80 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 10 client.17064 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 10 client.17064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 20 client.17064 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3edb7ab0 0x563e3edb9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 4 client.17064 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3ed10c10 con 0x563e3edb7ab0 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.269+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3edb7ab0 0x563e3edb9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3edb7ab0 0x563e3edb9f20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c44e08adad02a480 server_cookie=5dda2a114498846d in_seq=0 out_seq=0 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc19effd640 10 client.17064 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc19effd640 1 -- 172.21.15.5:0/1235306225 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e3ed10c10 con 0x563e3edb7ab0 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc19effd640 10 client.17064 handle_command_reply: tid=0 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 client.17064 shutdown 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 msgr2=0x563e3eda45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda45e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009780 tx=0x7fc1b4034f30 comp rx=0 tx=0).stop 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 shutdown_connections 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3edb7ab0 0x563e3edb9f20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3ed9ee80 0x563e3eda45e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3eda7a00 0x563e3eda4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc19effd640 0 client.17064 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.270+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1235306225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9e4d0 0x563e3eda7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.271+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 >> 172.21.15.5:0/1235306225 conn(0x563e3ed98ba0 msgr2=0x563e3ed9a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.271+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 shutdown_connections 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.271+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1235306225 wait complete. 2024-04-22T13:30:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.271+0000 7fc1c87a4740 20 client.17064 trim_cache size 0 max 16384 2024-04-22T13:30:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 Processor -- start 2024-04-22T13:30:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 -- start start 2024-04-22T13:30:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3ed9a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3eda6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01adb80 con 0x563e3eda7a00 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01af6c0 con 0x563e3ed9ee80 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01ae250 con 0x563e3ed9e4d0 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3eda6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3eda6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36278/0 (socket says 172.21.15.5:36278) 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 -- 172.21.15.5:0/1849880976 learned_addr learned my addr 172.21.15.5:0/1849880976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1beffd640 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3ed9a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 -- 172.21.15.5:0/1849880976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 msgr2=0x563e3ed9faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 -- 172.21.15.5:0/1849880976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 msgr2=0x563e3ed9a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.284+0000 7fc1bffff640 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3ed9a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1bffff640 1 -- 172.21.15.5:0/1849880976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3eda3590 con 0x563e3eda7a00 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1bffff640 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3eda6d10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8003f50 tx=0x7fc1b800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb623ca1efad078 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1a57fa640 1 -- 172.21.15.5:0/1849880976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b801c730 con 0x563e3eda7a00 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1a57fa640 1 -- 172.21.15.5:0/1849880976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b802b070 con 0x563e3eda7a00 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1a57fa640 1 -- 172.21.15.5:0/1849880976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800de20 con 0x563e3eda7a00 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1849880976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 msgr2=0x563e3eda6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3eda6d10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8003f50 tx=0x7fc1b800ef90 comp rx=0 tx=0).stop 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1849880976 shutdown_connections 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3eda6d10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3ed9a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/1849880976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3ed9faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.285+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1849880976 >> 172.21.15.5:0/1849880976 conn(0x563e3ed98ba0 msgr2=0x563e3ed99ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1849880976 shutdown_connections 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 -- 172.21.15.5:0/1849880976 wait complete. 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 -- start start 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 Processor -- start 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3edafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3edb01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3edacc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01aeff0 con 0x563e3eda7a00 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01ae920 con 0x563e3ed9ee80 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01abb60 con 0x563e3ed9e4d0 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1c87a4740 10 client.0 _ll_register_callbacks cb 0x7fc1b40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.286+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3edacc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3edacc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36284/0 (socket says 172.21.15.5:36284) 2024-04-22T13:30:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3edafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 -- 172.21.15.5:0/2919097285 learned_addr learned my addr 172.21.15.5:0/2919097285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1beffd640 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3edb01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 -- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 msgr2=0x563e3edafcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3edafcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 -- 172.21.15.5:0/2919097285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 msgr2=0x563e3edb01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3edb01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 -- 172.21.15.5:0/2919097285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3edad320 con 0x563e3eda7a00 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1bffff640 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3edacc80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d880 tx=0x7fc1b8003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa8a3ea12a3e79b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc19effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc19effd640 1 -- 172.21.15.5:0/2919097285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800dd90 con 0x563e3eda7a00 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc19effd640 1 -- 172.21.15.5:0/2919097285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc1b801c530 con 0x563e3eda7a00 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b802b070 con 0x563e3eda7a00 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc19effd640 1 -- 172.21.15.5:0/2919097285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1b801c730 con 0x563e3eda7a00 2024-04-22T13:30:31.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1c87a4740 10 client.17070 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3ee1ea10 con 0x563e3eda7a00 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.287+0000 7fc19effd640 1 -- 172.21.15.5:0/2919097285 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b802b070 con 0x563e3eda7a00 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc19effd640 1 -- 172.21.15.5:0/2919097285 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc1b800b040 con 0x563e3eda7a00 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 10 client.17070 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 10 client.17070 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 20 client.17070 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3ee1f0b0 0x563e3ee214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 4 client.17070 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e3ed0aa80 con 0x563e3ee1f0b0 2024-04-22T13:30:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3ee1f0b0 0x563e3ee214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc19effd640 10 client.17070 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1bf7fe640 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3ee1f0b0 0x563e3ee214d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=189eacbf504448c3 server_cookie=7f361eab77586ace in_seq=0 out_seq=0 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc19effd640 1 -- 172.21.15.5:0/2919097285 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e3ed0aa80 con 0x563e3ee1f0b0 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc19effd640 10 client.17070 handle_command_reply: tid=0 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.288+0000 7fc1c87a4740 1 client.17070 shutdown 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 msgr2=0x563e3edacc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3edacc80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d880 tx=0x7fc1b8003cc0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 shutdown_connections 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x563e3ee1f0b0 0x563e3ee214d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3eda7a00 0x563e3edacc80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563e3ed9ee80 0x563e3edb01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 --2- 172.21.15.5:0/2919097285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e3ed9e4d0 0x563e3edafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 >> 172.21.15.5:0/2919097285 conn(0x563e3ed98ba0 msgr2=0x563e3ed9f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc19effd640 0 client.17070 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:30:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 shutdown_connections 2024-04-22T13:30:31.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 1 -- 172.21.15.5:0/2919097285 wait complete. 2024-04-22T13:30:31.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c87a4740 20 client.17070 trim_cache size 0 max 16384 2024-04-22T13:30:31.292 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1a80a3d20 msgr2=0x7fc1a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1a80a3d20 0x7fc1a80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8005fd0 tx=0x7fc1b8010040 comp rx=0 tx=0).stop 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 msgr2=0x7fc1c019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 0x7fc1c019e3e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000e9f0 tx=0x7fc1b000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 shutdown_connections 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1a80a3d20 0x7fc1a80a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1c010c520 0x7fc1c0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1c0103b00 0x7fc1c019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 --2- 172.21.15.5:0/918607179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c01031e0 0x7fc1c019e3e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 >> 172.21.15.5:0/918607179 conn(0x7fc1c0077f40 msgr2=0x7fc1c00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.289+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 shutdown_connections 2024-04-22T13:30:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.290+0000 7fc1c5ad2640 1 -- 172.21.15.5:0/918607179 wait complete. 2024-04-22T13:30:31.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 -- 172.21.15.5:0/1723687806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890103400 msgr2=0x7f589010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 --2- 172.21.15.5:0/1723687806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890103400 0x7f589010de00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5888009cd0 tx=0x7f588802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 -- 172.21.15.5:0/1723687806 shutdown_connections 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 --2- 172.21.15.5:0/1723687806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890103400 0x7f589010de00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 --2- 172.21.15.5:0/1723687806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890102a60 0x7f5890102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 --2- 172.21.15.5:0/1723687806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5890101180 0x7f5890101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 -- 172.21.15.5:0/1723687806 >> 172.21.15.5:0/1723687806 conn(0x7f5890077f70 msgr2=0x7f5890078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 -- 172.21.15.5:0/1723687806 shutdown_connections 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.405+0000 7f58985f4640 1 -- 172.21.15.5:0/1723687806 wait complete. 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 Processor -- start 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 -- start start 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5890101180 0x7f589019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 0x7f589019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890103400 0x7f5890198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5890111220 con 0x7f5890102a60 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58901110a0 con 0x7f5890103400 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f58985f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58901113a0 con 0x7f5890101180 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 0x7f589019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5896390640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5890101180 0x7f589019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 0x7f589019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36314/0 (socket says 172.21.15.5:36314) 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 -- 172.21.15.5:0/3539557693 learned_addr learned my addr 172.21.15.5:0/3539557693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5896b91640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890103400 0x7f5890198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 -- 172.21.15.5:0/3539557693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5890101180 msgr2=0x7f589019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5890101180 0x7f589019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 -- 172.21.15.5:0/3539557693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890103400 msgr2=0x7f5890198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890103400 0x7f5890198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5895b8f640 1 -- 172.21.15.5:0/3539557693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5890198d80 con 0x7f5890102a60 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.406+0000 7f5896b91640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890103400 0x7f5890198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.407+0000 7f5895b8f640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 0x7f589019e990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f588000b4a0 tx=0x7f588000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.407+0000 7f58877fe640 1 -- 172.21.15.5:0/3539557693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880014070 con 0x7f5890102a60 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.407+0000 7f58877fe640 1 -- 172.21.15.5:0/3539557693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5880010ac0 con 0x7f5890102a60 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.407+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5890199070 con 0x7f5890102a60 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.407+0000 7f58877fe640 1 -- 172.21.15.5:0/3539557693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880017c30 con 0x7f5890102a60 2024-04-22T13:30:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.407+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58901a5310 con 0x7f5890102a60 2024-04-22T13:30:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.409+0000 7f58877fe640 1 -- 172.21.15.5:0/3539557693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5880017070 con 0x7f5890102a60 2024-04-22T13:30:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 Processor -- start 2024-04-22T13:30:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 -- start start 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f78790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 0x55e041f81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e041edd1f0 con 0x55e041f78d60 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e041e7a4b0 con 0x55e041f783b0 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f589b1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e041ed91b0 con 0x55e041f81e60 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f58877fe640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f587c0a3da0 0x7f587c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f5896390640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 0x55e041f81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f5896390640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 0x55e041f81890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36328/0 (socket says 172.21.15.5:36328) 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f5896390640 1 -- 172.21.15.5:0/2675770964 learned_addr learned my addr 172.21.15.5:0/2675770964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f5895b8f640 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f5896b91640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f587c0a3da0 0x7f587c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.410+0000 7f5896b91640 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f78790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896390640 1 -- 172.21.15.5:0/2675770964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 msgr2=0x55e041f822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896390640 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896390640 1 -- 172.21.15.5:0/2675770964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 msgr2=0x55e041f78790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896390640 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f78790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896390640 1 -- 172.21.15.5:0/2675770964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e041e6c1b0 con 0x55e041f78d60 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896b91640 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f58877fe640 1 -- 172.21.15.5:0/3539557693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f58800cfcf0 con 0x7f5890102a60 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896390640 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 0x55e041f81890 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f588c00b760 tx=0x7f588c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a55ab7693ae815 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5896b91640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f587c0a3da0 0x7f587c0a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5888005cc0 tx=0x7f588802e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5884bfe640 1 -- 172.21.15.5:0/2675770964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c003dd0 con 0x55e041f78d60 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5884bfe640 1 -- 172.21.15.5:0/2675770964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c015070 con 0x55e041f78d60 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.411+0000 7f5884bfe640 1 -- 172.21.15.5:0/2675770964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c004d90 con 0x55e041f78d60 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 -- 172.21.15.5:0/2675770964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 msgr2=0x55e041f81890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 0x55e041f81890 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f588c00b760 tx=0x7f588c00bc20 comp rx=0 tx=0).stop 2024-04-22T13:30:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 -- 172.21.15.5:0/2675770964 shutdown_connections 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f78d60 0x55e041f81890 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2675770964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 -- 172.21.15.5:0/2675770964 >> 172.21.15.5:0/2675770964 conn(0x55e041f72ba0 msgr2=0x55e041f72f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 -- 172.21.15.5:0/2675770964 shutdown_connections 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 -- 172.21.15.5:0/2675770964 wait complete. 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 -- start start 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.412+0000 7f589b1e8740 1 Processor -- start 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f88250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f78d60 0x55e041f855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 0x55e041f85b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e041f88da0 con 0x55e041f81e60 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e041e96500 con 0x55e041f783b0 2024-04-22T13:30:31.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e041f88f20 con 0x55e041f78d60 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 10 client.0 _ll_register_callbacks cb 0x55e041e64b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 0x55e041f85b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 0x55e041f85b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36332/0 (socket says 172.21.15.5:36332) 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 -- 172.21.15.5:0/1755098512 learned_addr learned my addr 172.21.15.5:0/1755098512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5895b8f640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f78d60 0x55e041f855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896390640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f88250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 -- 172.21.15.5:0/1755098512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f78d60 msgr2=0x55e041f855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f78d60 0x55e041f855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 -- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 msgr2=0x55e041f88250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f88250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 -- 172.21.15.5:0/1755098512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e041bef3f0 con 0x55e041f81e60 2024-04-22T13:30:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5896b91640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 0x55e041f85b20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5888006010 tx=0x7f588802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e3c9a9920aac88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5879ffb640 1 -- 172.21.15.5:0/1755098512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588804e020 con 0x55e041f81e60 2024-04-22T13:30:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f5879ffb640 1 -- 172.21.15.5:0/1755098512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5888004790 con 0x55e041f81e60 2024-04-22T13:30:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.413+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f588c015070 con 0x55e041f81e60 2024-04-22T13:30:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f5879ffb640 1 -- 172.21.15.5:0/1755098512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5888044030 con 0x55e041f81e60 2024-04-22T13:30:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 10 client.17082 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e041cb97d0 con 0x55e041f81e60 2024-04-22T13:30:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f5879ffb640 1 -- 172.21.15.5:0/1755098512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c015070 con 0x55e041f81e60 2024-04-22T13:30:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f5879ffb640 1 -- 172.21.15.5:0/1755098512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f588803f040 con 0x55e041f81e60 2024-04-22T13:30:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 10 client.17082 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 10 client.17082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:30:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 20 client.17082 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041f95f90 0x55e041f98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 4 client.17082 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e041e7c930 con 0x55e041f95f90 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.414+0000 7f5896390640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041f95f90 0x55e041f98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f5896390640 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041f95f90 0x55e041f98380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad58143e2dd766c server_cookie=265721a9ab95fd in_seq=0 out_seq=0 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f5879ffb640 10 client.17082 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f5879ffb640 1 -- 172.21.15.5:0/1755098512 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e041e7c930 con 0x55e041f95f90 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f5879ffb640 10 client.17082 handle_command_reply: tid=0 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f589b1e8740 1 client.17082 shutdown 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 msgr2=0x55e041f85b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.415+0000 7f589b1e8740 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 0x55e041f85b20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5888006010 tx=0x7f588802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 shutdown_connections 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041f95f90 0x55e041f98380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f81e60 0x55e041f85b20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f78d60 0x55e041f855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 --2- 172.21.15.5:0/1755098512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f783b0 0x55e041f88250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f5879ffb640 0 client.17082 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 >> 172.21.15.5:0/1755098512 conn(0x55e041f72ba0 msgr2=0x55e041f748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 shutdown_connections 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 1 -- 172.21.15.5:0/1755098512 wait complete. 2024-04-22T13:30:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.416+0000 7f589b1e8740 20 client.17082 trim_cache size 0 max 16384 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 Processor -- start 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 -- start start 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f91cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ab8e0 con 0x55e041f783b0 2024-04-22T13:30:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ad420 con 0x55e041f78d60 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f589b1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58901abfb0 con 0x55e041f81e60 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f5896390640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f91cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f5896390640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f91cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36336/0 (socket says 172.21.15.5:36336) 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f5896390640 1 -- 172.21.15.5:0/2344368592 learned_addr learned my addr 172.21.15.5:0/2344368592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.429+0000 7f5896b91640 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5895b8f640 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5896390640 1 -- 172.21.15.5:0/2344368592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 msgr2=0x55e041f87d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5896390640 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5896390640 1 -- 172.21.15.5:0/2344368592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 msgr2=0x55e041f746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5896390640 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5896390640 1 -- 172.21.15.5:0/2344368592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e041f83b30 con 0x55e041f783b0 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f5896390640 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f91cd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f588c0049e0 tx=0x7f588c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edb9b4f2bf5a952 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f587bfff640 1 -- 172.21.15.5:0/2344368592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c003cf0 con 0x55e041f783b0 2024-04-22T13:30:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f587bfff640 1 -- 172.21.15.5:0/2344368592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c02d070 con 0x55e041f783b0 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.430+0000 7f587bfff640 1 -- 172.21.15.5:0/2344368592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c011640 con 0x55e041f783b0 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- 172.21.15.5:0/2344368592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 msgr2=0x55e041f91cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f91cd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f588c0049e0 tx=0x7f588c004970 comp rx=0 tx=0).stop 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- 172.21.15.5:0/2344368592 shutdown_connections 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2344368592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f91cd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- 172.21.15.5:0/2344368592 >> 172.21.15.5:0/2344368592 conn(0x55e041f72ba0 msgr2=0x55e041f73fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- 172.21.15.5:0/2344368592 shutdown_connections 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- 172.21.15.5:0/2344368592 wait complete. 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- start start 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 Processor -- start 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f87770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58901acd50 con 0x55e041f783b0 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ac680 con 0x55e041f78d60 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58901a98e0 con 0x55e041f81e60 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f589b1e8740 10 client.0 _ll_register_callbacks cb 0x7f5888032930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f5896390640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f5896390640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f87230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36342/0 (socket says 172.21.15.5:36342) 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f5896390640 1 -- 172.21.15.5:0/2588768022 learned_addr learned my addr 172.21.15.5:0/2588768022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.431+0000 7f5895b8f640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f87770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896b91640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896390640 1 -- 172.21.15.5:0/2588768022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 msgr2=0x55e041f87cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896390640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896390640 1 -- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 msgr2=0x55e041f87770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896390640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f87770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896390640 1 -- 172.21.15.5:0/2588768022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e041ffb420 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896b91640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5896390640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f87230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f588c028a40 tx=0x7f588c028a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefe944edb7f131f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5879ffb640 1 -- 172.21.15.5:0/2588768022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c036070 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f588c02d070 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5879ffb640 1 -- 172.21.15.5:0/2588768022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f588c0280f0 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5879ffb640 1 -- 172.21.15.5:0/2588768022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f588c042050 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f589b1e8740 10 client.17088 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e041ffc640 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.432+0000 7f5879ffb640 1 -- 172.21.15.5:0/2588768022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c02d070 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f5879ffb640 1 -- 172.21.15.5:0/2588768022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f588c011a30 con 0x55e041f783b0 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 10 client.17088 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 10 client.17088 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 20 client.17088 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041ffcb30 0x55e041ffef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 4 client.17088 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e041eeac10 con 0x55e041ffcb30 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f5895b8f640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041ffcb30 0x55e041ffef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f5879ffb640 10 client.17088 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f5895b8f640 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041ffcb30 0x55e041ffef50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2596810861890e2a server_cookie=2c75ac2e912041d8 in_seq=0 out_seq=0 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f5879ffb640 1 -- 172.21.15.5:0/2588768022 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e041eeac10 con 0x55e041ffcb30 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f5879ffb640 10 client.17088 handle_command_reply: tid=0 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 1 client.17088 shutdown 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 msgr2=0x55e041f87230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.433+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f87230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f588c028a40 tx=0x7f588c028a70 comp rx=0 tx=0).stop 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 shutdown_connections 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55e041ffcb30 0x55e041ffef50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e041f81e60 0x55e041f87cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f5879ffb640 0 client.17088 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:30:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e041f78d60 0x55e041f87770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 --2- 172.21.15.5:0/2588768022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e041f783b0 0x55e041f87230 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 >> 172.21.15.5:0/2588768022 conn(0x55e041f72ba0 msgr2=0x55e041f71c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 shutdown_connections 2024-04-22T13:30:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 1 -- 172.21.15.5:0/2588768022 wait complete. 2024-04-22T13:30:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f589b1e8740 20 client.17088 trim_cache size 0 max 16384 2024-04-22T13:30:31.436 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f587c0a3da0 msgr2=0x7f587c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f587c0a3da0 0x7f587c0a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5888005cc0 tx=0x7f588802e040 comp rx=0 tx=0).stop 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 msgr2=0x7f589019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 0x7f589019e990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f588000b4a0 tx=0x7f588000b960 comp rx=0 tx=0).stop 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 shutdown_connections 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890102a60 0x7f589019e990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f587c0a3da0 0x7f587c0a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5890103400 0x7f5890198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 --2- 172.21.15.5:0/3539557693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5890101180 0x7f589019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 >> 172.21.15.5:0/3539557693 conn(0x7f5890077f70 msgr2=0x7f589006a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 shutdown_connections 2024-04-22T13:30:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.434+0000 7f58985f4640 1 -- 172.21.15.5:0/3539557693 wait complete. 2024-04-22T13:30:31.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:30:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 -- 172.21.15.5:0/2611861423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d0102190 msgr2=0x7fe1d0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 --2- 172.21.15.5:0/2611861423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d0102190 0x7fe1d0102570 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009900 tx=0x7fe1c802bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 -- 172.21.15.5:0/2611861423 shutdown_connections 2024-04-22T13:30:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 --2- 172.21.15.5:0/2611861423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 0x7fe1d010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 --2- 172.21.15.5:0/2611861423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102b40 0x7fe1d0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 --2- 172.21.15.5:0/2611861423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d0102190 0x7fe1d0102570 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 -- 172.21.15.5:0/2611861423 >> 172.21.15.5:0/2611861423 conn(0x7fe1d00fd9f0 msgr2=0x7fe1d00ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 -- 172.21.15.5:0/2611861423 shutdown_connections 2024-04-22T13:30:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.550+0000 7fe1d8606640 1 -- 172.21.15.5:0/2611861423 wait complete. 2024-04-22T13:30:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 Processor -- start 2024-04-22T13:30:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 -- start start 2024-04-22T13:30:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102190 0x7fe1d0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 0x7fe1d0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 0x7fe1d019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01121a0 con 0x7fe1d019d020 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0112020 con 0x7fe1d0102190 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d8606640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0112320 con 0x7fe1d0107260 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 0x7fe1d019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 0x7fe1d0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d63a2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102190 0x7fe1d0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 0x7fe1d019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36364/0 (socket says 172.21.15.5:36364) 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 0x7fe1d0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:37560/0 (socket says 172.21.15.5:37560) 2024-04-22T13:30:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 -- 172.21.15.5:0/1022671353 learned_addr learned my addr 172.21.15.5:0/1022671353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d63a2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102190 0x7fe1d0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:34846/0 (socket says 172.21.15.5:34846) 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 -- 172.21.15.5:0/1022671353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 msgr2=0x7fe1d0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 0x7fe1d0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 -- 172.21.15.5:0/1022671353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102190 msgr2=0x7fe1d0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102190 0x7fe1d0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 -- 172.21.15.5:0/1022671353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d019db10 con 0x7fe1d019d020 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.551+0000 7fe1d6ba3640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 0x7fe1d019d4d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00efd0 tx=0x7fe1cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.552+0000 7fe1c77fe640 1 -- 172.21.15.5:0/1022671353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc003b70 con 0x7fe1d019d020 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.552+0000 7fe1c77fe640 1 -- 172.21.15.5:0/1022671353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1cc003d10 con 0x7fe1d019d020 2024-04-22T13:30:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.552+0000 7fe1c77fe640 1 -- 172.21.15.5:0/1022671353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0108a0 con 0x7fe1d019d020 2024-04-22T13:30:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.552+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d019de00 con 0x7fe1d019d020 2024-04-22T13:30:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.552+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d01a55a0 con 0x7fe1d019d020 2024-04-22T13:30:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.554+0000 7fe1c77fe640 1 -- 172.21.15.5:0/1022671353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe1cc004020 con 0x7fe1d019d020 2024-04-22T13:30:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.554+0000 7fe1db1fc740 1 Processor -- start 2024-04-22T13:30:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.554+0000 7fe1db1fc740 1 -- start start 2024-04-22T13:30:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1525e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15267430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15269e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560f151c31f0 con 0x560f1525e4d0 2024-04-22T13:30:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f151604b0 con 0x560f15267a00 2024-04-22T13:30:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f151bf1b0 con 0x560f1525ee80 2024-04-22T13:30:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1525e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1525e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36368/0 (socket says 172.21.15.5:36368) 2024-04-22T13:30:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15267430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/192385926 learned_addr learned my addr 172.21.15.5:0/192385926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15269e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 -- 172.21.15.5:0/192385926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 msgr2=0x560f15269e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15269e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 -- 172.21.15.5:0/192385926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 msgr2=0x560f1525e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1c77fe640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe1bc0a3d20 0x7fe1bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1525e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 -- 172.21.15.5:0/192385926 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f151521b0 con 0x560f1525ee80 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d6ba3640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe1bc0a3d20 0x7fe1bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15267430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8000c00 tx=0x7fe1c80095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2e21071e3cc44e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1c77fe640 1 -- 172.21.15.5:0/1022671353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe1cc0d0bb0 con 0x7fe1d019d020 2024-04-22T13:30:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.555+0000 7fe1c4bfe640 1 -- 172.21.15.5:0/192385926 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8045020 con 0x560f1525ee80 2024-04-22T13:30:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1c4bfe640 1 -- 172.21.15.5:0/192385926 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c802c7e0 con 0x560f1525ee80 2024-04-22T13:30:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1c4bfe640 1 -- 172.21.15.5:0/192385926 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8005570 con 0x560f1525ee80 2024-04-22T13:30:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 -- 172.21.15.5:0/192385926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 msgr2=0x560f15267430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1d6ba3640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe1bc0a3d20 0x7fe1bc0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc098fe0 tx=0x7fe1cc09f040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15267430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8000c00 tx=0x7fe1c80095f0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 -- 172.21.15.5:0/192385926 shutdown_connections 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15269e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15267430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/192385926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1525e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 -- 172.21.15.5:0/192385926 >> 172.21.15.5:0/192385926 conn(0x560f15258ba0 msgr2=0x560f15258f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 -- 172.21.15.5:0/192385926 shutdown_connections 2024-04-22T13:30:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.556+0000 7fe1db1fc740 1 -- 172.21.15.5:0/192385926 wait complete. 2024-04-22T13:30:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 -- start start 2024-04-22T13:30:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 Processor -- start 2024-04-22T13:30:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525e4d0 0x560f15267330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f152646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 0x560f15264cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560f1526a910 con 0x560f15267a00 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f1517c500 con 0x560f1525e4d0 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f1526aa90 con 0x560f1525ee80 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1db1fc740 10 client.0 _ll_register_callbacks cb 0x560f1514ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 0x560f15264cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 0x560f15264cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36382/0 (socket says 172.21.15.5:36382) 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/3818549407 learned_addr learned my addr 172.21.15.5:0/3818549407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525e4d0 0x560f15267330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f152646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/3818549407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 msgr2=0x560f152646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f152646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525e4d0 msgr2=0x560f15267330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525e4d0 0x560f15267330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.557+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/3818549407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f14ed53f0 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 0x560f15264cd0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0011f50 tx=0x7fe1c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe7717727f85c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/3818549407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004030 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/3818549407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe1c00041d0 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c802c7e0 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/3818549407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1c00043d0 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 10 client.17100 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f14f9f7d0 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/3818549407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c802c7e0 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/3818549407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe1c002b450 con 0x560f15267a00 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 10 client.17100 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:31.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 20 client.17100 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f15277b80 0x560f15279fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 4 client.17100 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:31.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.558+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f15162930 con 0x560f15277b80 2024-04-22T13:30:31.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.559+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f15277b80 0x560f15279fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.559+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f15277b80 0x560f15279fd0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512e60ec33022216 server_cookie=96fe74b8db4acee7 in_seq=0 out_seq=0 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.559+0000 7fe1b9ffb640 10 client.17100 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/3818549407 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f15162930 con 0x560f15277b80 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1b9ffb640 10 client.17100 handle_command_reply: tid=0 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 client.17100 shutdown 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 msgr2=0x560f15264cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 0x560f15264cd0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0011f50 tx=0x7fe1c0005f90 comp rx=0 tx=0).stop 2024-04-22T13:30:31.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 shutdown_connections 2024-04-22T13:30:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f15277b80 0x560f15279fd0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f15267a00 0x560f15264cd0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1b9ffb640 0 client.17100 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f152646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/3818549407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525e4d0 0x560f15267330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 >> 172.21.15.5:0/3818549407 conn(0x560f15258ba0 msgr2=0x560f1525a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 shutdown_connections 2024-04-22T13:30:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 1 -- 172.21.15.5:0/3818549407 wait complete. 2024-04-22T13:30:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.560+0000 7fe1db1fc740 20 client.17100 trim_cache size 0 max 16384 2024-04-22T13:30:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 Processor -- start 2024-04-22T13:30:31.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 -- start start 2024-04-22T13:30:31.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f152737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15259ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15266df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ada20 con 0x560f1525e4d0 2024-04-22T13:30:31.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01af560 con 0x560f15267a00 2024-04-22T13:30:31.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ae0f0 con 0x560f1525ee80 2024-04-22T13:30:31.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f152737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f152737d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36394/0 (socket says 172.21.15.5:36394) 2024-04-22T13:30:31.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.574+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/769224452 learned_addr learned my addr 172.21.15.5:0/769224452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15259ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/769224452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 msgr2=0x560f15259ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15259ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/769224452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 msgr2=0x560f15266df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15266df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/769224452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1525aea0 con 0x560f1525e4d0 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f152737d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0011ce0 tx=0x7fe1c0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6a77d3c61a8d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1bbfff640 1 -- 172.21.15.5:0/769224452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0009850 con 0x560f1525e4d0 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1bbfff640 1 -- 172.21.15.5:0/769224452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c00054f0 con 0x560f1525e4d0 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.575+0000 7fe1bbfff640 1 -- 172.21.15.5:0/769224452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004570 con 0x560f1525e4d0 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- 172.21.15.5:0/769224452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 msgr2=0x560f152737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f152737d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0011ce0 tx=0x7fe1c0005d30 comp rx=0 tx=0).stop 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- 172.21.15.5:0/769224452 shutdown_connections 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f15267a00 0x560f15266df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f1525ee80 0x560f15259ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/769224452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f152737d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- 172.21.15.5:0/769224452 >> 172.21.15.5:0/769224452 conn(0x560f15258ba0 msgr2=0x560f15257c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- 172.21.15.5:0/769224452 shutdown_connections 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- 172.21.15.5:0/769224452 wait complete. 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- start start 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 Processor -- start 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1526b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525ee80 0x560f1526bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f15267a00 0x560f1526c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01aee90 con 0x560f1525e4d0 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ae7c0 con 0x560f1525ee80 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01aba00 con 0x560f15267a00 2024-04-22T13:30:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1db1fc740 10 client.0 _ll_register_callbacks cb 0x7fe1c0011da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.576+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1526b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1526b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36408/0 (socket says 172.21.15.5:36408) 2024-04-22T13:30:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/1681884377 learned_addr learned my addr 172.21.15.5:0/1681884377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f15267a00 0x560f1526c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525ee80 0x560f1526bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/1681884377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f15267a00 msgr2=0x560f1526c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f15267a00 0x560f1526c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525ee80 msgr2=0x560f1526bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525ee80 0x560f1526bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 -- 172.21.15.5:0/1681884377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f152dd110 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1d5ba1640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1526b6e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0005fd0 tx=0x7fe1c002b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b029faa96769f363 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/1681884377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c002bc40 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/1681884377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe1c0022420 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c00054f0 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/1681884377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1c0022620 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1db1fc740 10 client.17106 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f152de3e0 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.577+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/1681884377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c00054f0 con 0x560f1525e4d0 2024-04-22T13:30:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/1681884377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe1c0034780 con 0x560f1525e4d0 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 10 client.17106 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 20 client.17106 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f152de9a0 0x560f152e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 4 client.17106 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f151d0c10 con 0x560f152de9a0 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f152de9a0 0x560f152e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1b9ffb640 10 client.17106 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1d63a2640 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f152de9a0 0x560f152e0d90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a20e41042376f42 server_cookie=853c09f23c740a29 in_seq=0 out_seq=0 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1b9ffb640 1 -- 172.21.15.5:0/1681884377 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f151d0c10 con 0x560f152de9a0 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1b9ffb640 10 client.17106 handle_command_reply: tid=0 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.578+0000 7fe1db1fc740 1 client.17106 shutdown 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 msgr2=0x560f1526b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1526b6e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0005fd0 tx=0x7fe1c002b660 comp rx=0 tx=0).stop 2024-04-22T13:30:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 shutdown_connections 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560f152de9a0 0x560f152e0d90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f15267a00 0x560f1526c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f1525ee80 0x560f1526bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 --2- 172.21.15.5:0/1681884377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f1525e4d0 0x560f1526b6e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 >> 172.21.15.5:0/1681884377 conn(0x560f15258ba0 msgr2=0x560f15257c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1b9ffb640 0 client.17106 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 shutdown_connections 2024-04-22T13:30:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 1 -- 172.21.15.5:0/1681884377 wait complete. 2024-04-22T13:30:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1db1fc740 20 client.17106 trim_cache size 0 max 16384 2024-04-22T13:30:31.596 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:30:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe1bc0a3d20 msgr2=0x7fe1bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe1bc0a3d20 0x7fe1bc0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc098fe0 tx=0x7fe1cc09f040 comp rx=0 tx=0).stop 2024-04-22T13:30:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 msgr2=0x7fe1d019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 0x7fe1d019d4d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00efd0 tx=0x7fe1cc00c6a0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 shutdown_connections 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d019d020 0x7fe1d019d4d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe1bc0a3d20 0x7fe1bc0a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1d0107260 0x7fe1d0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 --2- 172.21.15.5:0/1022671353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1d0102190 0x7fe1d0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 >> 172.21.15.5:0/1022671353 conn(0x7fe1d00fd9f0 msgr2=0x7fe1d00fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:31.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 shutdown_connections 2024-04-22T13:30:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:31.579+0000 7fe1d8606640 1 -- 172.21.15.5:0/1022671353 wait complete. 2024-04-22T13:30:32.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:31 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2744067353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:32.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:31 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2744067353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:32.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:31 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2744067353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:32.602 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:30:32.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:30:32.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:30:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 -- 172.21.15.5:0/3098896313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64106160 msgr2=0x7eff6410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 --2- 172.21.15.5:0/3098896313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64106160 0x7eff6410dd30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009ca0 tx=0x7eff5c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 -- 172.21.15.5:0/3098896313 shutdown_connections 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 --2- 172.21.15.5:0/3098896313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff64106160 0x7eff6410dd30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 --2- 172.21.15.5:0/3098896313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff64101a40 0x7eff64105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 --2- 172.21.15.5:0/3098896313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff64101090 0x7eff64101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 -- 172.21.15.5:0/3098896313 >> 172.21.15.5:0/3098896313 conn(0x7eff64078000 msgr2=0x7eff64078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 -- 172.21.15.5:0/3098896313 shutdown_connections 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 -- 172.21.15.5:0/3098896313 wait complete. 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 Processor -- start 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.729+0000 7eff6ba93640 1 -- start start 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff64101090 0x7eff641975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff64106160 0x7eff64197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 0x7eff6419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff641110d0 con 0x7eff6419be90 2024-04-22T13:30:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff64110f50 con 0x7eff64106160 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111250 con 0x7eff64101090 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 0x7eff6419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 0x7eff6419c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36416/0 (socket says 172.21.15.5:36416) 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 -- 172.21.15.5:0/4239983562 learned_addr learned my addr 172.21.15.5:0/4239983562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6982f640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff64101090 0x7eff641975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6902e640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff64106160 0x7eff64197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 -- 172.21.15.5:0/4239983562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff64101090 msgr2=0x7eff641975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff64101090 0x7eff641975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 -- 172.21.15.5:0/4239983562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff64106160 msgr2=0x7eff64197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff64106160 0x7eff64197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 -- 172.21.15.5:0/4239983562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6419c980 con 0x7eff6419be90 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6a030640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 0x7eff6419c340 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005e30 tx=0x7eff5c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff5affd640 1 -- 172.21.15.5:0/4239983562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c041020 con 0x7eff6419be90 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6419cc10 con 0x7eff6419be90 2024-04-22T13:30:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.730+0000 7eff5affd640 1 -- 172.21.15.5:0/4239983562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c032970 con 0x7eff6419be90 2024-04-22T13:30:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.731+0000 7eff5affd640 1 -- 172.21.15.5:0/4239983562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c039040 con 0x7eff6419be90 2024-04-22T13:30:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.731+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641a4430 con 0x7eff6419be90 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.733+0000 7eff5affd640 1 -- 172.21.15.5:0/4239983562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7eff5c044020 con 0x7eff6419be90 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.733+0000 7eff6e765740 1 Processor -- start 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 -- start start 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 0x55f47c306ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c3074a0 0x55f47c30fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c312470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f47c268f00 con 0x55f47c306af0 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f47c206590 con 0x55f47c3074a0 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6e765740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f47c264ec0 con 0x55f47c310020 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 0x55f47c306ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff5affd640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7eff500a3d20 0x7eff500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 0x55f47c306ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36424/0 (socket says 172.21.15.5:36424) 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 -- 172.21.15.5:0/3061597395 learned_addr learned my addr 172.21.15.5:0/3061597395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c312470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6a030640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7eff500a3d20 0x7eff500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6982f640 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c3074a0 0x55f47c30fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff5affd640 1 -- 172.21.15.5:0/4239983562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7eff5c0f2240 con 0x7eff6419be90 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 -- 172.21.15.5:0/3061597395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 msgr2=0x55f47c312470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c312470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 -- 172.21.15.5:0/3061597395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c3074a0 msgr2=0x55f47c30fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c3074a0 0x55f47c30fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6902e640 1 -- 172.21.15.5:0/3061597395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47c0437e0 con 0x55f47c306af0 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.734+0000 7eff6982f640 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c3074a0 0x55f47c30fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.735+0000 7eff6902e640 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 0x55f47c306ed0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7eff54009f70 tx=0x7eff54005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25303faaf08791e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.735+0000 7eff6a030640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7eff500a3d20 0x7eff500a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0be300 tx=0x7eff5c003bc0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.735+0000 7eff4ffff640 1 -- 172.21.15.5:0/3061597395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54004050 con 0x55f47c306af0 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.735+0000 7eff4ffff640 1 -- 172.21.15.5:0/3061597395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff540054d0 con 0x55f47c306af0 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.735+0000 7eff4ffff640 1 -- 172.21.15.5:0/3061597395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54004b00 con 0x55f47c306af0 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 -- 172.21.15.5:0/3061597395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 msgr2=0x55f47c306ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 0x55f47c306ed0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7eff54009f70 tx=0x7eff54005850 comp rx=0 tx=0).stop 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 -- 172.21.15.5:0/3061597395 shutdown_connections 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c312470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c3074a0 0x55f47c30fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 --2- 172.21.15.5:0/3061597395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c306af0 0x55f47c306ed0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 -- 172.21.15.5:0/3061597395 >> 172.21.15.5:0/3061597395 conn(0x55f47c3011c0 msgr2=0x55f47c3015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 -- 172.21.15.5:0/3061597395 shutdown_connections 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 -- 172.21.15.5:0/3061597395 wait complete. 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 -- start start 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.736+0000 7eff6e765740 1 Processor -- start 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c30d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c30da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f47c312f90 con 0x55f47c310020 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f47c222580 con 0x55f47c306af0 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f47c313110 con 0x55f47c3074a0 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 10 client.0 _ll_register_callbacks cb 0x55f47c228650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c30da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c30da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36428/0 (socket says 172.21.15.5:36428) 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 -- 172.21.15.5:0/1174281596 learned_addr learned my addr 172.21.15.5:0/1174281596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6982f640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c30d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 -- 172.21.15.5:0/1174281596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 msgr2=0x55f47c30d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c30d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 -- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 msgr2=0x55f47c30d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 -- 172.21.15.5:0/1174281596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47c0a3f70 con 0x55f47c310020 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6902e640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c30da90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7eff54009940 tx=0x7eff540098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d850187f87718f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff4d7fa640 1 -- 172.21.15.5:0/1174281596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54028040 con 0x55f47c310020 2024-04-22T13:30:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff4d7fa640 1 -- 172.21.15.5:0/1174281596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7eff54004df0 con 0x55f47c310020 2024-04-22T13:30:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.737+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff540054d0 con 0x55f47c310020 2024-04-22T13:30:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff4d7fa640 1 -- 172.21.15.5:0/1174281596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff54034050 con 0x55f47c310020 2024-04-22T13:30:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 10 client.17118 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f47bf3b170 con 0x55f47c310020 2024-04-22T13:30:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff4d7fa640 1 -- 172.21.15.5:0/1174281596 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff540054d0 con 0x55f47c310020 2024-04-22T13:30:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff4d7fa640 1 -- 172.21.15.5:0/1174281596 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff5401d020 con 0x55f47c310020 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 10 client.17118 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 20 client.17118 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c320380 0x55f47c322770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 4 client.17118 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f47c276920 con 0x55f47c320380 2024-04-22T13:30:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.738+0000 7eff6982f640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c320380 0x55f47c322770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.739+0000 7eff4d7fa640 10 client.17118 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.739+0000 7eff6982f640 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c320380 0x55f47c322770 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13de751f943d4fa3 server_cookie=19c01a523f002f6a in_seq=0 out_seq=0 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff4d7fa640 1 -- 172.21.15.5:0/1174281596 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f47c276920 con 0x55f47c320380 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff4d7fa640 10 client.17118 handle_command_reply: tid=0 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 client.17118 shutdown 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 msgr2=0x55f47c30da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c30da90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7eff54009940 tx=0x7eff540098b0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 shutdown_connections 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c320380 0x55f47c322770 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c30da90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c30d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 --2- 172.21.15.5:0/1174281596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff4d7fa640 0 client.17118 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 >> 172.21.15.5:0/1174281596 conn(0x55f47c3011c0 msgr2=0x55f47c302f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 shutdown_connections 2024-04-22T13:30:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 1 -- 172.21.15.5:0/1174281596 wait complete. 2024-04-22T13:30:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.740+0000 7eff6e765740 20 client.17118 trim_cache size 0 max 16384 2024-04-22T13:30:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 Processor -- start 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 -- start start 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c31be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c2fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ab860 con 0x55f47c310020 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ad3a0 con 0x55f47c306af0 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.758+0000 7eff6e765740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7eff641abf30 con 0x55f47c3074a0 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c2fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c2fde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36436/0 (socket says 172.21.15.5:36436) 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 -- 172.21.15.5:0/2490320631 learned_addr learned my addr 172.21.15.5:0/2490320631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6982f640 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c31be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 -- 172.21.15.5:0/2490320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 msgr2=0x55f47c31be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c31be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 -- 172.21.15.5:0/2490320631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 msgr2=0x55f47c30f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 -- 172.21.15.5:0/2490320631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47c305d40 con 0x55f47c310020 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff6902e640 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c2fde90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7eff5401f7d0 tx=0x7eff54005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e14b3de4a1282e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff4f7fe640 1 -- 172.21.15.5:0/2490320631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54005470 con 0x55f47c310020 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff4f7fe640 1 -- 172.21.15.5:0/2490320631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff540046f0 con 0x55f47c310020 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.759+0000 7eff4f7fe640 1 -- 172.21.15.5:0/2490320631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400f500 con 0x55f47c310020 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 -- 172.21.15.5:0/2490320631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 msgr2=0x55f47c2fde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c2fde90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7eff5401f7d0 tx=0x7eff54005970 comp rx=0 tx=0).stop 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 -- 172.21.15.5:0/2490320631 shutdown_connections 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c310020 0x55f47c2fde90 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c3074a0 0x55f47c31be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 --2- 172.21.15.5:0/2490320631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c30f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 -- 172.21.15.5:0/2490320631 >> 172.21.15.5:0/2490320631 conn(0x55f47c3011c0 msgr2=0x55f47c2fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 -- 172.21.15.5:0/2490320631 shutdown_connections 2024-04-22T13:30:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 -- 172.21.15.5:0/2490320631 wait complete. 2024-04-22T13:30:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 -- start start 2024-04-22T13:30:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.760+0000 7eff6e765740 1 Processor -- start 2024-04-22T13:30:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c3186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 0x55f47c309980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c309ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff641accd0 con 0x55f47c3074a0 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ac600 con 0x55f47c306af0 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7eff641a9840 con 0x55f47c310020 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6e765740 10 client.0 _ll_register_callbacks cb 0x7eff54013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 0x55f47c309980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 0x55f47c309980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36438/0 (socket says 172.21.15.5:36438) 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 -- 172.21.15.5:0/3149190714 learned_addr learned my addr 172.21.15.5:0/3149190714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6902e640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c309ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6902e640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c3186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 -- 172.21.15.5:0/3149190714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 msgr2=0x55f47c309ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c309ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 -- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 msgr2=0x55f47c3186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c3186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 -- 172.21.15.5:0/3149190714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47c30a690 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6902e640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c3186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff6982f640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 0x55f47c309980 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7eff60009d70 tx=0x7eff60014a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8598bdb5c73d0b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.761+0000 7eff4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff4d7fa640 1 -- 172.21.15.5:0/3149190714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60004250 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff4d7fa640 1 -- 172.21.15.5:0/3149190714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7eff600043f0 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff540046f0 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff4d7fa640 1 -- 172.21.15.5:0/3149190714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff6002e050 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 10 client.17124 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f47c3868e0 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff4d7fa640 1 -- 172.21.15.5:0/3149190714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff540046f0 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff4d7fa640 1 -- 172.21.15.5:0/3149190714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff60003cc0 con 0x55f47c3074a0 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 10 client.17124 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 20 client.17124 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c387040 0x55f47c389460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 4 client.17124 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f47c270790 con 0x55f47c387040 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.762+0000 7eff6902e640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c387040 0x55f47c389460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.763+0000 7eff6902e640 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c387040 0x55f47c389460 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bbf5d1f91fc929 server_cookie=c9f9c190b04244ef in_seq=0 out_seq=0 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.763+0000 7eff4d7fa640 10 client.17124 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff4d7fa640 1 -- 172.21.15.5:0/3149190714 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f47c270790 con 0x55f47c387040 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff4d7fa640 10 client.17124 handle_command_reply: tid=0 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 client.17124 shutdown 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 msgr2=0x55f47c309980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 0x55f47c309980 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7eff60009d70 tx=0x7eff60014a60 comp rx=0 tx=0).stop 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 shutdown_connections 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f47c387040 0x55f47c389460 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f47c3074a0 0x55f47c309980 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f47c310020 0x55f47c309ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 --2- 172.21.15.5:0/3149190714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f47c306af0 0x55f47c3186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff4d7fa640 0 client.17124 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 >> 172.21.15.5:0/3149190714 conn(0x55f47c3011c0 msgr2=0x55f47c2fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 shutdown_connections 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 1 -- 172.21.15.5:0/3149190714 wait complete. 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.764+0000 7eff6e765740 20 client.17124 trim_cache size 0 max 16384 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6cbaabb1-eae8-4a18-818f-d084f16bd1b7", 2024-04-22T13:30:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:30:32.770 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:30:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7eff500a3d20 msgr2=0x7eff500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7eff500a3d20 0x7eff500a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0be300 tx=0x7eff5c003bc0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 msgr2=0x7eff6419c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 0x7eff6419c340 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005e30 tx=0x7eff5c0346d0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 shutdown_connections 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff6419be90 0x7eff6419c340 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7eff500a3d20 0x7eff500a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff64106160 0x7eff64197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 --2- 172.21.15.5:0/4239983562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff64101090 0x7eff641975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 >> 172.21.15.5:0/4239983562 conn(0x7eff64078000 msgr2=0x7eff640ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 shutdown_connections 2024-04-22T13:30:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.765+0000 7eff6ba93640 1 -- 172.21.15.5:0/4239983562 wait complete. 2024-04-22T13:30:32.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:32.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.881+0000 7f0d42f32640 1 -- 172.21.15.5:0/3761135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c1031c0 msgr2=0x7f0d3c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.881+0000 7f0d42f32640 1 --2- 172.21.15.5:0/3761135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c1035a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009900 tx=0x7f0d2c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- 172.21.15.5:0/3761135402 shutdown_connections 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 --2- 172.21.15.5:0/3761135402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c10c590 0x7f0d3c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 --2- 172.21.15.5:0/3761135402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c103ae0 0x7f0d3c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 --2- 172.21.15.5:0/3761135402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c1035a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- 172.21.15.5:0/3761135402 >> 172.21.15.5:0/3761135402 conn(0x7f0d3c077fa0 msgr2=0x7f0d3c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- 172.21.15.5:0/3761135402 shutdown_connections 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- 172.21.15.5:0/3761135402 wait complete. 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 Processor -- start 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- start start 2024-04-22T13:30:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c103ae0 0x7f0d3c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 0x7f0d3c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c112210 con 0x7f0d3c10c590 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c112090 con 0x7f0d3c1031c0 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.882+0000 7f0d42f32640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c112390 con 0x7f0d3c103ae0 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d414cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 0x7f0d3c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d414cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 0x7f0d3c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36456/0 (socket says 172.21.15.5:36456) 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d414cf640 1 -- 172.21.15.5:0/434893880 learned_addr learned my addr 172.21.15.5:0/434893880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c103ae0 0x7f0d3c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 -- 172.21.15.5:0/434893880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c103ae0 msgr2=0x7f0d3c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c103ae0 0x7f0d3c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 -- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 msgr2=0x7f0d3c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 0x7f0d3c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 -- 172.21.15.5:0/434893880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19d700 con 0x7f0d3c1031c0 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d414cf640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 0x7f0d3c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d40cce640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c198710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c02c210 tx=0x7f0d2c02c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d39ffb640 1 -- 172.21.15.5:0/434893880 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c043020 con 0x7f0d3c1031c0 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d39ffb640 1 -- 172.21.15.5:0/434893880 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d2c02dba0 con 0x7f0d3c1031c0 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19d990 con 0x7f0d3c1031c0 2024-04-22T13:30:32.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.883+0000 7f0d39ffb640 1 -- 172.21.15.5:0/434893880 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0038e0 con 0x7f0d3c1031c0 2024-04-22T13:30:32.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.884+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a5280 con 0x7f0d3c1031c0 2024-04-22T13:30:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 Processor -- start 2024-04-22T13:30:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 -- start start 2024-04-22T13:30:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d39ffb640 1 -- 172.21.15.5:0/434893880 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0d2c046020 con 0x7f0d3c1031c0 2024-04-22T13:30:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28f580 0x55a89c28f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c298540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c29ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a89c1f41e0 con 0x55a89c298a80 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a89c191390 con 0x55a89c28f580 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d45c04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a89c1f01a0 con 0x55a89c28fea0 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d3bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c29ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d3bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c29ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36466/0 (socket says 172.21.15.5:36466) 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d3bfff640 1 -- 172.21.15.5:0/1599553012 learned_addr learned my addr 172.21.15.5:0/1599553012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d40cce640 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c298540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.886+0000 7f0d414cf640 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28f580 0x55a89c28f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d3bfff640 1 -- 172.21.15.5:0/1599553012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 msgr2=0x55a89c298540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c298540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d3bfff640 1 -- 172.21.15.5:0/1599553012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28f580 msgr2=0x55a89c28f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28f580 0x55a89c28f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d3bfff640 1 -- 172.21.15.5:0/1599553012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89bfd07d0 con 0x55a89c298a80 2024-04-22T13:30:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d39ffb640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0d240a3d20 0x7f0d240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c29ae70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009c90 tx=0x7f0d30011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726fac1c395c135e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d414cf640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0d240a3d20 0x7f0d240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d39ffb640 1 -- 172.21.15.5:0/434893880 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0d2c0f2570 con 0x7f0d3c1031c0 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d22ffd640 1 -- 172.21.15.5:0/1599553012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3001e070 con 0x55a89c298a80 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d22ffd640 1 -- 172.21.15.5:0/1599553012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30019ae0 con 0x55a89c298a80 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.887+0000 7f0d22ffd640 1 -- 172.21.15.5:0/1599553012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000ea40 con 0x55a89c298a80 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 -- 172.21.15.5:0/1599553012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 msgr2=0x55a89c29ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c29ae70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009c90 tx=0x7f0d30011810 comp rx=0 tx=0).stop 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d414cf640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0d240a3d20 0x7f0d240a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d340049d0 tx=0x7f0d34009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 -- 172.21.15.5:0/1599553012 shutdown_connections 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c29ae70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c298540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 --2- 172.21.15.5:0/1599553012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28f580 0x55a89c28f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 -- 172.21.15.5:0/1599553012 >> 172.21.15.5:0/1599553012 conn(0x55a89c289cb0 msgr2=0x55a89c28a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 -- 172.21.15.5:0/1599553012 shutdown_connections 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 -- 172.21.15.5:0/1599553012 wait complete. 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 -- start start 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.888+0000 7f0d45c04740 1 Processor -- start 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c295a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c295f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c2964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a89c29b870 con 0x55a89c298a80 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a89c1ad2f0 con 0x55a89c28fea0 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a89c29b9f0 con 0x55a89c28f580 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 10 client.0 _ll_register_callbacks cb 0x55a89c1ba580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c2964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c2964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36478/0 (socket says 172.21.15.5:36478) 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 -- 172.21.15.5:0/3674470354 learned_addr learned my addr 172.21.15.5:0/3674470354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c295f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d40cce640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c295a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 -- 172.21.15.5:0/3674470354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 msgr2=0x55a89c295a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c295a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 -- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 msgr2=0x55a89c295f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c295f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 -- 172.21.15.5:0/3674470354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89bf063f0 con 0x55a89c298a80 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d414cf640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c2964b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400fab0 tx=0x7f0d34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64c1aa783a72948 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d1bfff640 1 -- 172.21.15.5:0/3674470354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34010c80 con 0x55a89c298a80 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d1bfff640 1 -- 172.21.15.5:0/3674470354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0d34028440 con 0x55a89c298a80 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.889+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30019ae0 con 0x55a89c298a80 2024-04-22T13:30:32.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d1bfff640 1 -- 172.21.15.5:0/3674470354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d34010e20 con 0x55a89c298a80 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 10 client.17136 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a89bec60f0 con 0x55a89c298a80 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d1bfff640 1 -- 172.21.15.5:0/3674470354 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30019ae0 con 0x55a89c298a80 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d1bfff640 1 -- 172.21.15.5:0/3674470354 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0d34026020 con 0x55a89c298a80 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 10 client.17136 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 20 client.17136 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c2a8a70 0x55a89c2aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 4 client.17136 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a89c201c00 con 0x55a89c2a8a70 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.890+0000 7f0d40cce640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c2a8a70 0x55a89c2aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d40cce640 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c2a8a70 0x55a89c2aaef0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83980f4f59cc2c5c server_cookie=c96ece68edaac476 in_seq=0 out_seq=0 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d1bfff640 10 client.17136 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d1bfff640 1 -- 172.21.15.5:0/3674470354 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a89c201c00 con 0x55a89c2a8a70 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d1bfff640 10 client.17136 handle_command_reply: tid=0 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 client.17136 shutdown 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 msgr2=0x55a89c2964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c2964b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400fab0 tx=0x7f0d34004ab0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 shutdown_connections 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c2a8a70 0x55a89c2aaef0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c2964b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c295f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d1bfff640 0 client.17136 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3674470354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c295a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.891+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 >> 172.21.15.5:0/3674470354 conn(0x55a89c289cb0 msgr2=0x55a89c28ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.892+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 shutdown_connections 2024-04-22T13:30:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.892+0000 7f0d45c04740 1 -- 172.21.15.5:0/3674470354 wait complete. 2024-04-22T13:30:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.892+0000 7f0d45c04740 20 client.17136 trim_cache size 0 max 16384 2024-04-22T13:30:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 Processor -- start 2024-04-22T13:30:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 -- start start 2024-04-22T13:30:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c298160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c2a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c294b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ada00 con 0x55a89c298a80 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1af540 con 0x55a89c28fea0 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d45c04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae0d0 con 0x55a89c28f580 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d414cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c294b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d414cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c294b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36494/0 (socket says 172.21.15.5:36494) 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d414cf640 1 -- 172.21.15.5:0/3142738895 learned_addr learned my addr 172.21.15.5:0/3142738895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c2a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.909+0000 7f0d40cce640 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c298160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d414cf640 1 -- 172.21.15.5:0/3142738895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 msgr2=0x55a89c298160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d414cf640 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c298160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d414cf640 1 -- 172.21.15.5:0/3142738895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 msgr2=0x55a89c2a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d414cf640 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c2a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d414cf640 1 -- 172.21.15.5:0/3142738895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c289ae0 con 0x55a89c298a80 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c2a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d414cf640 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c294b90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400fa80 tx=0x7f0d34005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a7242b67d581ac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d227fc640 1 -- 172.21.15.5:0/3142738895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34036030 con 0x55a89c298a80 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d227fc640 1 -- 172.21.15.5:0/3142738895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3400f810 con 0x55a89c298a80 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.910+0000 7f0d227fc640 1 -- 172.21.15.5:0/3142738895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340316a0 con 0x55a89c298a80 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- 172.21.15.5:0/3142738895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 msgr2=0x55a89c294b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c294b90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400fa80 tx=0x7f0d34005bb0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- 172.21.15.5:0/3142738895 shutdown_connections 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c298a80 0x55a89c294b90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c28fea0 0x55a89c2a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- 172.21.15.5:0/3142738895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28f580 0x55a89c298160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- 172.21.15.5:0/3142738895 >> 172.21.15.5:0/3142738895 conn(0x55a89c289cb0 msgr2=0x55a89c2884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- 172.21.15.5:0/3142738895 shutdown_connections 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- 172.21.15.5:0/3142738895 wait complete. 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- start start 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 Processor -- start 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 0x55a89c292660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c292ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c298a80 0x55a89c2930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1aee70 con 0x55a89c28f580 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae7a0 con 0x55a89c298a80 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.911+0000 7f0d45c04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ab9e0 con 0x55a89c28fea0 2024-04-22T13:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d45c04740 10 client.0 _ll_register_callbacks cb 0x7f0d340115c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 0x55a89c292660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 0x55a89c292660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36510/0 (socket says 172.21.15.5:36510) 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 -- 172.21.15.5:0/518944892 learned_addr learned my addr 172.21.15.5:0/518944892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d414cf640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c298a80 0x55a89c2930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c292ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 -- 172.21.15.5:0/518944892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 msgr2=0x55a89c292ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c292ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 -- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c298a80 msgr2=0x55a89c2930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c298a80 0x55a89c2930e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 -- 172.21.15.5:0/518944892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89c30e0c0 con 0x55a89c28f580 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d414cf640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c298a80 0x55a89c2930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:32.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c292ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d40cce640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 0x55a89c292660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0efeb0 tx=0x7f0d2c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74fd59fd7687011 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d1bfff640 1 -- 172.21.15.5:0/518944892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0be650 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d1bfff640 1 -- 172.21.15.5:0/518944892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0d2c0bc950 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3400f810 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d1bfff640 1 -- 172.21.15.5:0/518944892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c0c2a00 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d45c04740 10 client.17142 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.912+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a89c30f370 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d1bfff640 1 -- 172.21.15.5:0/518944892 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3400f810 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d1bfff640 1 -- 172.21.15.5:0/518944892 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0d2c0c6e30 con 0x55a89c28f580 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d45c04740 10 client.17142 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:32.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d45c04740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d45c04740 20 client.17142 populate_metadata read hostname 'smithi005' 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d45c04740 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c30fac0 0x55a89c311ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d45c04740 4 client.17142 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a89c1fba70 con 0x55a89c30fac0 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c30fac0 0x55a89c311ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d1bfff640 10 client.17142 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.913+0000 7f0d3bfff640 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c30fac0 0x55a89c311ee0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85996cafa0f98dd0 server_cookie=1a750a7e8a3420db in_seq=0 out_seq=0 2024-04-22T13:30:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d1bfff640 1 -- 172.21.15.5:0/518944892 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a89c1fba70 con 0x55a89c30fac0 2024-04-22T13:30:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d1bfff640 10 client.17142 handle_command_reply: tid=0 2024-04-22T13:30:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 client.17142 shutdown 2024-04-22T13:30:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 msgr2=0x55a89c292660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 0x55a89c292660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0efeb0 tx=0x7f0d2c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 shutdown_connections 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a89c30fac0 0x55a89c311ee0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a89c298a80 0x55a89c2930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a89c28fea0 0x55a89c292ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d1bfff640 0 client.17142 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 --2- 172.21.15.5:0/518944892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a89c28f580 0x55a89c292660 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 >> 172.21.15.5:0/518944892 conn(0x55a89c289cb0 msgr2=0x55a89c288d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 shutdown_connections 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 1 -- 172.21.15.5:0/518944892 wait complete. 2024-04-22T13:30:32.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d45c04740 20 client.17142 trim_cache size 0 max 16384 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "6cbaabb1-eae8-4a18-818f-d084f16bd1b7": { 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6cbaabb1-eae8-4a18-818f-d084f16bd1b7", 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:30:32.921 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:30:32.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0d240a3d20 msgr2=0x7f0d240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0d240a3d20 0x7f0d240a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d340049d0 tx=0x7f0d34009500 comp rx=0 tx=0).stop 2024-04-22T13:30:32.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c1031c0 msgr2=0x7f0d3c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:32.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c198710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c02c210 tx=0x7f0d2c02c700 comp rx=0 tx=0).stop 2024-04-22T13:30:32.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 shutdown_connections 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d3c10c590 0x7f0d3c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0d240a3d20 0x7f0d240a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c103ae0 0x7f0d3c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 --2- 172.21.15.5:0/434893880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d3c1031c0 0x7f0d3c198710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.914+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 >> 172.21.15.5:0/434893880 conn(0x7f0d3c077fa0 msgr2=0x7f0d3c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.915+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 shutdown_connections 2024-04-22T13:30:32.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:30:32.915+0000 7f0d42f32640 1 -- 172.21.15.5:0/434893880 wait complete. 2024-04-22T13:30:32.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:32 smithi005 ceph-mon[28006]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 MiB/s wr, 676 op/s 2024-04-22T13:30:32.925 INFO:tasks.cephfs.filesystem:scrub status for tag:6cbaabb1-eae8-4a18-818f-d084f16bd1b7 - {'path': '/', 'tag': '6cbaabb1-eae8-4a18-818f-d084f16bd1b7', 'options': 'recursive,force'} 2024-04-22T13:30:33.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:32 smithi049 ceph-mon[31379]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 MiB/s wr, 676 op/s 2024-04-22T13:30:33.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:32 smithi107 ceph-mon[31859]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 MiB/s wr, 676 op/s 2024-04-22T13:30:34.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:33 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:30:34.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:33 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:34.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:33 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:30:34.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:33 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:34.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:33 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:30:34.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:33 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:34.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:33 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:30:34.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:33 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:34.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:33 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:30:34.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:33 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:34.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:33 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:30:34.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:33 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:35.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:34 smithi049 ceph-mon[31379]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-22T13:30:35.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:34 smithi107 ceph-mon[31859]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-22T13:30:35.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:34 smithi005 ceph-mon[28006]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-22T13:30:36.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:35 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:30:36.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:35 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:30:36.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:35 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:30:37.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:36 smithi049 ceph-mon[31379]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 597 B/s wr, 19 op/s 2024-04-22T13:30:37.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:36 smithi107 ceph-mon[31859]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 597 B/s wr, 19 op/s 2024-04-22T13:30:37.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:36 smithi005 ceph-mon[28006]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 597 B/s wr, 19 op/s 2024-04-22T13:30:39.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:38 smithi049 ceph-mon[31379]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:38 smithi107 ceph-mon[31859]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:39.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:38 smithi005 ceph-mon[28006]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:41.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:40 smithi049 ceph-mon[31379]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:41.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:40 smithi107 ceph-mon[31859]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:41.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:40 smithi005 ceph-mon[28006]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:43.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:42 smithi049 ceph-mon[31379]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:30:43.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:42 smithi107 ceph-mon[31859]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:30:43.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:42 smithi005 ceph-mon[28006]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:30:45.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:44 smithi049 ceph-mon[31379]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:45.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:44 smithi107 ceph-mon[31859]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:45.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:44 smithi005 ceph-mon[28006]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:47.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:46 smithi049 ceph-mon[31379]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:47.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:46 smithi107 ceph-mon[31859]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:47.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:46 smithi005 ceph-mon[28006]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:49.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:48 smithi049 ceph-mon[31379]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:49.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:48 smithi107 ceph-mon[31859]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:48 smithi005 ceph-mon[28006]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:51.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:50 smithi049 ceph-mon[31379]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:51.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:50 smithi005 ceph-mon[28006]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:51.158 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:50 smithi107 ceph-mon[31859]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:53.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:52 smithi049 ceph-mon[31379]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:53.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:52 smithi107 ceph-mon[31859]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:52 smithi005 ceph-mon[28006]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:54 smithi049 ceph-mon[31379]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:55.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:54 smithi107 ceph-mon[31859]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:55.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:54 smithi005 ceph-mon[28006]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:56 smithi049 ceph-mon[31379]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:57.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:56 smithi107 ceph-mon[31859]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:56 smithi005 ceph-mon[28006]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:59.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:30:58 smithi049 ceph-mon[31379]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:30:58 smithi107 ceph-mon[31859]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:59.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:30:58 smithi005 ceph-mon[28006]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:01.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:00 smithi049 ceph-mon[31379]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:01.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:00 smithi107 ceph-mon[31859]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:00 smithi005 ceph-mon[28006]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:02.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:02.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:03.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:02 smithi049 ceph-mon[31379]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:03.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:02 smithi107 ceph-mon[31859]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 -- 172.21.15.5:0/3866653018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394104d50 msgr2=0x7fe394108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 --2- 172.21.15.5:0/3866653018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394104d50 0x7fe394108d90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe388009900 tx=0x7fe38802bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 -- 172.21.15.5:0/3866653018 shutdown_connections 2024-04-22T13:31:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 --2- 172.21.15.5:0/3866653018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394104d50 0x7fe394108d90 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 --2- 172.21.15.5:0/3866653018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe394109470 0x7fe394111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 --2- 172.21.15.5:0/3866653018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3941043a0 0x7fe394104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 -- 172.21.15.5:0/3866653018 >> 172.21.15.5:0/3866653018 conn(0x7fe3940ffc00 msgr2=0x7fe394102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 -- 172.21.15.5:0/3866653018 shutdown_connections 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.049+0000 7fe399e6a640 1 -- 172.21.15.5:0/3866653018 wait complete. 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 Processor -- start 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 -- start start 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3941043a0 0x7fe3941a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe394109470 0x7fe3941a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 0x7fe39419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941143b0 con 0x7fe39419a850 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe394114230 con 0x7fe3941043a0 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe399e6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe394114530 con 0x7fe394109470 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe393fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 0x7fe39419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe393fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 0x7fe39419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42248/0 (socket says 172.21.15.5:42248) 2024-04-22T13:31:03.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.050+0000 7fe393fff640 1 -- 172.21.15.5:0/987555421 learned_addr learned my addr 172.21.15.5:0/987555421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe3937fe640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3941043a0 0x7fe3941a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe392ffd640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe394109470 0x7fe3941a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe393fff640 1 -- 172.21.15.5:0/987555421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe394109470 msgr2=0x7fe3941a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe393fff640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe394109470 0x7fe3941a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe393fff640 1 -- 172.21.15.5:0/987555421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3941043a0 msgr2=0x7fe3941a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe393fff640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3941043a0 0x7fe3941a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe393fff640 1 -- 172.21.15.5:0/987555421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39419b3e0 con 0x7fe39419a850 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe392ffd640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe394109470 0x7fe3941a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe393fff640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 0x7fe39419ad00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009fd0 tx=0x7fe38c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe390ff9640 1 -- 172.21.15.5:0/987555421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c017070 con 0x7fe39419a850 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39419b6d0 con 0x7fe39419a850 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe390ff9640 1 -- 172.21.15.5:0/987555421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe38c003e20 con 0x7fe39419a850 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe390ff9640 1 -- 172.21.15.5:0/987555421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c010460 con 0x7fe39419a850 2024-04-22T13:31:03.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.051+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a77e0 con 0x7fe39419a850 2024-04-22T13:31:03.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.053+0000 7fe390ff9640 1 -- 172.21.15.5:0/987555421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe38c003970 con 0x7fe39419a850 2024-04-22T13:31:03.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 Processor -- start 2024-04-22T13:31:03.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 -- start start 2024-04-22T13:31:03.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c3b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe390ff9640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe37c0a3d20 0x7fe37c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d11b971e0 con 0x557d11c3ba80 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d11b34390 con 0x557d11c32580 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d11b931a0 con 0x557d11c32ea0 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c3de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42254/0 (socket says 172.21.15.5:42254) 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 -- 172.21.15.5:0/73970502 learned_addr learned my addr 172.21.15.5:0/73970502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe393fff640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe37c0a3d20 0x7fe37c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe3937fe640 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c3b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe390ff9640 1 -- 172.21.15.5:0/987555421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe38c0d0570 con 0x7fe39419a850 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 -- 172.21.15.5:0/73970502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 msgr2=0x557d11c3b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c3b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 -- 172.21.15.5:0/73970502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 msgr2=0x557d11c32960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.054+0000 7fe392ffd640 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c32960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.055+0000 7fe392ffd640 1 -- 172.21.15.5:0/73970502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d119737d0 con 0x557d11c3ba80 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.055+0000 7fe393fff640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe37c0a3d20 0x7fe37c0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0a4c20 tx=0x7fe38c0980e0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.055+0000 7fe392ffd640 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c3de70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe388005c40 tx=0x7fe388005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a118eda45513d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.055+0000 7fe379ffb640 1 -- 172.21.15.5:0/73970502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38802c7e0 con 0x557d11c3ba80 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.055+0000 7fe379ffb640 1 -- 172.21.15.5:0/73970502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe388048070 con 0x557d11c3ba80 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.055+0000 7fe379ffb640 1 -- 172.21.15.5:0/73970502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388004920 con 0x557d11c3ba80 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 -- 172.21.15.5:0/73970502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 msgr2=0x557d11c3de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c3de70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe388005c40 tx=0x7fe388005bd0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 -- 172.21.15.5:0/73970502 shutdown_connections 2024-04-22T13:31:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c3de70 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c3b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/73970502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 -- 172.21.15.5:0/73970502 >> 172.21.15.5:0/73970502 conn(0x557d11c2ccb0 msgr2=0x557d11c2d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 -- 172.21.15.5:0/73970502 shutdown_connections 2024-04-22T13:31:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 -- 172.21.15.5:0/73970502 wait complete. 2024-04-22T13:31:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 -- start start 2024-04-22T13:31:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.056+0000 7fe39ca5d740 1 Processor -- start 2024-04-22T13:31:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c38b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c3ba80 0x557d11c395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d11c3e810 con 0x557d11c32ea0 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557d11b502f0 con 0x557d11c32580 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d11c3e990 con 0x557d11c3ba80 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe39ca5d740 10 client.0 _ll_register_callbacks cb 0x557d11b5d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c39080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42262/0 (socket says 172.21.15.5:42262) 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 -- 172.21.15.5:0/3310968845 learned_addr learned my addr 172.21.15.5:0/3310968845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c3ba80 0x557d11c395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c38b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 -- 172.21.15.5:0/3310968845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c3ba80 msgr2=0x557d11c395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c3ba80 0x557d11c395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 -- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 msgr2=0x557d11c38b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c38b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 -- 172.21.15.5:0/3310968845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d118a93f0 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c38b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe3937fe640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c39080 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe384012740 tx=0x7fe384012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a2c0c17db219b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.057+0000 7fe372ffd640 1 -- 172.21.15.5:0/3310968845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38400f920 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe372ffd640 1 -- 172.21.15.5:0/3310968845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe38400fac0 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe388048070 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe372ffd640 1 -- 172.21.15.5:0/3310968845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe38400fd00 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 10 client.17154 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d118690f0 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe372ffd640 1 -- 172.21.15.5:0/3310968845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe388048070 con 0x557d11c32ea0 2024-04-22T13:31:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe372ffd640 1 -- 172.21.15.5:0/3310968845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe38401c420 con 0x557d11c32ea0 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 10 client.17154 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 20 client.17154 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11c4bb00 0x557d11c4df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 4 client.17154 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.058+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d11ba4c00 con 0x557d11c4bb00 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.059+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11c4bb00 0x557d11c4df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.059+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11c4bb00 0x557d11c4df60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b471371717ed77ed server_cookie=8509f8910f1660ec in_seq=0 out_seq=0 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.059+0000 7fe372ffd640 10 client.17154 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe372ffd640 1 -- 172.21.15.5:0/3310968845 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d11ba4c00 con 0x557d11c4bb00 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe372ffd640 10 client.17154 handle_command_reply: tid=0 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 client.17154 shutdown 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 msgr2=0x557d11c39080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c39080 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe384012740 tx=0x7fe384012c00 comp rx=0 tx=0).stop 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 shutdown_connections 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11c4bb00 0x557d11c4df60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c39080 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c3ba80 0x557d11c395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3310968845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c38b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 >> 172.21.15.5:0/3310968845 conn(0x557d11c2ccb0 msgr2=0x557d11c2ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe372ffd640 0 client.17154 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 shutdown_connections 2024-04-22T13:31:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3310968845 wait complete. 2024-04-22T13:31:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.060+0000 7fe39ca5d740 20 client.17154 trim_cache size 0 max 16384 2024-04-22T13:31:03.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:02 smithi005 ceph-mon[28006]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 Processor -- start 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 -- start start 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c3b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c2bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941afcd0 con 0x557d11c3ba80 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941b1810 con 0x557d11c32580 2024-04-22T13:31:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941b03a0 con 0x557d11c32ea0 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.078+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c2bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c3b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c3b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:46874/0 (socket says 172.21.15.5:46874) 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 -- 172.21.15.5:0/843384233 learned_addr learned my addr 172.21.15.5:0/843384233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe3937fe640 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 -- 172.21.15.5:0/843384233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 msgr2=0x557d11c476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 -- 172.21.15.5:0/843384233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 msgr2=0x557d11c2bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c2bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 -- 172.21.15.5:0/843384233 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d11c31d20 con 0x557d11c32580 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe392ffd640 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c3b270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe388009900 tx=0x7fe38802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df343a1a3efe064 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe3797fa640 1 -- 172.21.15.5:0/843384233 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388056dc0 con 0x557d11c32580 2024-04-22T13:31:03.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe3797fa640 1 -- 172.21.15.5:0/843384233 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe388048070 con 0x557d11c32580 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.079+0000 7fe3797fa640 1 -- 172.21.15.5:0/843384233 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388003e60 con 0x557d11c32580 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 -- 172.21.15.5:0/843384233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 msgr2=0x557d11c3b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c3b270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe388009900 tx=0x7fe38802bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 -- 172.21.15.5:0/843384233 shutdown_connections 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c3ba80 0x557d11c2bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32ea0 0x557d11c476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/843384233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c32580 0x557d11c3b270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 -- 172.21.15.5:0/843384233 >> 172.21.15.5:0/843384233 conn(0x557d11c2ccb0 msgr2=0x557d11c2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 -- 172.21.15.5:0/843384233 shutdown_connections 2024-04-22T13:31:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 -- 172.21.15.5:0/843384233 wait complete. 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.080+0000 7fe39ca5d740 1 -- start start 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 Processor -- start 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32580 0x557d11c3f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c3fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c3ba80 0x557d11c40310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941b1140 con 0x557d11c32ea0 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941b0a70 con 0x557d11c3ba80 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941adcb0 con 0x557d11c32580 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe39ca5d740 10 client.0 _ll_register_callbacks cb 0x7fe384009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c3fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c3fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42282/0 (socket says 172.21.15.5:42282) 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 -- 172.21.15.5:0/3910436128 learned_addr learned my addr 172.21.15.5:0/3910436128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32580 0x557d11c3f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c3ba80 0x557d11c40310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 -- 172.21.15.5:0/3910436128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32580 msgr2=0x557d11c3f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32580 0x557d11c3f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 -- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c3ba80 msgr2=0x557d11c40310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c3ba80 0x557d11c40310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 -- 172.21.15.5:0/3910436128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d11cb12f0 con 0x557d11c32ea0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32580 0x557d11c3f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe3937fe640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c3fdd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x557d11c3a110 tx=0x7fe384005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45951b456a4c874 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.081+0000 7fe372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe372ffd640 1 -- 172.21.15.5:0/3910436128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384029030 con 0x557d11c32ea0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe372ffd640 1 -- 172.21.15.5:0/3910436128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe384018070 con 0x557d11c32ea0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe388048070 con 0x557d11c32ea0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe372ffd640 1 -- 172.21.15.5:0/3910436128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe384035050 con 0x557d11c32ea0 2024-04-22T13:31:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 10 client.17160 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d11cb2670 con 0x557d11c32ea0 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe372ffd640 1 -- 172.21.15.5:0/3910436128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe388048070 con 0x557d11c32ea0 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe372ffd640 1 -- 172.21.15.5:0/3910436128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe384005040 con 0x557d11c32ea0 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 10 client.17160 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 20 client.17160 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11cb2e20 0x557d11cb5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 4 client.17160 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d11b9ea70 con 0x557d11cb2e20 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.082+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11cb2e20 0x557d11cb5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe372ffd640 10 client.17160 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe392ffd640 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11cb2e20 0x557d11cb5240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374f231a7bd77131 server_cookie=bf255e33600466a5 in_seq=0 out_seq=0 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe372ffd640 1 -- 172.21.15.5:0/3910436128 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d11b9ea70 con 0x557d11cb2e20 2024-04-22T13:31:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe372ffd640 10 client.17160 handle_command_reply: tid=0 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 client.17160 shutdown 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 msgr2=0x557d11c3fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c3fdd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x557d11c3a110 tx=0x7fe384005e30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 shutdown_connections 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557d11cb2e20 0x557d11cb5240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d11c32ea0 0x557d11c3fdd0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557d11c3ba80 0x557d11c40310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe372ffd640 0 client.17160 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 --2- 172.21.15.5:0/3910436128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d11c32580 0x557d11c3f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 >> 172.21.15.5:0/3910436128 conn(0x557d11c2ccb0 msgr2=0x557d11c2e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 shutdown_connections 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 1 -- 172.21.15.5:0/3910436128 wait complete. 2024-04-22T13:31:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.083+0000 7fe39ca5d740 20 client.17160 trim_cache size 0 max 16384 2024-04-22T13:31:03.087 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:31:03.087 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:31:03.087 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:31:03.087 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:31:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe37c0a3d20 msgr2=0x7fe37c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe37c0a3d20 0x7fe37c0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0a4c20 tx=0x7fe38c0980e0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 msgr2=0x7fe39419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 0x7fe39419ad00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009fd0 tx=0x7fe38c00b260 comp rx=0 tx=0).stop 2024-04-22T13:31:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 shutdown_connections 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39419a850 0x7fe39419ad00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe37c0a3d20 0x7fe37c0a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe394109470 0x7fe3941a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 --2- 172.21.15.5:0/987555421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3941043a0 0x7fe3941a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 >> 172.21.15.5:0/987555421 conn(0x7fe3940ffc00 msgr2=0x7fe394101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 shutdown_connections 2024-04-22T13:31:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.084+0000 7fe399e6a640 1 -- 172.21.15.5:0/987555421 wait complete. 2024-04-22T13:31:03.091 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:31:03.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:31:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 -- 172.21.15.5:0/344830896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 msgr2=0x7f6bc4108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 --2- 172.21.15.5:0/344830896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc4108440 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009930 tx=0x7f6bc002bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 -- 172.21.15.5:0/344830896 shutdown_connections 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 --2- 172.21.15.5:0/344830896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 0x7f6bc410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 --2- 172.21.15.5:0/344830896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bc406a640 0x7f6bc406aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 --2- 172.21.15.5:0/344830896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc4108440 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 -- 172.21.15.5:0/344830896 >> 172.21.15.5:0/344830896 conn(0x7f6bc4077c40 msgr2=0x7f6bc4078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 -- 172.21.15.5:0/344830896 shutdown_connections 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.197+0000 7f6bcc166640 1 -- 172.21.15.5:0/344830896 wait complete. 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 Processor -- start 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 -- start start 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bc406a640 0x7f6bc41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 0x7f6bc4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41121a0 con 0x7f6bc4108060 2024-04-22T13:31:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4112020 con 0x7f6bc406a640 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bcc166640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4112320 con 0x7f6bc406b000 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bca703640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bca703640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42300/0 (socket says 172.21.15.5:42300) 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bca703640 1 -- 172.21.15.5:0/3239716737 learned_addr learned my addr 172.21.15.5:0/3239716737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.198+0000 7f6bc9f02640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bc406a640 0x7f6bc41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bc9701640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 0x7f6bc4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bca703640 1 -- 172.21.15.5:0/3239716737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 msgr2=0x7f6bc4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bca703640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 0x7f6bc4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bca703640 1 -- 172.21.15.5:0/3239716737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bc406a640 msgr2=0x7f6bc41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bca703640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bc406a640 0x7f6bc41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bca703640 1 -- 172.21.15.5:0/3239716737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419d680 con 0x7f6bc4108060 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bc9701640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 0x7f6bc4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bca703640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc419cf50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00eaa0 tx=0x7f6bbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00cdb0 con 0x7f6bc4108060 2024-04-22T13:31:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bbc005020 con 0x7f6bc4108060 2024-04-22T13:31:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419d970 con 0x7f6bc4108060 2024-04-22T13:31:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0116d0 con 0x7f6bc4108060 2024-04-22T13:31:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.199+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a5200 con 0x7f6bc4108060 2024-04-22T13:31:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.200+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc4103650 con 0x7f6bc4108060 2024-04-22T13:31:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.202+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6bbc0051d0 con 0x7f6bc4108060 2024-04-22T13:31:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.202+0000 7f6bbaffd640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bb00a3d20 0x7f6bb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.202+0000 7f6bc9f02640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bb00a3d20 0x7f6bb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.203+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f6bbc0d0ce0 con 0x7f6bc4108060 2024-04-22T13:31:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.203+0000 7f6bc9f02640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bb00a3d20 0x7f6bb00a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0005ab0 tx=0x7f6bc0005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.206+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bbc016030 con 0x7f6bc4108060 2024-04-22T13:31:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.433+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bc41999e0 con 0x7f6bc4108060 2024-04-22T13:31:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.434+0000 7f6bbaffd640 1 -- 172.21.15.5:0/3239716737 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f6bbc09d3b0 con 0x7f6bc4108060 2024-04-22T13:31:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:31:03.430 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:31:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:31:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bb00a3d20 msgr2=0x7f6bb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bb00a3d20 0x7f6bb00a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0005ab0 tx=0x7f6bc0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 msgr2=0x7f6bc419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc419cf50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00eaa0 tx=0x7f6bbc00ef60 comp rx=0 tx=0).stop 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 shutdown_connections 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc4108060 0x7f6bc419cf50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bb00a3d20 0x7f6bb00a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc406b000 0x7f6bc4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 --2- 172.21.15.5:0/3239716737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bc406a640 0x7f6bc41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 >> 172.21.15.5:0/3239716737 conn(0x7f6bc4077c40 msgr2=0x7f6bc40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 shutdown_connections 2024-04-22T13:31:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.436+0000 7f6bcc166640 1 -- 172.21.15.5:0/3239716737 wait complete. 2024-04-22T13:31:03.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 -- 172.21.15.5:0/987686225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea41044f0 msgr2=0x7f3ea410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/987686225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea41044f0 0x7f3ea410eef0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009930 tx=0x7f3e9402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 -- 172.21.15.5:0/987686225 shutdown_connections 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/987686225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea41044f0 0x7f3ea410eef0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/987686225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea4103b50 0x7f3ea4103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/987686225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4102270 0x7f3ea4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 -- 172.21.15.5:0/987686225 >> 172.21.15.5:0/987686225 conn(0x7f3ea40fda10 msgr2=0x7f3ea40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 -- 172.21.15.5:0/987686225 shutdown_connections 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.554+0000 7f3eab3e6640 1 -- 172.21.15.5:0/987686225 wait complete. 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 Processor -- start 2024-04-22T13:31:03.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 -- start start 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 0x7f3ea419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4103b50 0x7f3ea419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea41044f0 0x7f3ea4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41122f0 con 0x7f3ea4102270 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4112170 con 0x7f3ea4103b50 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3eab3e6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4112470 con 0x7f3ea41044f0 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 0x7f3ea419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 0x7f3ea419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42320/0 (socket says 172.21.15.5:42320) 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 -- 172.21.15.5:0/4154610461 learned_addr learned my addr 172.21.15.5:0/4154610461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea8981640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4103b50 0x7f3ea419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9983640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea41044f0 0x7f3ea4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 -- 172.21.15.5:0/4154610461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea41044f0 msgr2=0x7f3ea4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea41044f0 0x7f3ea4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 -- 172.21.15.5:0/4154610461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4103b50 msgr2=0x7f3ea419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4103b50 0x7f3ea419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 -- 172.21.15.5:0/4154610461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea4198db0 con 0x7f3ea4102270 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9983640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea41044f0 0x7f3ea4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.555+0000 7f3ea9182640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 0x7f3ea419e430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00ba50 tx=0x7f3e9c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.556+0000 7f3ea27fc640 1 -- 172.21.15.5:0/4154610461 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c019070 con 0x7f3ea4102270 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.556+0000 7f3ea27fc640 1 -- 172.21.15.5:0/4154610461 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9c00cbe0 con 0x7f3ea4102270 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.556+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea41990a0 con 0x7f3ea4102270 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.556+0000 7f3ea27fc640 1 -- 172.21.15.5:0/4154610461 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c0075d0 con 0x7f3ea4102270 2024-04-22T13:31:03.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.556+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41a52f0 con 0x7f3ea4102270 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.558+0000 7f3ea27fc640 1 -- 172.21.15.5:0/4154610461 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3e9c01c070 con 0x7f3ea4102270 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.558+0000 7f3eadfd4740 1 Processor -- start 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.558+0000 7f3eadfd4740 1 -- start start 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 0x55fa91b45710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b4e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b4f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa91aaa1f0 con 0x55fa91b45330 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fa91a474b0 con 0x55fa91b45ce0 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa91aa61b0 con 0x55fa91b4ee40 2024-04-22T13:31:03.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 0x55fa91b45710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 0x55fa91b45710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42336/0 (socket says 172.21.15.5:42336) 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 -- 172.21.15.5:0/669079076 learned_addr learned my addr 172.21.15.5:0/669079076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea27fc640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3e8c0a3d20 0x7f3e8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea8981640 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b4f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 -- 172.21.15.5:0/669079076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 msgr2=0x55fa91b4f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9182640 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b4e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b4f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 -- 172.21.15.5:0/669079076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 msgr2=0x55fa91b4e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea27fc640 1 -- 172.21.15.5:0/4154610461 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3e9c0cfc90 con 0x7f3ea4102270 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b4e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 -- 172.21.15.5:0/669079076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa91a391b0 con 0x55fa91b45330 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9182640 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b4e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3e8c0a3d20 0x7f3e8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.559+0000 7f3ea9983640 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 0x55fa91b45710 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009230 tx=0x7f3e9402c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5078e6cba10e3277 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3e8b7fe640 1 -- 172.21.15.5:0/669079076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94045070 con 0x55fa91b45330 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3ea9983640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3e8c0a3d20 0x7f3e8c0a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e9404b000 tx=0x7f3e9404c000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3e8b7fe640 1 -- 172.21.15.5:0/669079076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e94004b70 con 0x55fa91b45330 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3e8b7fe640 1 -- 172.21.15.5:0/669079076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940038e0 con 0x55fa91b45330 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 -- 172.21.15.5:0/669079076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 msgr2=0x55fa91b45710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 0x55fa91b45710 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009230 tx=0x7f3e9402c700 comp rx=0 tx=0).stop 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 -- 172.21.15.5:0/669079076 shutdown_connections 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b4f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b4e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/669079076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45330 0x55fa91b45710 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 -- 172.21.15.5:0/669079076 >> 172.21.15.5:0/669079076 conn(0x55fa91b3fba0 msgr2=0x55fa91b3ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 -- 172.21.15.5:0/669079076 shutdown_connections 2024-04-22T13:31:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.560+0000 7f3eadfd4740 1 -- 172.21.15.5:0/669079076 wait complete. 2024-04-22T13:31:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 -- start start 2024-04-22T13:31:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 Processor -- start 2024-04-22T13:31:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45ce0 0x55fa91b52600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b52ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa91b55dc0 con 0x55fa91b4ee40 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55fa91a63500 con 0x55fa91b45330 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa91b55f40 con 0x55fa91b45ce0 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3eadfd4740 10 client.0 _ll_register_callbacks cb 0x55fa91a31b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3ea9983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b52ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.561+0000 7f3ea9983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b52ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42344/0 (socket says 172.21.15.5:42344) 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 -- 172.21.15.5:0/2660478639 learned_addr learned my addr 172.21.15.5:0/2660478639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea8981640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45ce0 0x55fa91b52600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9182640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 -- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45ce0 msgr2=0x55fa91b52600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45ce0 0x55fa91b52600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 -- 172.21.15.5:0/2660478639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 msgr2=0x55fa91b55270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b55270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 -- 172.21.15.5:0/2660478639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa917bc3f0 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3ea9983640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b52ba0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3e940055b0 tx=0x7f3e94009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d86f309f761b503 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3e88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3e88ff9640 1 -- 172.21.15.5:0/2660478639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94058070 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3e88ff9640 1 -- 172.21.15.5:0/2660478639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3e94004f20 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e94004b70 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3e88ff9640 1 -- 172.21.15.5:0/2660478639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e94064050 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3eadfd4740 10 client.17178 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.562+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa918867d0 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3e88ff9640 1 -- 172.21.15.5:0/2660478639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e94004b70 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3e88ff9640 1 -- 172.21.15.5:0/2660478639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3e94053040 con 0x55fa91b4ee40 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3eadfd4740 10 client.17178 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3eadfd4740 10 client.17178 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3eadfd4740 20 client.17178 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91b62ea0 0x55fa91b65290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3eadfd4740 4 client.17178 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa91a49930 con 0x55fa91b62ea0 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3ea9182640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91b62ea0 0x55fa91b65290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3ea9182640 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91b62ea0 0x55fa91b65290 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6a26ad747a2a5db server_cookie=e5d1285bf6207c77 in_seq=0 out_seq=0 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.563+0000 7f3e88ff9640 10 client.17178 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3e88ff9640 1 -- 172.21.15.5:0/2660478639 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa91a49930 con 0x55fa91b62ea0 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3e88ff9640 10 client.17178 handle_command_reply: tid=0 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 client.17178 shutdown 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 msgr2=0x55fa91b52ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b52ba0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3e940055b0 tx=0x7f3e94009260 comp rx=0 tx=0).stop 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 shutdown_connections 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91b62ea0 0x55fa91b65290 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b52ba0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3e88ff9640 0 client.17178 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45ce0 0x55fa91b52600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/2660478639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b55270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 >> 172.21.15.5:0/2660478639 conn(0x55fa91b3fba0 msgr2=0x55fa91b418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 shutdown_connections 2024-04-22T13:31:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 1 -- 172.21.15.5:0/2660478639 wait complete. 2024-04-22T13:31:03.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.564+0000 7f3eadfd4740 20 client.17178 trim_cache size 0 max 16384 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 Processor -- start 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 -- start start 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b50230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 0x55fa91b54d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41ab820 con 0x55fa91b45ce0 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41ad360 con 0x55fa91b45330 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41abef0 con 0x55fa91b4ee40 2024-04-22T13:31:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 0x55fa91b54d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 0x55fa91b54d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42352/0 (socket says 172.21.15.5:42352) 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 -- 172.21.15.5:0/1890899426 learned_addr learned my addr 172.21.15.5:0/1890899426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea9983640 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea9182640 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b50230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 -- 172.21.15.5:0/1890899426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 msgr2=0x55fa91b55270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b55270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 -- 172.21.15.5:0/1890899426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 msgr2=0x55fa91b50230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b50230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.578+0000 7f3ea8981640 1 -- 172.21.15.5:0/1890899426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa91b44560 con 0x55fa91b45ce0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3ea8981640 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 0x55fa91b54d30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005d00 tx=0x7f3e98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648c93a580320db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3e8affd640 1 -- 172.21.15.5:0/1890899426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004020 con 0x55fa91b45ce0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3e8affd640 1 -- 172.21.15.5:0/1890899426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e980054d0 con 0x55fa91b45ce0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3e8affd640 1 -- 172.21.15.5:0/1890899426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004ca0 con 0x55fa91b45ce0 2024-04-22T13:31:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 -- 172.21.15.5:0/1890899426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 msgr2=0x55fa91b54d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 0x55fa91b54d30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005d00 tx=0x7f3e98011810 comp rx=0 tx=0).stop 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 -- 172.21.15.5:0/1890899426 shutdown_connections 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b45ce0 0x55fa91b54d30 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b4ee40 0x55fa91b55270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/1890899426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45330 0x55fa91b50230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 -- 172.21.15.5:0/1890899426 >> 172.21.15.5:0/1890899426 conn(0x55fa91b3fba0 msgr2=0x55fa91b5ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 -- 172.21.15.5:0/1890899426 shutdown_connections 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.579+0000 7f3eadfd4740 1 -- 172.21.15.5:0/1890899426 wait complete. 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 -- start start 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 Processor -- start 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45330 0x55fa91b68730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b65b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41acc90 con 0x55fa91b4ee40 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41ac5c0 con 0x55fa91b45ce0 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41a9800 con 0x55fa91b45330 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3eadfd4740 10 client.0 _ll_register_callbacks cb 0x7f3e9402c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b66160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42356/0 (socket says 172.21.15.5:42356) 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 -- 172.21.15.5:0/3767367984 learned_addr learned my addr 172.21.15.5:0/3767367984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea8981640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b65b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9182640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45330 0x55fa91b68730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 -- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45330 msgr2=0x55fa91b68730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45330 0x55fa91b68730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 -- 172.21.15.5:0/3767367984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 msgr2=0x55fa91b65b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.580+0000 7f3ea9983640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b65b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3ea9983640 1 -- 172.21.15.5:0/3767367984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa91b66960 con 0x55fa91b4ee40 2024-04-22T13:31:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3ea8981640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b65b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3ea9983640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b66160 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3e9402c3a0 tx=0x7f3e940049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66abe31ba2e86a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3e88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3e88ff9640 1 -- 172.21.15.5:0/3767367984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94058040 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3e88ff9640 1 -- 172.21.15.5:0/3767367984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3e94004b70 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e980054d0 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3e88ff9640 1 -- 172.21.15.5:0/3767367984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e94045040 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 10 client.17184 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa91bc91b0 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3e88ff9640 1 -- 172.21.15.5:0/3767367984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e980054d0 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3e88ff9640 1 -- 172.21.15.5:0/3767367984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3e94053040 con 0x55fa91b4ee40 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 10 client.17184 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 10 client.17184 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:31:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 20 client.17184 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91bc9830 0x55fa91bcbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 4 client.17184 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.581+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa91ab7c10 con 0x55fa91bc9830 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.582+0000 7f3ea9182640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91bc9830 0x55fa91bcbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.582+0000 7f3ea9182640 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91bc9830 0x55fa91bcbc50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74da1d8264280f54 server_cookie=532e15012012f135 in_seq=0 out_seq=0 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.582+0000 7f3e88ff9640 10 client.17184 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.582+0000 7f3e88ff9640 1 -- 172.21.15.5:0/3767367984 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa91ab7c10 con 0x55fa91bc9830 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.582+0000 7f3e88ff9640 10 client.17184 handle_command_reply: tid=0 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.582+0000 7f3eadfd4740 1 client.17184 shutdown 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 msgr2=0x55fa91b66160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b66160 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3e9402c3a0 tx=0x7f3e940049d0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 shutdown_connections 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55fa91bc9830 0x55fa91bcbc50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa91b4ee40 0x55fa91b66160 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55fa91b45ce0 0x55fa91b65b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3e88ff9640 0 client.17184 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 --2- 172.21.15.5:0/3767367984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa91b45330 0x55fa91b68730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 >> 172.21.15.5:0/3767367984 conn(0x55fa91b3fba0 msgr2=0x55fa91b3e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 shutdown_connections 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 1 -- 172.21.15.5:0/3767367984 wait complete. 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eadfd4740 20 client.17184 trim_cache size 0 max 16384 2024-04-22T13:31:03.588 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3e8c0a3d20 msgr2=0x7f3e8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3e8c0a3d20 0x7f3e8c0a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e9404b000 tx=0x7f3e9404c000 comp rx=0 tx=0).stop 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 msgr2=0x7f3ea419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 0x7f3ea419e430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00ba50 tx=0x7f3e9c00bf10 comp rx=0 tx=0).stop 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 shutdown_connections 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3e8c0a3d20 0x7f3e8c0a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ea41044f0 0x7f3ea4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea4103b50 0x7f3ea419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 --2- 172.21.15.5:0/4154610461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea4102270 0x7f3ea419e430 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 >> 172.21.15.5:0/4154610461 conn(0x7f3ea40fda10 msgr2=0x7f3ea40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 shutdown_connections 2024-04-22T13:31:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.583+0000 7f3eab3e6640 1 -- 172.21.15.5:0/4154610461 wait complete. 2024-04-22T13:31:03.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:31:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.699+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1372666085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c102b30 msgr2=0x7f8b2c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.699+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1372666085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c102b30 0x7f8b2c106b70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009900 tx=0x7f8b2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1372666085 shutdown_connections 2024-04-22T13:31:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1372666085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c102b30 0x7f8b2c106b70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1372666085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b2c107250 0x7f8b2c10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1372666085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b2c102180 0x7f8b2c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1372666085 >> 172.21.15.5:0/1372666085 conn(0x7f8b2c0fda00 msgr2=0x7f8b2c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1372666085 shutdown_connections 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1372666085 wait complete. 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 Processor -- start 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- start start 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b2c102180 0x7f8b2c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 0x7f8b2c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b2c19cfd0 0x7f8b2c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c112170 con 0x7f8b2c107250 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c111ff0 con 0x7f8b2c19cfd0 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.700+0000 7f8b31a5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1122f0 con 0x7f8b2c102180 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 0x7f8b2c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 0x7f8b2c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42376/0 (socket says 172.21.15.5:42376) 2024-04-22T13:31:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1336104130 learned_addr learned my addr 172.21.15.5:0/1336104130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b2c19cfd0 0x7f8b2c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2affd640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b2c102180 0x7f8b2c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1336104130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b2c102180 msgr2=0x7f8b2c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b2c102180 0x7f8b2c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1336104130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b2c19cfd0 msgr2=0x7f8b2c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b2c19cfd0 0x7f8b2c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1336104130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c19dac0 con 0x7f8b2c107250 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 0x7f8b2c198c50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8b20005ec0 tx=0x7f8b2002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b30a5a640 1 -- 172.21.15.5:0/1336104130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20043020 con 0x7f8b2c107250 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b30a5a640 1 -- 172.21.15.5:0/1336104130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b200052b0 con 0x7f8b2c107250 2024-04-22T13:31:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c19dd50 con 0x7f8b2c107250 2024-04-22T13:31:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b30a5a640 1 -- 172.21.15.5:0/1336104130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20003d40 con 0x7f8b2c107250 2024-04-22T13:31:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.701+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a5570 con 0x7f8b2c107250 2024-04-22T13:31:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b30a5a640 1 -- 172.21.15.5:0/1336104130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8b20046020 con 0x7f8b2c107250 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 Processor -- start 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 -- start start 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56094f0331f0 con 0x56094f0cee80 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56094efd04b0 con 0x56094f0ce4d0 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b34654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56094f02f1b0 con 0x56094f0d7a00 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b30a5a640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8b140a3d20 0x7f8b140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.704+0000 7f8b2affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42386/0 (socket says 172.21.15.5:42386) 2024-04-22T13:31:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8b140a3d20 0x7f8b140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 -- 172.21.15.5:0/3668607931 learned_addr learned my addr 172.21.15.5:0/3668607931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b30a5a640 1 -- 172.21.15.5:0/1336104130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8b200f2a50 con 0x7f8b2c107250 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 -- 172.21.15.5:0/3668607931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 msgr2=0x56094f0d9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 -- 172.21.15.5:0/3668607931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 msgr2=0x56094f0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 -- 172.21.15.5:0/3668607931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094efc21b0 con 0x56094f0cee80 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2affd640 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d7430 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00cc60 tx=0x7f8b1c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef053f54c6f3671 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8b140a3d20 0x7f8b140a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005cc0 tx=0x7f8b2400ad50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b11ffb640 1 -- 172.21.15.5:0/3668607931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c014070 con 0x56094f0cee80 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b11ffb640 1 -- 172.21.15.5:0/3668607931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1c0047a0 con 0x56094f0cee80 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.705+0000 7f8b11ffb640 1 -- 172.21.15.5:0/3668607931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c010690 con 0x56094f0cee80 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 -- 172.21.15.5:0/3668607931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 msgr2=0x56094f0d7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d7430 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00cc60 tx=0x7f8b1c00e720 comp rx=0 tx=0).stop 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 -- 172.21.15.5:0/3668607931 shutdown_connections 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d7430 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 --2- 172.21.15.5:0/3668607931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 -- 172.21.15.5:0/3668607931 >> 172.21.15.5:0/3668607931 conn(0x56094f0c8ba0 msgr2=0x56094f0c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 -- 172.21.15.5:0/3668607931 shutdown_connections 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 -- 172.21.15.5:0/3668607931 wait complete. 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 -- start start 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.706+0000 7f8b34654740 1 Processor -- start 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56094f0da970 con 0x56094f0cee80 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56094efec500 con 0x56094f0ce4d0 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56094f0daaf0 con 0x56094f0d7a00 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b34654740 10 client.0 _ll_register_callbacks cb 0x56094efbab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42390/0 (socket says 172.21.15.5:42390) 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/952824034 learned_addr learned my addr 172.21.15.5:0/952824034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2affd640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/952824034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 msgr2=0x56094f0d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 msgr2=0x56094f0d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/952824034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094ed453f0 con 0x56094f0cee80 2024-04-22T13:31:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d4630 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8b200bb620 tx=0x7f8b200be850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6f2837a3ba1e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b0affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b0affd640 1 -- 172.21.15.5:0/952824034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200d8020 con 0x56094f0cee80 2024-04-22T13:31:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.707+0000 7f8b0affd640 1 -- 172.21.15.5:0/952824034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8b200bb2c0 con 0x56094f0cee80 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b1c0047a0 con 0x56094f0cee80 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b0affd640 1 -- 172.21.15.5:0/952824034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8b200d81c0 con 0x56094f0cee80 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 10 client.17196 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56094ee0f7d0 con 0x56094f0cee80 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b0affd640 1 -- 172.21.15.5:0/952824034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1c0047a0 con 0x56094f0cee80 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b0affd640 1 -- 172.21.15.5:0/952824034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8b200c7430 con 0x56094f0cee80 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 10 client.17196 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 10 client.17196 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 20 client.17196 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f0e7c10 0x56094f0ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 4 client.17196 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56094efd2930 con 0x56094f0e7c10 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.708+0000 7f8b2affd640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f0e7c10 0x56094f0ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.709+0000 7f8b0affd640 10 client.17196 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.709+0000 7f8b2affd640 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f0e7c10 0x56094f0ea040 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7244223f653a224f server_cookie=204b3720d2f43551 in_seq=0 out_seq=0 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b0affd640 1 -- 172.21.15.5:0/952824034 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56094efd2930 con 0x56094f0e7c10 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b0affd640 10 client.17196 handle_command_reply: tid=0 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 client.17196 shutdown 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 msgr2=0x56094f0d4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d4630 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8b200bb620 tx=0x7f8b200be850 comp rx=0 tx=0).stop 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 shutdown_connections 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f0e7c10 0x56094f0ea040 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0d4630 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0d7a00 0x56094f0d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b0affd640 0 client.17196 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 --2- 172.21.15.5:0/952824034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0ce4d0 0x56094f0d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 >> 172.21.15.5:0/952824034 conn(0x56094f0c8ba0 msgr2=0x56094f0ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 shutdown_connections 2024-04-22T13:31:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 1 -- 172.21.15.5:0/952824034 wait complete. 2024-04-22T13:31:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.710+0000 7f8b34654740 20 client.17196 trim_cache size 0 max 16384 2024-04-22T13:31:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 Processor -- start 2024-04-22T13:31:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 -- start start 2024-04-22T13:31:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0ce4d0 0x56094f0e3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1ada20 con 0x56094f0cee80 2024-04-22T13:31:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1af560 con 0x56094f0d7a00 2024-04-22T13:31:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.723+0000 7f8b34654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1ae0f0 con 0x56094f0ce4d0 2024-04-22T13:31:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42398/0 (socket says 172.21.15.5:42398) 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1871878745 learned_addr learned my addr 172.21.15.5:0/1871878745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2affd640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0ce4d0 0x56094f0e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1871878745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0ce4d0 msgr2=0x56094f0e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0ce4d0 0x56094f0e3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1871878745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 msgr2=0x56094f0d6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0d6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 -- 172.21.15.5:0/1871878745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094f0ce080 con 0x56094f0cee80 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2affd640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0ce4d0 0x56094f0e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0d6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0cf9c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8b200bebe0 tx=0x7f8b200bd160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c3c234c4f7f7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b117fa640 1 -- 172.21.15.5:0/1871878745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200a1070 con 0x56094f0cee80 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b117fa640 1 -- 172.21.15.5:0/1871878745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b200d81c0 con 0x56094f0cee80 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.724+0000 7f8b117fa640 1 -- 172.21.15.5:0/1871878745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b200c7430 con 0x56094f0cee80 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 -- 172.21.15.5:0/1871878745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 msgr2=0x56094f0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0cf9c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8b200bebe0 tx=0x7f8b200bd160 comp rx=0 tx=0).stop 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 -- 172.21.15.5:0/1871878745 shutdown_connections 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0cee80 0x56094f0cf9c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0d6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 --2- 172.21.15.5:0/1871878745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0ce4d0 0x56094f0e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 -- 172.21.15.5:0/1871878745 >> 172.21.15.5:0/1871878745 conn(0x56094f0c8ba0 msgr2=0x56094f0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 -- 172.21.15.5:0/1871878745 shutdown_connections 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 -- 172.21.15.5:0/1871878745 wait complete. 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 -- start start 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.725+0000 7f8b34654740 1 Processor -- start 2024-04-22T13:31:03.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 0x56094f0dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0cee80 0x56094f0db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1aee90 con 0x56094f0ce4d0 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1ae7c0 con 0x56094f0d7a00 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1aba20 con 0x56094f0cee80 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 10 client.0 _ll_register_callbacks cb 0x7f8b200baef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 0x56094f0dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 0x56094f0dff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42404/0 (socket says 172.21.15.5:42404) 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 -- 172.21.15.5:0/4110500389 learned_addr learned my addr 172.21.15.5:0/4110500389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0cee80 0x56094f0db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 -- 172.21.15.5:0/4110500389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0cee80 msgr2=0x56094f0db2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0cee80 0x56094f0db2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 -- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 msgr2=0x56094f0db810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0db810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 -- 172.21.15.5:0/4110500389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094f0dbe50 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2b7fe640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0cee80 0x56094f0db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b2affd640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 0x56094f0dff60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00cc60 tx=0x7f8b1c00c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f7fd8bbe6f60bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b0affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b0affd640 1 -- 172.21.15.5:0/4110500389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c014040 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b0affd640 1 -- 172.21.15.5:0/4110500389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8b1c004240 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b200d81c0 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b0affd640 1 -- 172.21.15.5:0/4110500389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8b1c004440 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.726+0000 7f8b34654740 10 client.17202 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56094f14df90 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b0affd640 1 -- 172.21.15.5:0/4110500389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b200d81c0 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b0affd640 1 -- 172.21.15.5:0/4110500389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8b1c003930 con 0x56094f0ce4d0 2024-04-22T13:31:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 10 client.17202 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 10 client.17202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 20 client.17202 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f14e460 0x56094f150880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 4 client.17202 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56094f0d4f00 con 0x56094f14e460 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.727+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f14e460 0x56094f150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b2a7fc640 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f14e460 0x56094f150880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15deb327480d5801 server_cookie=ba257dddfe41c84d in_seq=0 out_seq=0 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b0affd640 10 client.17202 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b0affd640 1 -- 172.21.15.5:0/4110500389 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56094f0d4f00 con 0x56094f14e460 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b0affd640 10 client.17202 handle_command_reply: tid=0 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 client.17202 shutdown 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 msgr2=0x56094f0dff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 0x56094f0dff60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00cc60 tx=0x7f8b1c00c340 comp rx=0 tx=0).stop 2024-04-22T13:31:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 shutdown_connections 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56094f14e460 0x56094f150880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56094f0d7a00 0x56094f0db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56094f0cee80 0x56094f0db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b0affd640 0 client.17202 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 --2- 172.21.15.5:0/4110500389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56094f0ce4d0 0x56094f0dff60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 >> 172.21.15.5:0/4110500389 conn(0x56094f0c8ba0 msgr2=0x56094f0c7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 shutdown_connections 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 1 -- 172.21.15.5:0/4110500389 wait complete. 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.728+0000 7f8b34654740 20 client.17202 trim_cache size 0 max 16384 2024-04-22T13:31:03.731 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:03.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8b140a3d20 msgr2=0x7f8b140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8b140a3d20 0x7f8b140a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005cc0 tx=0x7f8b2400ad50 comp rx=0 tx=0).stop 2024-04-22T13:31:03.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 msgr2=0x7f8b2c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 0x7f8b2c198c50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8b20005ec0 tx=0x7f8b2002ca10 comp rx=0 tx=0).stop 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 shutdown_connections 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b2c107250 0x7f8b2c198c50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8b140a3d20 0x7f8b140a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b2c19cfd0 0x7f8b2c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 --2- 172.21.15.5:0/1336104130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b2c102180 0x7f8b2c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 >> 172.21.15.5:0/1336104130 conn(0x7f8b2c0fda00 msgr2=0x7f8b2c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 shutdown_connections 2024-04-22T13:31:03.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.729+0000 7f8b31a5c640 1 -- 172.21.15.5:0/1336104130 wait complete. 2024-04-22T13:31:03.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.844+0000 7f1fc9c81640 1 -- 172.21.15.5:0/648394807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc4103ae0 msgr2=0x7f1fc410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.844+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/648394807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc4103ae0 0x7f1fc410bfc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009900 tx=0x7f1fb802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 -- 172.21.15.5:0/648394807 shutdown_connections 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/648394807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc4103ae0 0x7f1fc410bfc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/648394807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc410c500 0x7f1fc410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/648394807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc41031c0 0x7f1fc41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 -- 172.21.15.5:0/648394807 >> 172.21.15.5:0/648394807 conn(0x7f1fc4078000 msgr2=0x7f1fc4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 -- 172.21.15.5:0/648394807 shutdown_connections 2024-04-22T13:31:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 -- 172.21.15.5:0/648394807 wait complete. 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 Processor -- start 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.845+0000 7f1fc9c81640 1 -- start start 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 0x7f1fc4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc4103ae0 0x7f1fc4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc410c500 0x7f1fc419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc4112170 con 0x7f1fc41031c0 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc4111ff0 con 0x7f1fc4103ae0 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41122f0 con 0x7f1fc410c500 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 0x7f1fc4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 0x7f1fc4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42414/0 (socket says 172.21.15.5:42414) 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3317903907 learned_addr learned my addr 172.21.15.5:0/3317903907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc4103ae0 0x7f1fc4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc410c500 0x7f1fc419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3317903907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc410c500 msgr2=0x7f1fc419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc410c500 0x7f1fc419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3317903907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc4103ae0 msgr2=0x7f1fc4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc4103ae0 0x7f1fc4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3317903907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc419d6f0 con 0x7f1fc41031c0 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc410c500 0x7f1fc419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 0x7f1fc4198750 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400d950 tx=0x7f1fb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc0ff9640 1 -- 172.21.15.5:0/3317903907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4017030 con 0x7f1fc41031c0 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc0ff9640 1 -- 172.21.15.5:0/3317903907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb4004020 con 0x7f1fc41031c0 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc419d9e0 con 0x7f1fc41031c0 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc0ff9640 1 -- 172.21.15.5:0/3317903907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40047a0 con 0x7f1fc41031c0 2024-04-22T13:31:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.846+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc41a5270 con 0x7f1fc41031c0 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fc0ff9640 1 -- 172.21.15.5:0/3317903907 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1fb4009a30 con 0x7f1fc41031c0 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 Processor -- start 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 -- start start 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75e94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a75df91f0 con 0x555a75e94c00 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555a75d964b0 con 0x555a75ea1e10 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fcc874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555a75df51b0 con 0x555a75e94250 2024-04-22T13:31:03.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42426/0 (socket says 172.21.15.5:42426) 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3290733651 learned_addr learned my addr 172.21.15.5:0/3290733651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.849+0000 7f1fc0ff9640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1fac0a3d20 0x7f1fac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75e94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3290733651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 msgr2=0x555a75e94630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75e94630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3290733651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 msgr2=0x555a75ea4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc37fe640 1 -- 172.21.15.5:0/3290733651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a75d881b0 con 0x555a75e94c00 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75e94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1fac0a3d20 0x7f1fac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc0ff9640 1 -- 172.21.15.5:0/3317903907 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1fb40d0230 con 0x7f1fc41031c0 2024-04-22T13:31:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1840 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cd950 tx=0x7f1fb405bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ea533ef45a83b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fa9ffb640 1 -- 172.21.15.5:0/3290733651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb409bfd0 con 0x555a75e94c00 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1fac0a3d20 0x7f1fac0a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0092f0 tx=0x7f1fbc004a40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fa9ffb640 1 -- 172.21.15.5:0/3290733651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb409c900 con 0x555a75e94c00 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.850+0000 7f1fa9ffb640 1 -- 172.21.15.5:0/3290733651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40071c0 con 0x555a75e94c00 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 -- 172.21.15.5:0/3290733651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 msgr2=0x555a75ea1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1840 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cd950 tx=0x7f1fb405bca0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 -- 172.21.15.5:0/3290733651 shutdown_connections 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1840 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 --2- 172.21.15.5:0/3290733651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75e94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 -- 172.21.15.5:0/3290733651 >> 172.21.15.5:0/3290733651 conn(0x555a75e8eba0 msgr2=0x555a75e8ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 -- 172.21.15.5:0/3290733651 shutdown_connections 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 -- 172.21.15.5:0/3290733651 wait complete. 2024-04-22T13:31:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 -- start start 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.851+0000 7f1fcc874740 1 Processor -- start 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94250 0x555a75ea1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75e9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75e9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a75ea4ce0 con 0x555a75e94c00 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555a75db2500 con 0x555a75e94250 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555a75ea4e60 con 0x555a75ea1e10 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fcc874740 10 client.0 _ll_register_callbacks cb 0x555a75d80b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75e9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75e9ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42428/0 (socket says 172.21.15.5:42428) 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/618074135 learned_addr learned my addr 172.21.15.5:0/618074135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94250 0x555a75ea1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75e9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 msgr2=0x555a75e9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75e9ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/618074135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94250 msgr2=0x555a75ea1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94250 0x555a75ea1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/618074135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a75b0b3f0 con 0x555a75e94c00 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75e9ea00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009900 tx=0x7f1fb8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f36646e7b1e88e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.852+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/618074135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8044070 con 0x555a75e94c00 2024-04-22T13:31:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb409c900 con 0x555a75e94c00 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/618074135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1fb802c880 con 0x555a75e94c00 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/618074135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fb8050050 con 0x555a75e94c00 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 10 client.17214 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a75bd57d0 con 0x555a75e94c00 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/618074135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb409c900 con 0x555a75e94c00 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/618074135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fb80035f0 con 0x555a75e94c00 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 10 client.17214 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 10 client.17214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 20 client.17214 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75eb1e60 0x555a75eb42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 4 client.17214 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.853+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a75d98930 con 0x555a75eb1e60 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.854+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75eb1e60 0x555a75eb42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.854+0000 7f1fa2ffd640 10 client.17214 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.854+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75eb1e60 0x555a75eb42e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f11ec0fc6fa8c7f4 server_cookie=bffe0ba3ff671b88 in_seq=0 out_seq=0 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/618074135 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a75d98930 con 0x555a75eb1e60 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fa2ffd640 10 client.17214 handle_command_reply: tid=0 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 client.17214 shutdown 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 msgr2=0x555a75e9ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75e9ea00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009900 tx=0x7f1fb8005250 comp rx=0 tx=0).stop 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 shutdown_connections 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75eb1e60 0x555a75eb42e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75e9ea00 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75e9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fa2ffd640 0 client.17214 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 --2- 172.21.15.5:0/618074135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94250 0x555a75ea1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 >> 172.21.15.5:0/618074135 conn(0x555a75e8eba0 msgr2=0x555a75e908f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 shutdown_connections 2024-04-22T13:31:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 1 -- 172.21.15.5:0/618074135 wait complete. 2024-04-22T13:31:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.855+0000 7f1fcc874740 20 client.17214 trim_cache size 0 max 16384 2024-04-22T13:31:03.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.868+0000 7f1fcc874740 1 Processor -- start 2024-04-22T13:31:03.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.868+0000 7f1fcc874740 1 -- start start 2024-04-22T13:31:03.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.868+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75eadc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fcc874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41ab960 con 0x555a75e94c00 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fcc874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41ad4a0 con 0x555a75ea1e10 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fcc874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41ac030 con 0x555a75e94250 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42436/0 (socket says 172.21.15.5:42436) 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/2922807381 learned_addr learned my addr 172.21.15.5:0/2922807381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75eadc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/2922807381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 msgr2=0x555a75eadc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75eadc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/2922807381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 msgr2=0x555a75ea1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 -- 172.21.15.5:0/2922807381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a75e93560 con 0x555a75e94c00 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1130 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009850 tx=0x7f1fb8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8fc443dd015de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fa97fa640 1 -- 172.21.15.5:0/2922807381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb804d070 con 0x555a75e94c00 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fa97fa640 1 -- 172.21.15.5:0/2922807381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb80055d0 con 0x555a75e94c00 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.869+0000 7f1fa97fa640 1 -- 172.21.15.5:0/2922807381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8044070 con 0x555a75e94c00 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 -- 172.21.15.5:0/2922807381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 msgr2=0x555a75ea1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1130 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009850 tx=0x7f1fb8005d50 comp rx=0 tx=0).stop 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 -- 172.21.15.5:0/2922807381 shutdown_connections 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94c00 0x555a75ea1130 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75ea1e10 0x555a75ea1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 --2- 172.21.15.5:0/2922807381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75e94250 0x555a75eadc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 -- 172.21.15.5:0/2922807381 >> 172.21.15.5:0/2922807381 conn(0x555a75e8eba0 msgr2=0x555a75e9a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 -- 172.21.15.5:0/2922807381 shutdown_connections 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 -- 172.21.15.5:0/2922807381 wait complete. 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 -- start start 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.870+0000 7f1fcc874740 1 Processor -- start 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 0x555a75eaa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94c00 0x555a75ea58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75ea5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41acdd0 con 0x555a75e94250 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41ac700 con 0x555a75e94c00 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41a9940 con 0x555a75ea1e10 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 10 client.0 _ll_register_callbacks cb 0x7f1fb8004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:03.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 0x555a75eaa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 0x555a75eaa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42446/0 (socket says 172.21.15.5:42446) 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 -- 172.21.15.5:0/1723018668 learned_addr learned my addr 172.21.15.5:0/1723018668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75ea5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94c00 0x555a75ea58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 -- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 msgr2=0x555a75ea5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75ea5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 -- 172.21.15.5:0/1723018668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94c00 msgr2=0x555a75ea58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94c00 0x555a75ea58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 -- 172.21.15.5:0/1723018668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a75ea66a0 con 0x555a75e94250 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94c00 0x555a75ea58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 0x555a75eaa580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40045f0 tx=0x7f1fb40d01c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33a2154addf38ab server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/1723018668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b5060 con 0x555a75e94250 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/1723018668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1fb4009d20 con 0x555a75e94250 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb80055d0 con 0x555a75e94250 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/1723018668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fb409bb60 con 0x555a75e94250 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 10 client.17220 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.871+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a75f18520 con 0x555a75e94250 2024-04-22T13:31:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/1723018668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb80055d0 con 0x555a75e94250 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/1723018668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fb40cdc30 con 0x555a75e94250 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fcc874740 10 client.17220 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fcc874740 10 client.17220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fcc874740 20 client.17220 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fcc874740 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75f188b0 0x555a75f1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fcc874740 4 client.17220 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a75e06c10 con 0x555a75f188b0 2024-04-22T13:31:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75f188b0 0x555a75f1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fc2ffd640 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75f188b0 0x555a75f1aca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66e0f174d8961af2 server_cookie=9456ca99cac5c238 in_seq=0 out_seq=0 2024-04-22T13:31:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.872+0000 7f1fa2ffd640 10 client.17220 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fa2ffd640 1 -- 172.21.15.5:0/1723018668 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a75e06c10 con 0x555a75f188b0 2024-04-22T13:31:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fa2ffd640 10 client.17220 handle_command_reply: tid=0 2024-04-22T13:31:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 client.17220 shutdown 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 msgr2=0x555a75eaa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 0x555a75eaa580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40045f0 tx=0x7f1fb40d01c0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 shutdown_connections 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555a75f188b0 0x555a75f1aca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a75ea1e10 0x555a75ea5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555a75e94c00 0x555a75ea58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fa2ffd640 0 client.17220 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 --2- 172.21.15.5:0/1723018668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a75e94250 0x555a75eaa580 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 >> 172.21.15.5:0/1723018668 conn(0x555a75e8eba0 msgr2=0x555a75e8d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 shutdown_connections 2024-04-22T13:31:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 1 -- 172.21.15.5:0/1723018668 wait complete. 2024-04-22T13:31:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fcc874740 20 client.17220 trim_cache size 0 max 16384 2024-04-22T13:31:03.881 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1fac0a3d20 msgr2=0x7f1fac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1fac0a3d20 0x7f1fac0a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0092f0 tx=0x7f1fbc004a40 comp rx=0 tx=0).stop 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 msgr2=0x7f1fc4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 0x7f1fc4198750 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400d950 tx=0x7f1fb400de10 comp rx=0 tx=0).stop 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.873+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 shutdown_connections 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1fac0a3d20 0x7f1fac0a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fc410c500 0x7f1fc419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc4103ae0 0x7f1fc4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 --2- 172.21.15.5:0/3317903907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41031c0 0x7f1fc4198750 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 >> 172.21.15.5:0/3317903907 conn(0x7f1fc4078000 msgr2=0x7f1fc40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 shutdown_connections 2024-04-22T13:31:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.874+0000 7f1fc9c81640 1 -- 172.21.15.5:0/3317903907 wait complete. 2024-04-22T13:31:03.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 -- 172.21.15.5:0/1812485675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c041031c0 msgr2=0x7f8c041035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 --2- 172.21.15.5:0/1812485675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c041031c0 0x7f8c041035a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009900 tx=0x7f8bf402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 -- 172.21.15.5:0/1812485675 shutdown_connections 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 --2- 172.21.15.5:0/1812485675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0410c6d0 0x7f8c0410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 --2- 172.21.15.5:0/1812485675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c04103b70 0x7f8c0410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 --2- 172.21.15.5:0/1812485675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c041031c0 0x7f8c041035a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 -- 172.21.15.5:0/1812485675 >> 172.21.15.5:0/1812485675 conn(0x7f8c04077f70 msgr2=0x7f8c04078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 -- 172.21.15.5:0/1812485675 shutdown_connections 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 -- 172.21.15.5:0/1812485675 wait complete. 2024-04-22T13:31:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.993+0000 7f8c0a175640 1 Processor -- start 2024-04-22T13:31:03.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 -- start start 2024-04-22T13:31:03.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c041031c0 0x7f8c0419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 0x7f8c0419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c041988c0 0x7f8c04198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04112390 con 0x7f8c0410c6d0 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04112210 con 0x7f8c041031c0 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c0a175640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04112510 con 0x7f8c041988c0 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 0x7f8c0419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 0x7f8c0419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42460/0 (socket says 172.21.15.5:42460) 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 -- 172.21.15.5:0/4029117545 learned_addr learned my addr 172.21.15.5:0/4029117545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c03fff640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c041988c0 0x7f8c04198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c037fe640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c041031c0 0x7f8c0419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 -- 172.21.15.5:0/4029117545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c041988c0 msgr2=0x7f8c04198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c041988c0 0x7f8c04198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 -- 172.21.15.5:0/4029117545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c041031c0 msgr2=0x7f8c0419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c041031c0 0x7f8c0419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.994+0000 7f8c02ffd640 1 -- 172.21.15.5:0/4029117545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041993e0 con 0x7f8c0410c6d0 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.995+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 0x7f8c0419ec00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d750 tx=0x7f8bf800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.995+0000 7f8c00ff9640 1 -- 172.21.15.5:0/4029117545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x7f8c0410c6d0 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.995+0000 7f8c00ff9640 1 -- 172.21.15.5:0/4029117545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf80054d0 con 0x7f8c0410c6d0 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.995+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c041996d0 con 0x7f8c0410c6d0 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.995+0000 7f8c00ff9640 1 -- 172.21.15.5:0/4029117545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8004c30 con 0x7f8c0410c6d0 2024-04-22T13:31:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.995+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041a5880 con 0x7f8c0410c6d0 2024-04-22T13:31:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c00ff9640 1 -- 172.21.15.5:0/4029117545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8bf8004dd0 con 0x7f8c0410c6d0 2024-04-22T13:31:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 Processor -- start 2024-04-22T13:31:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 -- start start 2024-04-22T13:31:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b260e871f0 con 0x55b260f224d0 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b260e244b0 con 0x55b260f22e80 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b260e831b0 con 0x55b260f2ba00 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.997+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42476/0 (socket says 172.21.15.5:42476) 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 -- 172.21.15.5:0/1807120877 learned_addr learned my addr 172.21.15.5:0/1807120877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c00ff9640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bec0a3d20 0x7f8bec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c037fe640 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bec0a3d20 0x7f8bec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c00ff9640 1 -- 172.21.15.5:0/4029117545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bf80d01e0 con 0x7f8c0410c6d0 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 -- 172.21.15.5:0/1807120877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 msgr2=0x55b260f2de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 -- 172.21.15.5:0/1807120877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 msgr2=0x55b260f2b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 -- 172.21.15.5:0/1807120877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b260e161b0 con 0x55b260f224d0 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f228b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc014a60 tx=0x7f8bfc014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cff7bf47e80289 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8be9ffb640 1 -- 172.21.15.5:0/1807120877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc01ec50 con 0x55b260f224d0 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8be9ffb640 1 -- 172.21.15.5:0/1807120877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bfc023070 con 0x55b260f224d0 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8c03fff640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bec0a3d20 0x7f8bec0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc022000 tx=0x7f8bfc01e000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.998+0000 7f8be9ffb640 1 -- 172.21.15.5:0/1807120877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc005540 con 0x55b260f224d0 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1807120877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 msgr2=0x55b260f228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f228b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc014a60 tx=0x7f8bfc014f20 comp rx=0 tx=0).stop 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1807120877 shutdown_connections 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1807120877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f228b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1807120877 >> 172.21.15.5:0/1807120877 conn(0x55b260f1cba0 msgr2=0x55b260f1cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1807120877 shutdown_connections 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1807120877 wait complete. 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:03.999+0000 7f8c0cd67740 1 -- start start 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 Processor -- start 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f28700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b260f2e970 con 0x55b260f224d0 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b260e40500 con 0x55b260f22e80 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b260f2eaf0 con 0x55b260f2ba00 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c0cd67740 10 client.0 _ll_register_callbacks cb 0x55b260e0eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f2b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42492/0 (socket says 172.21.15.5:42492) 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 -- 172.21.15.5:0/4294420480 learned_addr learned my addr 172.21.15.5:0/4294420480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f28700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c03fff640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 -- 172.21.15.5:0/4294420480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 msgr2=0x55b260f28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 -- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 msgr2=0x55b260f28700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f28700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 -- 172.21.15.5:0/4294420480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b260b993f0 con 0x55b260f224d0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c03fff640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8c037fe640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f2b370 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005940 tx=0x7f8bf402c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a67f0ad323b51e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.000+0000 7f8be2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8be2ffd640 1 -- 172.21.15.5:0/4294420480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4005230 con 0x55b260f224d0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8be2ffd640 1 -- 172.21.15.5:0/4294420480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bf40053d0 con 0x55b260f224d0 2024-04-22T13:31:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bfc023070 con 0x55b260f224d0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8be2ffd640 1 -- 172.21.15.5:0/4294420480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bf40055d0 con 0x55b260f224d0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 10 client.17232 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b260c637d0 con 0x55b260f224d0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8be2ffd640 1 -- 172.21.15.5:0/4294420480 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bfc023070 con 0x55b260f224d0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8be2ffd640 1 -- 172.21.15.5:0/4294420480 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bf40037f0 con 0x55b260f224d0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 10 client.17232 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 10 client.17232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 20 client.17232 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260f3bbf0 0x55b260f3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 4 client.17232 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.001+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b260e26930 con 0x55b260f3bbf0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.002+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260f3bbf0 0x55b260f3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.002+0000 7f8be2ffd640 10 client.17232 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.002+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260f3bbf0 0x55b260f3e020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c37536ed7b24675 server_cookie=c6fb6ba6acc6ae39 in_seq=0 out_seq=0 2024-04-22T13:31:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8be2ffd640 1 -- 172.21.15.5:0/4294420480 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b260e26930 con 0x55b260f3bbf0 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8be2ffd640 10 client.17232 handle_command_reply: tid=0 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 client.17232 shutdown 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 msgr2=0x55b260f2b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f2b370 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005940 tx=0x7f8bf402c9f0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 shutdown_connections 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260f3bbf0 0x55b260f3e020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f28700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8be2ffd640 0 client.17232 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/4294420480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f2b370 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 >> 172.21.15.5:0/4294420480 conn(0x55b260f1cba0 msgr2=0x55b260f1e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 shutdown_connections 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 1 -- 172.21.15.5:0/4294420480 wait complete. 2024-04-22T13:31:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.003+0000 7f8c0cd67740 20 client.17232 trim_cache size 0 max 16384 2024-04-22T13:31:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.016+0000 7f8c0cd67740 1 Processor -- start 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 -- start start 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041adb60 con 0x55b260f224d0 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041af6a0 con 0x55b260f22e80 2024-04-22T13:31:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041ae230 con 0x55b260f2ba00 2024-04-22T13:31:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f37830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42498/0 (socket says 172.21.15.5:42498) 2024-04-22T13:31:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 -- 172.21.15.5:0/2477560062 learned_addr learned my addr 172.21.15.5:0/2477560062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 -- 172.21.15.5:0/2477560062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 msgr2=0x55b260f2b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c03fff640 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 -- 172.21.15.5:0/2477560062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 msgr2=0x55b260f2ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 -- 172.21.15.5:0/2477560062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b260f29080 con 0x55b260f224d0 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c03fff640 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8c037fe640 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f37830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8bf402c170 tx=0x7f8bf402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2aeee098a6365e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8be97fa640 1 -- 172.21.15.5:0/2477560062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4000e60 con 0x55b260f224d0 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.017+0000 7f8be97fa640 1 -- 172.21.15.5:0/2477560062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf402c5d0 con 0x55b260f224d0 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8be97fa640 1 -- 172.21.15.5:0/2477560062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4052ce0 con 0x55b260f224d0 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 -- 172.21.15.5:0/2477560062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 msgr2=0x55b260f37830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f37830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8bf402c170 tx=0x7f8bf402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 -- 172.21.15.5:0/2477560062 shutdown_connections 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f2ba00 0x55b260f2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/2477560062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f224d0 0x55b260f37830 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 -- 172.21.15.5:0/2477560062 >> 172.21.15.5:0/2477560062 conn(0x55b260f1cba0 msgr2=0x55b260f1bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 -- 172.21.15.5:0/2477560062 shutdown_connections 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 -- 172.21.15.5:0/2477560062 wait complete. 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.018+0000 7f8c0cd67740 1 -- start start 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 Processor -- start 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f224d0 0x55b260f2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 0x55b260fa1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041aefd0 con 0x55b260f2ba00 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041ae900 con 0x55b260f22e80 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041abb60 con 0x55b260f224d0 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c0cd67740 10 client.0 _ll_register_callbacks cb 0x7f8bf40043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 0x55b260fa1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 0x55b260fa1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42508/0 (socket says 172.21.15.5:42508) 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 -- 172.21.15.5:0/1799145984 learned_addr learned my addr 172.21.15.5:0/1799145984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c037fe640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f224d0 0x55b260f2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 -- 172.21.15.5:0/1799145984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f224d0 msgr2=0x55b260f2fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f224d0 0x55b260f2fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 -- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 msgr2=0x55b260f302d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f302d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c03fff640 1 -- 172.21.15.5:0/1799145984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b260fa16a0 con 0x55b260f2ba00 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.019+0000 7f8c02ffd640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c03fff640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 0x55b260fa1160 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009d10 tx=0x7f8bfc009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31959c4ce43b4b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8be2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8be2ffd640 1 -- 172.21.15.5:0/1799145984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc012c70 con 0x55b260f2ba00 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf402c5d0 con 0x55b260f2ba00 2024-04-22T13:31:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8be2ffd640 1 -- 172.21.15.5:0/1799145984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8bfc012e10 con 0x55b260f2ba00 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8be2ffd640 1 -- 172.21.15.5:0/1799145984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc040050 con 0x55b260f2ba00 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 10 client.17238 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b260fa2810 con 0x55b260f2ba00 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8be2ffd640 1 -- 172.21.15.5:0/1799145984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf402c5d0 con 0x55b260f2ba00 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8be2ffd640 1 -- 172.21.15.5:0/1799145984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bfc00f040 con 0x55b260f2ba00 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 10 client.17238 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 20 client.17238 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260fa2c40 0x55b260fa5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 4 client.17238 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.020+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b260e94c10 con 0x55b260fa2c40 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c037fe640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260fa2c40 0x55b260fa5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c037fe640 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260fa2c40 0x55b260fa5060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f95f296b47b7cd81 server_cookie=b5681742b400c9a3 in_seq=0 out_seq=0 2024-04-22T13:31:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8be2ffd640 10 client.17238 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8be2ffd640 1 -- 172.21.15.5:0/1799145984 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b260e94c10 con 0x55b260fa2c40 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8be2ffd640 10 client.17238 handle_command_reply: tid=0 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 client.17238 shutdown 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 msgr2=0x55b260fa1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 0x55b260fa1160 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009d10 tx=0x7f8bfc009a00 comp rx=0 tx=0).stop 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 shutdown_connections 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55b260fa2c40 0x55b260fa5060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b260f2ba00 0x55b260fa1160 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b260f22e80 0x55b260f302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 --2- 172.21.15.5:0/1799145984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b260f224d0 0x55b260f2fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8be2ffd640 0 client.17238 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 >> 172.21.15.5:0/1799145984 conn(0x55b260f1cba0 msgr2=0x55b260f1df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 shutdown_connections 2024-04-22T13:31:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.021+0000 7f8c0cd67740 1 -- 172.21.15.5:0/1799145984 wait complete. 2024-04-22T13:31:04.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0cd67740 20 client.17238 trim_cache size 0 max 16384 2024-04-22T13:31:04.030 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bec0a3d20 msgr2=0x7f8bec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bec0a3d20 0x7f8bec0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc022000 tx=0x7f8bfc01e000 comp rx=0 tx=0).stop 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 msgr2=0x7f8c0419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 0x7f8c0419ec00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d750 tx=0x7f8bf800dc10 comp rx=0 tx=0).stop 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 shutdown_connections 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0410c6d0 0x7f8c0419ec00 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8bec0a3d20 0x7f8bec0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c041988c0 0x7f8c04198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 --2- 172.21.15.5:0/4029117545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c041031c0 0x7f8c0419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 >> 172.21.15.5:0/4029117545 conn(0x7f8c04077f70 msgr2=0x7f8c040ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 shutdown_connections 2024-04-22T13:31:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.022+0000 7f8c0a175640 1 -- 172.21.15.5:0/4029117545 wait complete. 2024-04-22T13:31:04.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:03 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3239716737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:04.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:03 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3239716737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:04.034 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:03 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3239716737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 -- 172.21.15.5:0/3203030005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 msgr2=0x7f73a8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 --2- 172.21.15.5:0/3203030005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a8101470 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7398009900 tx=0x7f739802bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 -- 172.21.15.5:0/3203030005 shutdown_connections 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 --2- 172.21.15.5:0/3203030005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 0x7f73a810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 --2- 172.21.15.5:0/3203030005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73a8101a40 0x7f73a8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 --2- 172.21.15.5:0/3203030005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a8101470 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 -- 172.21.15.5:0/3203030005 >> 172.21.15.5:0/3203030005 conn(0x7f73a8078000 msgr2=0x7f73a8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 -- 172.21.15.5:0/3203030005 shutdown_connections 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 -- 172.21.15.5:0/3203030005 wait complete. 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 Processor -- start 2024-04-22T13:31:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.142+0000 7f73b0784640 1 -- start start 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 0x7f73a819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73a8198590 0x7f73a8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81110c0 con 0x7f73a8101090 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8110f40 con 0x7f73a8106160 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8111240 con 0x7f73a8198590 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42526/0 (socket says 172.21.15.5:42526) 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 -- 172.21.15.5:0/2898908320 learned_addr learned my addr 172.21.15.5:0/2898908320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73aed21640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73a8198590 0x7f73a8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73add1f640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 0x7f73a819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 -- 172.21.15.5:0/2898908320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73a8198590 msgr2=0x7f73a8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73a8198590 0x7f73a8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 -- 172.21.15.5:0/2898908320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 msgr2=0x7f73a819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 0x7f73a819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 -- 172.21.15.5:0/2898908320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a81990b0 con 0x7f73a8101090 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73add1f640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 0x7f73a819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73ae520640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a819e3e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7398005940 tx=0x7f739802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f739f7fe640 1 -- 172.21.15.5:0/2898908320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398043020 con 0x7f73a8101090 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a8199340 con 0x7f73a8101090 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f739f7fe640 1 -- 172.21.15.5:0/2898908320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739802ca70 con 0x7f73a8101090 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.143+0000 7f739f7fe640 1 -- 172.21.15.5:0/2898908320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980039f0 con 0x7f73a8101090 2024-04-22T13:31:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.144+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a5590 con 0x7f73a8101090 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 Processor -- start 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 -- start start 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f739f7fe640 1 -- 172.21.15.5:0/2898908320 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7398046020 con 0x7f73a8101090 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1c988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1ca1a00 0x55adf1ca3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adf1bfd1f0 con 0x55adf1c98e80 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55adf1b9a4b0 con 0x55adf1c984d0 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73b3379740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55adf1bf91b0 con 0x55adf1ca1a00 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73ae520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73ae520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42530/0 (socket says 172.21.15.5:42530) 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.146+0000 7f73ae520640 1 -- 172.21.15.5:0/3798491351 learned_addr learned my addr 172.21.15.5:0/3798491351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73aed21640 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1c988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73add1f640 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1ca1a00 0x55adf1ca3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73ae520640 1 -- 172.21.15.5:0/3798491351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1ca1a00 msgr2=0x55adf1ca3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73ae520640 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1ca1a00 0x55adf1ca3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73ae520640 1 -- 172.21.15.5:0/3798491351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 msgr2=0x55adf1c988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73ae520640 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1c988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73ae520640 1 -- 172.21.15.5:0/3798491351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf1b8c1b0 con 0x55adf1c98e80 2024-04-22T13:31:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f739f7fe640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f73940a3d20 0x7f73940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73aed21640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f73940a3d20 0x7f73940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f73ae520640 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca1430 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f73980effe0 tx=0x7f73980f8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e0b50cb19a7299 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f739f7fe640 1 -- 172.21.15.5:0/2898908320 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f73980f27e0 con 0x7f73a8101090 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f739cff9640 1 -- 172.21.15.5:0/3798491351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980be690 con 0x55adf1c98e80 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f739cff9640 1 -- 172.21.15.5:0/3798491351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73980befc0 con 0x55adf1c98e80 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.147+0000 7f739cff9640 1 -- 172.21.15.5:0/3798491351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980be220 con 0x55adf1c98e80 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 -- 172.21.15.5:0/3798491351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 msgr2=0x55adf1ca1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73aed21640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f73940a3d20 0x7f73940a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a0009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca1430 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f73980effe0 tx=0x7f73980f8b40 comp rx=0 tx=0).stop 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 -- 172.21.15.5:0/3798491351 shutdown_connections 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca1430 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1ca1a00 0x55adf1ca3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 --2- 172.21.15.5:0/3798491351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1c988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 -- 172.21.15.5:0/3798491351 >> 172.21.15.5:0/3798491351 conn(0x55adf1c92ba0 msgr2=0x55adf1c92f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 -- 172.21.15.5:0/3798491351 shutdown_connections 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.148+0000 7f73b3379740 1 -- 172.21.15.5:0/3798491351 wait complete. 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 -- start start 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 Processor -- start 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c98e80 0x55adf1c9e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1c9ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adf1ca4730 con 0x55adf1ca1a00 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55adf1bb6500 con 0x55adf1c98e80 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55adf1ca48b0 con 0x55adf1c984d0 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1c9ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73b3379740 10 client.0 _ll_register_callbacks cb 0x55adf1b84b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1c9ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42540/0 (socket says 172.21.15.5:42540) 2024-04-22T13:31:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 -- 172.21.15.5:0/3374445917 learned_addr learned my addr 172.21.15.5:0/3374445917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73ae520640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73add1f640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c98e80 0x55adf1c9e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 -- 172.21.15.5:0/3374445917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 msgr2=0x55adf1ca1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 -- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c98e80 msgr2=0x55adf1c9e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c98e80 0x55adf1c9e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73aed21640 1 -- 172.21.15.5:0/3374445917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf190f3f0 con 0x55adf1ca1a00 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.149+0000 7f73add1f640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c98e80 0x55adf1c9e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73aed21640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1c9ec10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009910 tx=0x7f73a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108b81d62bc45c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f7391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f7391ffb640 1 -- 172.21.15.5:0/3374445917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004b20 con 0x55adf1ca1a00 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f7391ffb640 1 -- 172.21.15.5:0/3374445917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f73a0004cc0 con 0x55adf1ca1a00 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73980befc0 con 0x55adf1ca1a00 2024-04-22T13:31:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f7391ffb640 1 -- 172.21.15.5:0/3374445917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73a000f830 con 0x55adf1ca1a00 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 10 client.17250 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55adf19d97d0 con 0x55adf1ca1a00 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f7391ffb640 1 -- 172.21.15.5:0/3374445917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73980befc0 con 0x55adf1ca1a00 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f7391ffb640 1 -- 172.21.15.5:0/3374445917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73a0026070 con 0x55adf1ca1a00 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 10 client.17250 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 20 client.17250 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1cb1a20 0x55adf1cb3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 4 client.17250 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.150+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adf1b9c930 con 0x55adf1cb1a20 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.151+0000 7f73ae520640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1cb1a20 0x55adf1cb3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.151+0000 7f7391ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.151+0000 7f73ae520640 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1cb1a20 0x55adf1cb3e80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d2fd2a2e89db24d server_cookie=b3c91d0f62041a96 in_seq=0 out_seq=0 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f7391ffb640 1 -- 172.21.15.5:0/3374445917 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adf1b9c930 con 0x55adf1cb1a20 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f7391ffb640 10 client.17250 handle_command_reply: tid=0 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 client.17250 shutdown 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 msgr2=0x55adf1c9ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1c9ec10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009910 tx=0x7f73a0004a40 comp rx=0 tx=0).stop 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 shutdown_connections 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1cb1a20 0x55adf1cb3e80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1c9ec10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c98e80 0x55adf1c9e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f7391ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 --2- 172.21.15.5:0/3374445917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 >> 172.21.15.5:0/3374445917 conn(0x55adf1c92ba0 msgr2=0x55adf1c948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 shutdown_connections 2024-04-22T13:31:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 1 -- 172.21.15.5:0/3374445917 wait complete. 2024-04-22T13:31:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.152+0000 7f73b3379740 20 client.17250 trim_cache size 0 max 16384 2024-04-22T13:31:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.165+0000 7f73b3379740 1 Processor -- start 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.165+0000 7f73b3379740 1 -- start start 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73b3379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1cad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73b3379740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c98e80 0x55adf1c91c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73b3379740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1ca0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73b3379740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81adbb0 con 0x55adf1ca1a00 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73b3379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81af6f0 con 0x55adf1c984d0 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73b3379740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ae280 con 0x55adf1c98e80 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1ca0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1ca0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42548/0 (socket says 172.21.15.5:42548) 2024-04-22T13:31:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 -- 172.21.15.5:0/643508027 learned_addr learned my addr 172.21.15.5:0/643508027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73ae520640 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1cad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73add1f640 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c98e80 0x55adf1c91c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 -- 172.21.15.5:0/643508027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c98e80 msgr2=0x55adf1c91c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c98e80 0x55adf1c91c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 -- 172.21.15.5:0/643508027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 msgr2=0x55adf1cad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1cad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 -- 172.21.15.5:0/643508027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf1c9d640 con 0x55adf1ca1a00 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73add1f640 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c98e80 0x55adf1c91c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f73aed21640 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1ca0dc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f73a00098e0 tx=0x7f73a0011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d117336950073ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f7393fff640 1 -- 172.21.15.5:0/643508027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0034030 con 0x55adf1ca1a00 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f7393fff640 1 -- 172.21.15.5:0/643508027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a0004ab0 con 0x55adf1ca1a00 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.166+0000 7f7393fff640 1 -- 172.21.15.5:0/643508027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0005260 con 0x55adf1ca1a00 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 -- 172.21.15.5:0/643508027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 msgr2=0x55adf1ca0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1ca0dc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f73a00098e0 tx=0x7f73a0011f90 comp rx=0 tx=0).stop 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 -- 172.21.15.5:0/643508027 shutdown_connections 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1ca1a00 0x55adf1ca0dc0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c98e80 0x55adf1c91c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 --2- 172.21.15.5:0/643508027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1c984d0 0x55adf1cad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 -- 172.21.15.5:0/643508027 >> 172.21.15.5:0/643508027 conn(0x55adf1c92ba0 msgr2=0x55adf1c92f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 -- 172.21.15.5:0/643508027 shutdown_connections 2024-04-22T13:31:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 -- 172.21.15.5:0/643508027 wait complete. 2024-04-22T13:31:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 -- start start 2024-04-22T13:31:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.167+0000 7f73b3379740 1 Processor -- start 2024-04-22T13:31:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1ca1a00 0x55adf1d10f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81af020 con 0x55adf1c98e80 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ae950 con 0x55adf1ca1a00 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81abb90 con 0x55adf1c984d0 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 10 client.0 _ll_register_callbacks cb 0x7f73a0003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42556/0 (socket says 172.21.15.5:42556) 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 -- 172.21.15.5:0/3437567654 learned_addr learned my addr 172.21.15.5:0/3437567654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73ae520640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73aed21640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1ca1a00 0x55adf1d10f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 -- 172.21.15.5:0/3437567654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 msgr2=0x55adf1ca5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 -- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1ca1a00 msgr2=0x55adf1d10f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1ca1a00 0x55adf1d10f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 -- 172.21.15.5:0/3437567654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf1d115d0 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73ae520640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73aed21640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1ca1a00 0x55adf1d10f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73add1f640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca6160 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f73a40095c0 tx=0x7f73a4013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beafedd0a58f8234 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f7391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f7391ffb640 1 -- 172.21.15.5:0/3437567654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ea70 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a0004ab0 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.168+0000 7f7391ffb640 1 -- 172.21.15.5:0/3437567654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f73a400ec10 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f7391ffb640 1 -- 172.21.15.5:0/3437567654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73a400ee10 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 10 client.17256 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55adf1d18940 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f7391ffb640 1 -- 172.21.15.5:0/3437567654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a0004ab0 con 0x55adf1c98e80 2024-04-22T13:31:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f7391ffb640 1 -- 172.21.15.5:0/3437567654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73a400f040 con 0x55adf1c98e80 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 10 client.17256 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 20 client.17256 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1d19140 0x55adf1d1b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 4 client.17256 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adf1c0ac10 con 0x55adf1d19140 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.169+0000 7f73ae520640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1d19140 0x55adf1d1b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73ae520640 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1d19140 0x55adf1d1b560 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5702867e718efbea server_cookie=e5cc119eda04f12a in_seq=0 out_seq=0 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f7391ffb640 10 client.17256 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f7391ffb640 1 -- 172.21.15.5:0/3437567654 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adf1c0ac10 con 0x55adf1d19140 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f7391ffb640 10 client.17256 handle_command_reply: tid=0 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 client.17256 shutdown 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 msgr2=0x55adf1ca6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca6160 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f73a40095c0 tx=0x7f73a4013b10 comp rx=0 tx=0).stop 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 shutdown_connections 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55adf1d19140 0x55adf1d1b560 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adf1c98e80 0x55adf1ca6160 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55adf1ca1a00 0x55adf1d10f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f7391ffb640 0 client.17256 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 --2- 172.21.15.5:0/3437567654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55adf1c984d0 0x55adf1ca5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 >> 172.21.15.5:0/3437567654 conn(0x55adf1c92ba0 msgr2=0x55adf1c99af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 shutdown_connections 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 1 -- 172.21.15.5:0/3437567654 wait complete. 2024-04-22T13:31:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.170+0000 7f73b3379740 20 client.17256 trim_cache size 0 max 16384 2024-04-22T13:31:04.180 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f73940a3d20 msgr2=0x7f73940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f73940a3d20 0x7f73940a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a0009500 comp rx=0 tx=0).stop 2024-04-22T13:31:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 msgr2=0x7f73a819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a819e3e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7398005940 tx=0x7f739802c5f0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 shutdown_connections 2024-04-22T13:31:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f73940a3d20 0x7f73940a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73a8198590 0x7f73a8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73a8106160 0x7f73a819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 --2- 172.21.15.5:0/2898908320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73a8101090 0x7f73a819e3e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 >> 172.21.15.5:0/2898908320 conn(0x7f73a8078000 msgr2=0x7f73a80696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 shutdown_connections 2024-04-22T13:31:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:04.171+0000 7f73b0784640 1 -- 172.21.15.5:0/2898908320 wait complete. 2024-04-22T13:31:05.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:04 smithi049 ceph-mon[31379]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:05.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:04 smithi107 ceph-mon[31859]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:04 smithi005 ceph-mon[28006]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:05.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:31:05.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:31:05.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:31:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 -- 172.21.15.5:0/392997615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348103400 msgr2=0x7f534810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 --2- 172.21.15.5:0/392997615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348103400 0x7f534810de00 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f5340009cd0 tx=0x7f534002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 -- 172.21.15.5:0/392997615 shutdown_connections 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 --2- 172.21.15.5:0/392997615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348103400 0x7f534810de00 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 --2- 172.21.15.5:0/392997615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 0x7f5348102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 --2- 172.21.15.5:0/392997615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5348101180 0x7f5348101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 -- 172.21.15.5:0/392997615 >> 172.21.15.5:0/392997615 conn(0x7f5348077f70 msgr2=0x7f5348078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 -- 172.21.15.5:0/392997615 shutdown_connections 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.306+0000 7f53505db640 1 -- 172.21.15.5:0/392997615 wait complete. 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 Processor -- start 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 -- start start 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 0x7f534819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 0x7f534819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5348103400 0x7f53481985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53481111a0 con 0x7f5348101180 2024-04-22T13:31:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5348111020 con 0x7f5348103400 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f53505db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5348111320 con 0x7f5348102a60 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 0x7f534819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 0x7f534819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42584/0 (socket says 172.21.15.5:42584) 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.307+0000 7f534e377640 1 -- 172.21.15.5:0/2131901462 learned_addr learned my addr 172.21.15.5:0/2131901462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534eb78640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5348103400 0x7f53481985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534db76640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 0x7f534819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534e377640 1 -- 172.21.15.5:0/2131901462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 msgr2=0x7f534819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534e377640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 0x7f534819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534e377640 1 -- 172.21.15.5:0/2131901462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5348103400 msgr2=0x7f53481985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534e377640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5348103400 0x7f53481985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534e377640 1 -- 172.21.15.5:0/2131901462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5348198d20 con 0x7f5348101180 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534db76640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 0x7f534819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f534e377640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 0x7f534819e3f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f534400d950 tx=0x7f534400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f533f7fe640 1 -- 172.21.15.5:0/2131901462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f5348101180 2024-04-22T13:31:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5348199010 con 0x7f5348101180 2024-04-22T13:31:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f533f7fe640 1 -- 172.21.15.5:0/2131901462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344009c30 con 0x7f5348101180 2024-04-22T13:31:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f533f7fe640 1 -- 172.21.15.5:0/2131901462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400f620 con 0x7f5348101180 2024-04-22T13:31:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.308+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53481a52b0 con 0x7f5348101180 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.311+0000 7f53531cf740 1 Processor -- start 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.311+0000 7f533f7fe640 1 -- 172.21.15.5:0/2131901462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5344010460 con 0x7f5348101180 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 -- start start 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b95cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8ba1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616c8af9f00 con 0x5616c8b96290 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5616c8a97590 con 0x5616c8b958e0 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f53531cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5616c8af5ec0 con 0x5616c8b9ee10 2024-04-22T13:31:05.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42600/0 (socket says 172.21.15.5:42600) 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f534e377640 1 -- 172.21.15.5:0/126304658 learned_addr learned my addr 172.21.15.5:0/126304658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f534db76640 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8ba1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f533f7fe640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f53340a3d20 0x7f53340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.312+0000 7f534eb78640 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b95cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534db76640 1 -- 172.21.15.5:0/126304658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 msgr2=0x5616c8b95cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534db76640 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b95cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534eb78640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f53340a3d20 0x7f53340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534db76640 1 -- 172.21.15.5:0/126304658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 msgr2=0x5616c8b9e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534db76640 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f533f7fe640 1 -- 172.21.15.5:0/2131901462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f53440d0430 con 0x7f5348101180 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534db76640 1 -- 172.21.15.5:0/126304658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616c88d47e0 con 0x5616c8b9ee10 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534e377640 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534db76640 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8ba1260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f53481996e0 tx=0x7f53380117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6bedc2f7ddbc377 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f534eb78640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f53340a3d20 0x7f53340a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5340005cc0 tx=0x7f5340005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f533cbfe640 1 -- 172.21.15.5:0/126304658 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338005020 con 0x5616c8b9ee10 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f533cbfe640 1 -- 172.21.15.5:0/126304658 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53380047a0 con 0x5616c8b9ee10 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.313+0000 7f533cbfe640 1 -- 172.21.15.5:0/126304658 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53380126b0 con 0x5616c8b9ee10 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 -- 172.21.15.5:0/126304658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 msgr2=0x5616c8ba1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8ba1260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f53481996e0 tx=0x7f53380117b0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 -- 172.21.15.5:0/126304658 shutdown_connections 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8ba1260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 --2- 172.21.15.5:0/126304658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b95cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 -- 172.21.15.5:0/126304658 >> 172.21.15.5:0/126304658 conn(0x5616c8b8ffb0 msgr2=0x5616c8b90390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 -- 172.21.15.5:0/126304658 shutdown_connections 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 -- 172.21.15.5:0/126304658 wait complete. 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.314+0000 7f53531cf740 1 -- start start 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 Processor -- start 2024-04-22T13:31:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616c8ba1d80 con 0x5616c8b96290 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5616c8ab3580 con 0x5616c8b958e0 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5616c8ba1f00 con 0x5616c8b9ee10 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f53531cf740 10 client.0 _ll_register_callbacks cb 0x5616c8ab9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42606/0 (socket says 172.21.15.5:42606) 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 -- 172.21.15.5:0/101400233 learned_addr learned my addr 172.21.15.5:0/101400233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534eb78640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534e377640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 -- 172.21.15.5:0/101400233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 msgr2=0x5616c8b9c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 -- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 msgr2=0x5616c8b9bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b9bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 -- 172.21.15.5:0/101400233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616c8934f70 con 0x5616c8b96290 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534e377640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b9bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f534db76640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9c280 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5338004680 tx=0x7f53380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8071a3e547e5b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.315+0000 7f5331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f5331ffb640 1 -- 172.21.15.5:0/101400233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338028020 con 0x5616c8b96290 2024-04-22T13:31:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f5331ffb640 1 -- 172.21.15.5:0/101400233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5338005210 con 0x5616c8b96290 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53380047a0 con 0x5616c8b96290 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f5331ffb640 1 -- 172.21.15.5:0/101400233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5338033050 con 0x5616c8b96290 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 10 client.17268 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616c87cc170 con 0x5616c8b96290 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f5331ffb640 1 -- 172.21.15.5:0/101400233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53380047a0 con 0x5616c8b96290 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f5331ffb640 1 -- 172.21.15.5:0/101400233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5338012450 con 0x5616c8b96290 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 10 client.17268 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 20 client.17268 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8baf250 0x5616c8bb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 4 client.17268 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.316+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616c8b07920 con 0x5616c8baf250 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.317+0000 7f534e377640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8baf250 0x5616c8bb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.317+0000 7f5331ffb640 10 client.17268 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.317+0000 7f534e377640 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8baf250 0x5616c8bb1640 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3531fe3d758db024 server_cookie=efb3f149de806567 in_seq=0 out_seq=0 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f5331ffb640 1 -- 172.21.15.5:0/101400233 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616c8b07920 con 0x5616c8baf250 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f5331ffb640 10 client.17268 handle_command_reply: tid=0 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 client.17268 shutdown 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 msgr2=0x5616c8b9c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9c280 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5338004680 tx=0x7f53380046b0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 shutdown_connections 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8baf250 0x5616c8bb1640 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b96290 0x5616c8b9c280 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f5331ffb640 0 client.17268 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 --2- 172.21.15.5:0/101400233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b958e0 0x5616c8b9bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 >> 172.21.15.5:0/101400233 conn(0x5616c8b8ffb0 msgr2=0x5616c8b91d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 shutdown_connections 2024-04-22T13:31:05.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 1 -- 172.21.15.5:0/101400233 wait complete. 2024-04-22T13:31:05.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.318+0000 7f53531cf740 20 client.17268 trim_cache size 0 max 16384 2024-04-22T13:31:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 Processor -- start 2024-04-22T13:31:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 -- start start 2024-04-22T13:31:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b9e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8baaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53481ada30 con 0x5616c8b958e0 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53481af570 con 0x5616c8b96290 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f53531cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53481ae100 con 0x5616c8b9ee10 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b9e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b9e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42618/0 (socket says 172.21.15.5:42618) 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 -- 172.21.15.5:0/1988931803 learned_addr learned my addr 172.21.15.5:0/1988931803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534eb78640 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 -- 172.21.15.5:0/1988931803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 msgr2=0x5616c8b9c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 -- 172.21.15.5:0/1988931803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 msgr2=0x5616c8baaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8baaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.336+0000 7f534e377640 1 -- 172.21.15.5:0/1988931803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616c8bb19b0 con 0x5616c8b958e0 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f534e377640 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b9e470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f53440cdc60 tx=0x7f5344010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aca2b0392a2324 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f5333fff640 1 -- 172.21.15.5:0/1988931803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344020040 con 0x5616c8b958e0 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f5333fff640 1 -- 172.21.15.5:0/1988931803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344009de0 con 0x5616c8b958e0 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f5333fff640 1 -- 172.21.15.5:0/1988931803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53440cde30 con 0x5616c8b958e0 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 -- 172.21.15.5:0/1988931803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 msgr2=0x5616c8b9e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b9e470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f53440cdc60 tx=0x7f5344010cb0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 -- 172.21.15.5:0/1988931803 shutdown_connections 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8b9c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8baaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 --2- 172.21.15.5:0/1988931803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b9e470 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 -- 172.21.15.5:0/1988931803 >> 172.21.15.5:0/1988931803 conn(0x5616c8b8ffb0 msgr2=0x5616c8b9bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 -- 172.21.15.5:0/1988931803 shutdown_connections 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.337+0000 7f53531cf740 1 -- 172.21.15.5:0/1988931803 wait complete. 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 -- start start 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 Processor -- start 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8b99630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8c18a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53481aeea0 con 0x5616c8b958e0 2024-04-22T13:31:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53481ae7d0 con 0x5616c8b96290 2024-04-22T13:31:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53481aba10 con 0x5616c8b9ee10 2024-04-22T13:31:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f53531cf740 10 client.0 _ll_register_callbacks cb 0x7f53440d3e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42628/0 (socket says 172.21.15.5:42628) 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 -- 172.21.15.5:0/1160770968 learned_addr learned my addr 172.21.15.5:0/1160770968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534eb78640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8c18a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534db76640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8b99630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 -- 172.21.15.5:0/1160770968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 msgr2=0x5616c8c18a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8c18a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 -- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 msgr2=0x5616c8b99630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8b99630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534e377640 1 -- 172.21.15.5:0/1160770968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616c8c18fc0 con 0x5616c8b958e0 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.338+0000 7f534db76640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8b99630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f534e377640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b990f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f5344098860 tx=0x7f534400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c017795389e2222a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f5331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f5331ffb640 1 -- 172.21.15.5:0/1160770968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344020070 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f5331ffb640 1 -- 172.21.15.5:0/1160770968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5344098a70 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5344009de0 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f5331ffb640 1 -- 172.21.15.5:0/1160770968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53440b5b80 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 10 client.17274 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616c8c1a260 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f5331ffb640 1 -- 172.21.15.5:0/1160770968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344009de0 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f5331ffb640 1 -- 172.21.15.5:0/1160770968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f53440981e0 con 0x5616c8b958e0 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 10 client.17274 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 20 client.17274 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8c1a920 0x5616c8c1cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 4 client.17274 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:31:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.339+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616c8b01790 con 0x5616c8c1a920 2024-04-22T13:31:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.340+0000 7f534db76640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8c1a920 0x5616c8c1cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.340+0000 7f534db76640 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8c1a920 0x5616c8c1cd10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4440d6b8689f85cb server_cookie=68cc9197f9a491ae in_seq=0 out_seq=0 2024-04-22T13:31:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.340+0000 7f5331ffb640 10 client.17274 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f5331ffb640 1 -- 172.21.15.5:0/1160770968 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616c8b01790 con 0x5616c8c1a920 2024-04-22T13:31:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f5331ffb640 10 client.17274 handle_command_reply: tid=0 2024-04-22T13:31:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 client.17274 shutdown 2024-04-22T13:31:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 msgr2=0x5616c8b990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b990f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f5344098860 tx=0x7f534400bf60 comp rx=0 tx=0).stop 2024-04-22T13:31:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 shutdown_connections 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5616c8c1a920 0x5616c8c1cd10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616c8b9ee10 0x5616c8c18a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f5331ffb640 0 client.17274 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5616c8b96290 0x5616c8b99630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 --2- 172.21.15.5:0/1160770968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616c8b958e0 0x5616c8b990f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 >> 172.21.15.5:0/1160770968 conn(0x5616c8b8ffb0 msgr2=0x5616c8b9c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 shutdown_connections 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 1 -- 172.21.15.5:0/1160770968 wait complete. 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.341+0000 7f53531cf740 20 client.17274 trim_cache size 0 max 16384 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:31:05.345 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:31:05.346 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e4fe9cd4-f817-459b-af55-c0372a45042a", 2024-04-22T13:31:05.346 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:31:05.346 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f53340a3d20 msgr2=0x7f53340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f53340a3d20 0x7f53340a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5340005cc0 tx=0x7f5340005c50 comp rx=0 tx=0).stop 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 msgr2=0x7f534819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 0x7f534819e3f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f534400d950 tx=0x7f534400de10 comp rx=0 tx=0).stop 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 shutdown_connections 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f53340a3d20 0x7f53340a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5348103400 0x7f53481985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5348102a60 0x7f534819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 --2- 172.21.15.5:0/2131901462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5348101180 0x7f534819e3f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 >> 172.21.15.5:0/2131901462 conn(0x7f5348077f70 msgr2=0x7f534806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 shutdown_connections 2024-04-22T13:31:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.342+0000 7f53505db640 1 -- 172.21.15.5:0/2131901462 wait complete. 2024-04-22T13:31:05.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:05.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:05.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 -- 172.21.15.5:0/959064976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4101050 msgr2=0x7f8cd4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 --2- 172.21.15.5:0/959064976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4101050 0x7f8cd4101430 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009900 tx=0x7f8cc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 -- 172.21.15.5:0/959064976 shutdown_connections 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 --2- 172.21.15.5:0/959064976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4106120 0x7f8cd410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 --2- 172.21.15.5:0/959064976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4101a00 0x7f8cd4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 --2- 172.21.15.5:0/959064976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4101050 0x7f8cd4101430 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 -- 172.21.15.5:0/959064976 >> 172.21.15.5:0/959064976 conn(0x7f8cd4077fa0 msgr2=0x7f8cd40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 -- 172.21.15.5:0/959064976 shutdown_connections 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 -- 172.21.15.5:0/959064976 wait complete. 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.458+0000 7f8cda132640 1 Processor -- start 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 -- start start 2024-04-22T13:31:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4101050 0x7f8cd419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 0x7f8cd419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4198590 0x7f8cd4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111060 con 0x7f8cd4106120 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4110ee0 con 0x7f8cd4101050 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cda132640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41111e0 con 0x7f8cd4198590 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 0x7f8cd419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 0x7f8cd419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42638/0 (socket says 172.21.15.5:42638) 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 -- 172.21.15.5:0/4047562725 learned_addr learned my addr 172.21.15.5:0/4047562725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4198590 0x7f8cd4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4101050 0x7f8cd419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 -- 172.21.15.5:0/4047562725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4198590 msgr2=0x7f8cd4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4198590 0x7f8cd4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 -- 172.21.15.5:0/4047562725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4101050 msgr2=0x7f8cd419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4101050 0x7f8cd419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8ccaffd640 1 -- 172.21.15.5:0/4047562725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd41990b0 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4198590 0x7f8cd4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.459+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4101050 0x7f8cd419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.460+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 0x7f8cd419e920 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005fd0 tx=0x7f8cbc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.460+0000 7f8cd17fa640 1 -- 172.21.15.5:0/4047562725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc017070 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.460+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd41993a0 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.460+0000 7f8cd17fa640 1 -- 172.21.15.5:0/4047562725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cbc009c30 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.460+0000 7f8cd17fa640 1 -- 172.21.15.5:0/4047562725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc003d30 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.460+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd41a55f0 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cd17fa640 1 -- 172.21.15.5:0/4047562725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8cbc003ed0 con 0x7f8cd4106120 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 Processor -- start 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 -- start start 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5be960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 0x556f5b5c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5c7a80 0x556f5b5c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556f5b5231e0 con 0x556f5b5beea0 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f5b4c0390 con 0x556f5b5be580 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.462+0000 7f8cdce04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556f5b51f1a0 con 0x556f5b5c7a80 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 0x556f5b5c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 0x556f5b5c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42644/0 (socket says 172.21.15.5:42644) 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 -- 172.21.15.5:0/2743776864 learned_addr learned my addr 172.21.15.5:0/2743776864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd17fa640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8cc00a3d20 0x7f8cc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5c7a80 0x556f5b5c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8cc00a3d20 0x7f8cc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 -- 172.21.15.5:0/2743776864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5c7a80 msgr2=0x556f5b5c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5c7a80 0x556f5b5c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 -- 172.21.15.5:0/2743776864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 msgr2=0x556f5b5be960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5be960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 -- 172.21.15.5:0/2743776864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5b2ff7d0 con 0x556f5b5beea0 2024-04-22T13:31:05.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd17fa640 1 -- 172.21.15.5:0/4047562725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8cbc0d04d0 con 0x7f8cd4106120 2024-04-22T13:31:05.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 0x556f5b5c7540 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4036040 tx=0x7f8cc402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d96f5874f032 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8cc00a3d20 0x7f8cc00a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005fd0 tx=0x7f8ccc004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cc9ffb640 1 -- 172.21.15.5:0/2743776864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4043020 con 0x556f5b5beea0 2024-04-22T13:31:05.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cc9ffb640 1 -- 172.21.15.5:0/2743776864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc402c7e0 con 0x556f5b5beea0 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.463+0000 7f8cc9ffb640 1 -- 172.21.15.5:0/2743776864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4005590 con 0x556f5b5beea0 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 -- 172.21.15.5:0/2743776864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 msgr2=0x556f5b5c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 0x556f5b5c7540 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4036040 tx=0x7f8cc402c680 comp rx=0 tx=0).stop 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 -- 172.21.15.5:0/2743776864 shutdown_connections 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5beea0 0x556f5b5c7540 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5c7a80 0x556f5b5c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2743776864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5be960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 -- 172.21.15.5:0/2743776864 >> 172.21.15.5:0/2743776864 conn(0x556f5b5b8cb0 msgr2=0x556f5b5b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 -- 172.21.15.5:0/2743776864 shutdown_connections 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 -- 172.21.15.5:0/2743776864 wait complete. 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.464+0000 7f8cdce04740 1 -- start start 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 Processor -- start 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5be580 0x556f5b5c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5beea0 0x556f5b5c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5c5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556f5b5ca810 con 0x556f5b5c7a80 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556f5b4dc2f0 con 0x556f5b5beea0 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556f5b5ca990 con 0x556f5b5be580 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cdce04740 10 client.0 _ll_register_callbacks cb 0x556f5b4e9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42660/0 (socket says 172.21.15.5:42660) 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 -- 172.21.15.5:0/2811389577 learned_addr learned my addr 172.21.15.5:0/2811389577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5be580 0x556f5b5c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5beea0 0x556f5b5c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 -- 172.21.15.5:0/2811389577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5be580 msgr2=0x556f5b5c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5be580 0x556f5b5c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 -- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5beea0 msgr2=0x556f5b5c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5beea0 0x556f5b5c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 -- 172.21.15.5:0/2811389577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f5b2353f0 con 0x556f5b5c7a80 2024-04-22T13:31:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.465+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5c5530 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc012bc0 tx=0x7f8ccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bb13db6f3e6c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/2811389577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc01a070 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/2811389577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8ccc029400 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc402c7e0 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/2811389577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc014020 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 10 client.17286 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f5b1f50f0 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/2811389577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc402c7e0 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/2811389577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8ccc00f040 con 0x556f5b5c7a80 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 10 client.17286 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 20 client.17286 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b5d7b10 0x556f5b5d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 4 client.17286 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.466+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f5b530c00 con 0x556f5b5d7b10 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.467+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b5d7b10 0x556f5b5d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.467+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b5d7b10 0x556f5b5d9f40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc7f8c3f81a67b85 server_cookie=dbdc8920e7eef88d in_seq=0 out_seq=0 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.467+0000 7f8cb2ffd640 10 client.17286 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/2811389577 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f5b530c00 con 0x556f5b5d7b10 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cb2ffd640 10 client.17286 handle_command_reply: tid=0 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 client.17286 shutdown 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 msgr2=0x556f5b5c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5c5530 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc012bc0 tx=0x7f8ccc004a40 comp rx=0 tx=0).stop 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 shutdown_connections 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b5d7b10 0x556f5b5d9f40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5c5530 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cb2ffd640 0 client.17286 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5beea0 0x556f5b5c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 --2- 172.21.15.5:0/2811389577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5be580 0x556f5b5c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 >> 172.21.15.5:0/2811389577 conn(0x556f5b5b8cb0 msgr2=0x556f5b5baa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 shutdown_connections 2024-04-22T13:31:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 1 -- 172.21.15.5:0/2811389577 wait complete. 2024-04-22T13:31:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.468+0000 7f8cdce04740 20 client.17286 trim_cache size 0 max 16384 2024-04-22T13:31:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 Processor -- start 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 -- start start 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41adb80 con 0x556f5b5c7a80 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41af6c0 con 0x556f5b5be580 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cdce04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41ae250 con 0x556f5b5beea0 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5dfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42674/0 (socket says 172.21.15.5:42674) 2024-04-22T13:31:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 -- 172.21.15.5:0/3984883372 learned_addr learned my addr 172.21.15.5:0/3984883372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 -- 172.21.15.5:0/3984883372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 msgr2=0x556f5b5df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 -- 172.21.15.5:0/3984883372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 msgr2=0x556f5b5d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.487+0000 7f8cd3fff640 1 -- 172.21.15.5:0/3984883372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5b5d6660 con 0x556f5b5c7a80 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.488+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.488+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5dfd50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009850 tx=0x7f8ccc0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b83d8c64c3665 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.488+0000 7f8cc97fa640 1 -- 172.21.15.5:0/3984883372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc01a070 con 0x556f5b5c7a80 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.488+0000 7f8cc97fa640 1 -- 172.21.15.5:0/3984883372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc011e30 con 0x556f5b5c7a80 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.488+0000 7f8cc97fa640 1 -- 172.21.15.5:0/3984883372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc0214d0 con 0x556f5b5c7a80 2024-04-22T13:31:05.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- 172.21.15.5:0/3984883372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 msgr2=0x556f5b5dfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5dfd50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009850 tx=0x7f8ccc0095f0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- 172.21.15.5:0/3984883372 shutdown_connections 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5c7a80 0x556f5b5dfd50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- 172.21.15.5:0/3984883372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5be580 0x556f5b5d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- 172.21.15.5:0/3984883372 >> 172.21.15.5:0/3984883372 conn(0x556f5b5b8cb0 msgr2=0x556f5b5c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- 172.21.15.5:0/3984883372 shutdown_connections 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- 172.21.15.5:0/3984883372 wait complete. 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- start start 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 Processor -- start 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 0x556f5b5c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5c7a80 0x556f5b63d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41aeff0 con 0x556f5b5be580 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41ae920 con 0x556f5b5c7a80 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41abb80 con 0x556f5b5beea0 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.489+0000 7f8cdce04740 10 client.0 _ll_register_callbacks cb 0x7f8ccc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 0x556f5b5c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 0x556f5b5c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42688/0 (socket says 172.21.15.5:42688) 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 -- 172.21.15.5:0/954760426 learned_addr learned my addr 172.21.15.5:0/954760426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd3fff640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5c7a80 0x556f5b63d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 -- 172.21.15.5:0/954760426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 msgr2=0x556f5b5c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 -- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5c7a80 msgr2=0x556f5b63d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5c7a80 0x556f5b63d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 -- 172.21.15.5:0/954760426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f5b5c9e70 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cd37fe640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 0x556f5b5c1b10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005260 tx=0x7f8cc40051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df067382c5126ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/954760426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4043040 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/954760426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8cc40048c0 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ccc011e30 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/954760426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8cc402c2b0 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cdce04740 10 client.17292 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.490+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f5b63e500 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/954760426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc011e30 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/954760426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8cc40346e0 con 0x556f5b5be580 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cdce04740 10 client.17292 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cdce04740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cdce04740 20 client.17292 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cdce04740 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b63e7a0 0x556f5b640bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cdce04740 4 client.17292 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f5b52aa70 con 0x556f5b63e7a0 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.491+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b63e7a0 0x556f5b640bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8ccaffd640 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b63e7a0 0x556f5b640bc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eddca5aeaa9709e server_cookie=7a60ae41591fa935 in_seq=0 out_seq=0 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cb2ffd640 10 client.17292 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cb2ffd640 1 -- 172.21.15.5:0/954760426 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556f5b52aa70 con 0x556f5b63e7a0 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cb2ffd640 10 client.17292 handle_command_reply: tid=0 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 client.17292 shutdown 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 msgr2=0x556f5b5c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 0x556f5b5c1b10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005260 tx=0x7f8cc40051f0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 shutdown_connections 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556f5b63e7a0 0x556f5b640bc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f5b5c7a80 0x556f5b63d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f5b5beea0 0x556f5b5c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cb2ffd640 0 client.17292 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 --2- 172.21.15.5:0/954760426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f5b5be580 0x556f5b5c1b10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 >> 172.21.15.5:0/954760426 conn(0x556f5b5b8cb0 msgr2=0x556f5b5c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 shutdown_connections 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 1 -- 172.21.15.5:0/954760426 wait complete. 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.492+0000 7f8cdce04740 20 client.17292 trim_cache size 0 max 16384 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stdout: "e4fe9cd4-f817-459b-af55-c0372a45042a": { 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:31:05.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e4fe9cd4-f817-459b-af55-c0372a45042a", 2024-04-22T13:31:05.496 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:31:05.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:31:05.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:31:05.496 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:31:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8cc00a3d20 msgr2=0x7f8cc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8cc00a3d20 0x7f8cc00a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005fd0 tx=0x7f8ccc004ab0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 msgr2=0x7f8cd419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 0x7f8cd419e920 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005fd0 tx=0x7f8cbc00dde0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 shutdown_connections 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd4106120 0x7f8cd419e920 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8cc00a3d20 0x7f8cc00a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd4198590 0x7f8cd4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 --2- 172.21.15.5:0/4047562725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cd4101050 0x7f8cd419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 >> 172.21.15.5:0/4047562725 conn(0x7f8cd4077fa0 msgr2=0x7f8cd40696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 shutdown_connections 2024-04-22T13:31:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:05.493+0000 7f8cda132640 1 -- 172.21.15.5:0/4047562725 wait complete. 2024-04-22T13:31:05.499 INFO:tasks.cephfs.filesystem:scrub status for tag:e4fe9cd4-f817-459b-af55-c0372a45042a - {'path': '/', 'tag': 'e4fe9cd4-f817-459b-af55-c0372a45042a', 'options': 'recursive,force'} 2024-04-22T13:31:06.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:05 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:31:06.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:05 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:06.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:05 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:31:06.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:05 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:06.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:05 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:31:06.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:05 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:31:06.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:05 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:06.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:05 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:31:06.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:05 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:06.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:05 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:31:06.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:05 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:31:06.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:05 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:06.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:05 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:31:06.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:05 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:06.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:05 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:31:07.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:06 smithi049 ceph-mon[31379]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:07.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:06 smithi107 ceph-mon[31859]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:06 smithi005 ceph-mon[28006]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:09.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:08 smithi049 ceph-mon[31379]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:09.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:08 smithi107 ceph-mon[31859]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:08 smithi005 ceph-mon[28006]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:10 smithi107 ceph-mon[31859]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:10 smithi005 ceph-mon[28006]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:10 smithi049 ceph-mon[31379]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:13.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:12 smithi107 ceph-mon[31859]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:13.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:12 smithi005 ceph-mon[28006]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:13.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:12 smithi049 ceph-mon[31379]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:14 smithi107 ceph-mon[31859]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:15.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:14 smithi005 ceph-mon[28006]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:15.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:14 smithi049 ceph-mon[31379]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:31:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:16 smithi107 ceph-mon[31859]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:17.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:16 smithi005 ceph-mon[28006]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:17.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:16 smithi049 ceph-mon[31379]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:31:19.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:18 smithi107 ceph-mon[31859]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:31:19.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:18 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:31:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:18 smithi005 ceph-mon[28006]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:31:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:18 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:31:19.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:18 smithi049 ceph-mon[31379]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:31:19.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:18 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:31:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:31:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:31:20.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:31:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:31:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:31:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:31:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:31:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:31:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:31:21.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:20 smithi049 ceph-mon[31379]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:31:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:20 smithi107 ceph-mon[31859]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:31:21.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:20 smithi005 ceph-mon[28006]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:31:23.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:22 smithi049 ceph-mon[31379]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:31:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:22 smithi107 ceph-mon[31859]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:31:23.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:22 smithi005 ceph-mon[28006]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:31:25.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:24 smithi049 ceph-mon[31379]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T13:31:25.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:24 smithi107 ceph-mon[31859]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T13:31:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:24 smithi005 ceph-mon[28006]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T13:31:27.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:26 smithi049 ceph-mon[31379]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:31:27.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:26 smithi107 ceph-mon[31859]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:31:27.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:26 smithi005 ceph-mon[28006]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:31:29.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:28 smithi049 ceph-mon[31379]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:29.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:28 smithi107 ceph-mon[31859]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:29.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:28 smithi005 ceph-mon[28006]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:31.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:30 smithi049 ceph-mon[31379]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:31.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:30 smithi107 ceph-mon[31859]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:30 smithi005 ceph-mon[28006]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:33.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:32 smithi049 ceph-mon[31379]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:33.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:32 smithi107 ceph-mon[31859]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:33.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:32 smithi005 ceph-mon[28006]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:35.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:34 smithi049 ceph-mon[31379]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:35.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:34 smithi107 ceph-mon[31859]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:35.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:34 smithi005 ceph-mon[28006]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:35.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:35.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/1786950339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 msgr2=0x7f2e241035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/1786950339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e241035a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009900 tx=0x7f2e1402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/1786950339 shutdown_connections 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/1786950339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e2410c590 0x7f2e2410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/1786950339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e24103ae0 0x7f2e2410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/1786950339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e241035a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/1786950339 >> 172.21.15.5:0/1786950339 conn(0x7f2e24077f40 msgr2=0x7f2e24078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/1786950339 shutdown_connections 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.624+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/1786950339 wait complete. 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 Processor -- start 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 -- start start 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e24100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e24103ae0 0x7f2e24101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e2410c590 0x7f2e24101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241121e0 con 0x7f2e241031c0 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24112060 con 0x7f2e24103ae0 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e2ac0f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24112360 con 0x7f2e2410c590 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e289ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e24100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e289ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e24100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38198/0 (socket says 172.21.15.5:38198) 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e289ab640 1 -- 172.21.15.5:0/655338672 learned_addr learned my addr 172.21.15.5:0/655338672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.625+0000 7f2e23fff640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e24103ae0 0x7f2e24101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e291ac640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e2410c590 0x7f2e24101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e289ab640 1 -- 172.21.15.5:0/655338672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e2410c590 msgr2=0x7f2e24101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e289ab640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e2410c590 0x7f2e24101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e289ab640 1 -- 172.21.15.5:0/655338672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e24103ae0 msgr2=0x7f2e24101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e289ab640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e24103ae0 0x7f2e24101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e289ab640 1 -- 172.21.15.5:0/655338672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e241a6be0 con 0x7f2e241031c0 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e291ac640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e2410c590 0x7f2e24101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e289ab640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e24100e10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005940 tx=0x7f2e1402d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e21ffb640 1 -- 172.21.15.5:0/655338672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14043020 con 0x7f2e241031c0 2024-04-22T13:31:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e21ffb640 1 -- 172.21.15.5:0/655338672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1402dc90 con 0x7f2e241031c0 2024-04-22T13:31:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e241a6e70 con 0x7f2e241031c0 2024-04-22T13:31:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e21ffb640 1 -- 172.21.15.5:0/655338672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14003ab0 con 0x7f2e241031c0 2024-04-22T13:31:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.626+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241a7330 con 0x7f2e241031c0 2024-04-22T13:31:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.628+0000 7f2e21ffb640 1 -- 172.21.15.5:0/655338672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2e14046020 con 0x7f2e241031c0 2024-04-22T13:31:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 Processor -- start 2024-04-22T13:31:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 -- start start 2024-04-22T13:31:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523904960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c52390d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e21ffb640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2e100a3d20 0x7f2e100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c52390fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5238691e0 con 0x55c523904ea0 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c523806390 con 0x55c52390da80 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c5238651a0 con 0x55c523904580 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e289ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c52390d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e23fff640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2e100a3d20 0x7f2e100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e289ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c52390d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38202/0 (socket says 172.21.15.5:38202) 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e21ffb640 1 -- 172.21.15.5:0/655338672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2e140f2880 con 0x7f2e241031c0 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e289ab640 1 -- 172.21.15.5:0/2709695066 learned_addr learned my addr 172.21.15.5:0/2709695066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c52390fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.629+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523904960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e289ab640 1 -- 172.21.15.5:0/2709695066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 msgr2=0x55c523904960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523904960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e289ab640 1 -- 172.21.15.5:0/2709695066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 msgr2=0x55c52390fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c52390fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e289ab640 1 -- 172.21.15.5:0/2709695066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5236457d0 con 0x55c523904ea0 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e23fff640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2e100a3d20 0x7f2e100a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c005fd0 tx=0x7f2e0c005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c52390d540 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2e140be780 tx=0x7f2e1403ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6928ec3ed6b9f9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e1affd640 1 -- 172.21.15.5:0/2709695066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140be3e0 con 0x55c523904ea0 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e1affd640 1 -- 172.21.15.5:0/2709695066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1402de40 con 0x55c523904ea0 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.630+0000 7f2e1affd640 1 -- 172.21.15.5:0/2709695066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14042170 con 0x55c523904ea0 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2709695066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 msgr2=0x55c52390d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c52390d540 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2e140be780 tx=0x7f2e1403ac10 comp rx=0 tx=0).stop 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2709695066 shutdown_connections 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c52390d540 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c52390fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2709695066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523904960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2709695066 >> 172.21.15.5:0/2709695066 conn(0x55c5238fecb0 msgr2=0x55c5238ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2709695066 shutdown_connections 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2709695066 wait complete. 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 -- start start 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.631+0000 7f2e2d7fd740 1 Processor -- start 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c52390af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c52390b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c523910630 con 0x55c52390da80 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c5238222f0 con 0x55c523904580 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c5239107b0 con 0x55c523904ea0 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e2d7fd740 10 client.0 _ll_register_callbacks cb 0x55c52382f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:35.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c52390b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c52390b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38214/0 (socket says 172.21.15.5:38214) 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 -- 172.21.15.5:0/2375083781 learned_addr learned my addr 172.21.15.5:0/2375083781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e23fff640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c52390af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 -- 172.21.15.5:0/2375083781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 msgr2=0x55c52390af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c52390af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 -- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 msgr2=0x55c52390aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 -- 172.21.15.5:0/2375083781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52357b3f0 con 0x55c52390da80 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c52390b4b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c012d00 tx=0x7f2e1c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99818a35a92dfe9d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e03fff640 1 -- 172.21.15.5:0/2375083781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c01d430 con 0x55c52390da80 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.632+0000 7f2e03fff640 1 -- 172.21.15.5:0/2375083781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2e1c01c3d0 con 0x55c52390da80 2024-04-22T13:31:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c523876c00 con 0x55c52390da80 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e03fff640 1 -- 172.21.15.5:0/2375083781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c01d5d0 con 0x55c52390da80 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 10 client.17304 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c52353b0f0 con 0x55c52390da80 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e03fff640 1 -- 172.21.15.5:0/2375083781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c523876c00 con 0x55c52390da80 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e03fff640 1 -- 172.21.15.5:0/2375083781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e1c01c8d0 con 0x55c52390da80 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 10 client.17304 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 20 client.17304 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c52391d830 0x55c52391fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 4 client.17304 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c523870a70 con 0x55c52391d830 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.633+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c52391d830 0x55c52391fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.634+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c52391d830 0x55c52391fc60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb48219b4c3b94c server_cookie=601d85cf640345ab in_seq=0 out_seq=0 2024-04-22T13:31:35.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.634+0000 7f2e03fff640 10 client.17304 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.634+0000 7f2e03fff640 1 -- 172.21.15.5:0/2375083781 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c523870a70 con 0x55c52391d830 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.634+0000 7f2e03fff640 10 client.17304 handle_command_reply: tid=0 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.634+0000 7f2e2d7fd740 1 client.17304 shutdown 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 msgr2=0x55c52390b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c52390b4b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c012d00 tx=0x7f2e1c0107a0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 shutdown_connections 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c52391d830 0x55c52391fc60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c52390b4b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e03fff640 0 client.17304 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c52390af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2375083781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 >> 172.21.15.5:0/2375083781 conn(0x55c5238fecb0 msgr2=0x55c523900a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 shutdown_connections 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2375083781 wait complete. 2024-04-22T13:31:35.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.635+0000 7f2e2d7fd740 20 client.17304 trim_cache size 0 max 16384 2024-04-22T13:31:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 Processor -- start 2024-04-22T13:31:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 -- start start 2024-04-22T13:31:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c5239194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c5238fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241afc80 con 0x55c52390da80 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b17c0 con 0x55c523904580 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b0350 con 0x55c523904ea0 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c5238fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c5238fdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38224/0 (socket says 172.21.15.5:38224) 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 -- 172.21.15.5:0/2206577721 learned_addr learned my addr 172.21.15.5:0/2206577721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e23fff640 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c5239194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e289ab640 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 -- 172.21.15.5:0/2206577721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 msgr2=0x55c5239194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c5239194f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 -- 172.21.15.5:0/2206577721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 msgr2=0x55c52390d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 -- 172.21.15.5:0/2206577721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5238feae0 con 0x55c52390da80 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.653+0000 7f2e291ac640 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c5238fdd70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c01cd10 tx=0x7f2e1c00af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5de8296cf277a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e1a7fc640 1 -- 172.21.15.5:0/2206577721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c02a070 con 0x55c52390da80 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e1a7fc640 1 -- 172.21.15.5:0/2206577721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c01d770 con 0x55c52390da80 2024-04-22T13:31:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e1a7fc640 1 -- 172.21.15.5:0/2206577721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c010ea0 con 0x55c52390da80 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2206577721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 msgr2=0x55c5238fdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c5238fdd70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c01cd10 tx=0x7f2e1c00af80 comp rx=0 tx=0).stop 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2206577721 shutdown_connections 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c52390da80 0x55c5238fdd70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904ea0 0x55c5239194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/2206577721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c523904580 0x55c52390d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2206577721 >> 172.21.15.5:0/2206577721 conn(0x55c5238fecb0 msgr2=0x55c523909970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2206577721 shutdown_connections 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.654+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/2206577721 wait complete. 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 -- start start 2024-04-22T13:31:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 Processor -- start 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523907880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c523907dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c523908300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b10f0 con 0x55c523904ea0 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b0a20 con 0x55c52390da80 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241adc60 con 0x55c523904580 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e2d7fd740 10 client.0 _ll_register_callbacks cb 0x7f2e1c012910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c523907dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c523907dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38232/0 (socket says 172.21.15.5:38232) 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e23fff640 1 -- 172.21.15.5:0/1053356413 learned_addr learned my addr 172.21.15.5:0/1053356413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e289ab640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523907880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.655+0000 7f2e291ac640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c523908300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e23fff640 1 -- 172.21.15.5:0/1053356413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 msgr2=0x55c523907880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e23fff640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523907880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e23fff640 1 -- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 msgr2=0x55c523908300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e23fff640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c523908300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e23fff640 1 -- 172.21.15.5:0/1053356413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52390fe70 con 0x55c523904ea0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e291ac640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c523908300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e23fff640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c523907dc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0058c0 tx=0x7f2e0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d991aecc70a240e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e03fff640 1 -- 172.21.15.5:0/1053356413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c01f7c0 con 0x55c523904ea0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e03fff640 1 -- 172.21.15.5:0/1053356413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2e0c01f960 con 0x55c523904ea0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1c01d770 con 0x55c523904ea0 2024-04-22T13:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e03fff640 1 -- 172.21.15.5:0/1053356413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e0c01fb60 con 0x55c523904ea0 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e2d7fd740 10 client.17310 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5239840f0 con 0x55c523904ea0 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e03fff640 1 -- 172.21.15.5:0/1053356413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c01d770 con 0x55c523904ea0 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.656+0000 7f2e03fff640 1 -- 172.21.15.5:0/1053356413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e0c01e420 con 0x55c523904ea0 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 10 client.17310 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 20 client.17310 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c523984920 0x55c523986d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 4 client.17310 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5237ec590 con 0x55c523984920 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e289ab640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c523984920 0x55c523986d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e289ab640 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c523984920 0x55c523986d40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fd202336a14546 server_cookie=d051c0ec8d5e3009 in_seq=0 out_seq=0 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e03fff640 10 client.17310 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e03fff640 1 -- 172.21.15.5:0/1053356413 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c5237ec590 con 0x55c523984920 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e03fff640 10 client.17310 handle_command_reply: tid=0 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.657+0000 7f2e2d7fd740 1 client.17310 shutdown 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 msgr2=0x55c523907dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c523907dc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0058c0 tx=0x7f2e0c0046c0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 shutdown_connections 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c523984920 0x55c523986d40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c523904ea0 0x55c523907dc0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c52390da80 0x55c523908300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e03fff640 0 client.17310 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 --2- 172.21.15.5:0/1053356413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c523904580 0x55c523907880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 >> 172.21.15.5:0/1053356413 conn(0x55c5238fecb0 msgr2=0x55c5238fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 shutdown_connections 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 1 -- 172.21.15.5:0/1053356413 wait complete. 2024-04-22T13:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2d7fd740 20 client.17310 trim_cache size 0 max 16384 2024-04-22T13:31:35.660 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:31:35.660 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:31:35.660 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:31:35.660 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2e100a3d20 msgr2=0x7f2e100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2e100a3d20 0x7f2e100a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c005fd0 tx=0x7f2e0c005850 comp rx=0 tx=0).stop 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 msgr2=0x7f2e24100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e24100e10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005940 tx=0x7f2e1402d890 comp rx=0 tx=0).stop 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 shutdown_connections 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2e100a3d20 0x7f2e100a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e2410c590 0x7f2e24101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e24103ae0 0x7f2e24101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 --2- 172.21.15.5:0/655338672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e241031c0 0x7f2e24100e10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.658+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 >> 172.21.15.5:0/655338672 conn(0x7f2e24077f40 msgr2=0x7f2e240fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.659+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 shutdown_connections 2024-04-22T13:31:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.659+0000 7f2e2ac0f640 1 -- 172.21.15.5:0/655338672 wait complete. 2024-04-22T13:31:35.665 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:31:35.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 -- 172.21.15.5:0/2885049510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc102b50 msgr2=0x7f13cc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 --2- 172.21.15.5:0/2885049510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc102b50 0x7f13cc106b90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009900 tx=0x7f13c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 -- 172.21.15.5:0/2885049510 shutdown_connections 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 --2- 172.21.15.5:0/2885049510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc102b50 0x7f13cc106b90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 --2- 172.21.15.5:0/2885049510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13cc107270 0x7f13cc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 --2- 172.21.15.5:0/2885049510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13cc1021a0 0x7f13cc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 -- 172.21.15.5:0/2885049510 >> 172.21.15.5:0/2885049510 conn(0x7f13cc0793e0 msgr2=0x7f13cc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 -- 172.21.15.5:0/2885049510 shutdown_connections 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.772+0000 7f13d2504640 1 -- 172.21.15.5:0/2885049510 wait complete. 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 Processor -- start 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 -- start start 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13cc1021a0 0x7f13cc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13cc107270 0x7f13cc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 0x7f13cc19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1121b0 con 0x7f13cc19cff0 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc112030 con 0x7f13cc1021a0 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d2504640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc112330 con 0x7f13cc107270 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d0aa1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 0x7f13cc19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d0aa1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 0x7f13cc19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38252/0 (socket says 172.21.15.5:38252) 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13d0aa1640 1 -- 172.21.15.5:0/468350946 learned_addr learned my addr 172.21.15.5:0/468350946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13cb7fe640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13cc107270 0x7f13cc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.773+0000 7f13cbfff640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13cc1021a0 0x7f13cc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d0aa1640 1 -- 172.21.15.5:0/468350946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13cc107270 msgr2=0x7f13cc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d0aa1640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13cc107270 0x7f13cc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d0aa1640 1 -- 172.21.15.5:0/468350946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13cc1021a0 msgr2=0x7f13cc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d0aa1640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13cc1021a0 0x7f13cc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d0aa1640 1 -- 172.21.15.5:0/468350946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc19dae0 con 0x7f13cc19cff0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d0aa1640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 0x7f13cc19d4a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f13c400eaa0 tx=0x7f13c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c400cdb0 con 0x7f13cc19cff0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c4005020 con 0x7f13cc19cff0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc19ddd0 con 0x7f13cc19cff0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40116d0 con 0x7f13cc19cff0 2024-04-22T13:31:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.774+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1a5590 con 0x7f13cc19cff0 2024-04-22T13:31:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.775+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc069250 con 0x7f13cc19cff0 2024-04-22T13:31:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.776+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f13c40051d0 con 0x7f13cc19cff0 2024-04-22T13:31:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.777+0000 7f13c97fa640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f13b40a3d20 0x7f13b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.777+0000 7f13cbfff640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f13b40a3d20 0x7f13b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.777+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f13c40d0850 con 0x7f13cc19cff0 2024-04-22T13:31:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.778+0000 7f13cbfff640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f13b40a3d20 0x7f13b40a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005d00 tx=0x7f13bc00b410 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:35.782+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13c4016030 con 0x7f13cc19cff0 2024-04-22T13:31:36.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.009+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13cc102b50 con 0x7f13cc19cff0 2024-04-22T13:31:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.010+0000 7f13c97fa640 1 -- 172.21.15.5:0/468350946 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f13c409c8d0 con 0x7f13cc19cff0 2024-04-22T13:31:36.006 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:31:36.006 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:31:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f13b40a3d20 msgr2=0x7f13b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f13b40a3d20 0x7f13b40a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005d00 tx=0x7f13bc00b410 comp rx=0 tx=0).stop 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 msgr2=0x7f13cc19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 0x7f13cc19d4a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f13c400eaa0 tx=0x7f13c400ef60 comp rx=0 tx=0).stop 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 shutdown_connections 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13cc19cff0 0x7f13cc19d4a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f13b40a3d20 0x7f13b40a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13cc107270 0x7f13cc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 --2- 172.21.15.5:0/468350946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13cc1021a0 0x7f13cc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 >> 172.21.15.5:0/468350946 conn(0x7f13cc0793e0 msgr2=0x7f13cc100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 shutdown_connections 2024-04-22T13:31:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.012+0000 7f13d2504640 1 -- 172.21.15.5:0/468350946 wait complete. 2024-04-22T13:31:36.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:31:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 -- 172.21.15.5:0/2754704981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a8103b70 msgr2=0x7fa7a810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 --2- 172.21.15.5:0/2754704981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a8103b70 0x7fa7a810bfc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa798009900 tx=0x7fa79802bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 -- 172.21.15.5:0/2754704981 shutdown_connections 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 --2- 172.21.15.5:0/2754704981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a8103b70 0x7fa7a810bfc0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 --2- 172.21.15.5:0/2754704981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 0x7fa7a810eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 --2- 172.21.15.5:0/2754704981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a81031c0 0x7fa7a81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 -- 172.21.15.5:0/2754704981 >> 172.21.15.5:0/2754704981 conn(0x7fa7a8077f70 msgr2=0x7fa7a8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 -- 172.21.15.5:0/2754704981 shutdown_connections 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.127+0000 7fa7af72b640 1 -- 172.21.15.5:0/2754704981 wait complete. 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 Processor -- start 2024-04-22T13:31:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 -- start start 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 0x7fa7a8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 0x7fa7a8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8101890 0x7fa7a81a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8112350 con 0x7fa7a81031c0 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81121d0 con 0x7fa7a8101890 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7af72b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81124d0 con 0x7fa7a810c6d0 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7ad4c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 0x7fa7a8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7ad4c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 0x7fa7a8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38266/0 (socket says 172.21.15.5:38266) 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/4044874456 learned_addr learned my addr 172.21.15.5:0/4044874456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7accc6640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 0x7fa7a8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.128+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8101890 0x7fa7a81a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/4044874456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 msgr2=0x7fa7a8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 0x7fa7a8101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/4044874456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8101890 msgr2=0x7fa7a81a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8101890 0x7fa7a81a6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/4044874456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a81a72b0 con 0x7fa7a81031c0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7accc6640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 0x7fa7a8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8101890 0x7fa7a81a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 0x7fa7a8100e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa79e7fc640 1 -- 172.21.15.5:0/4044874456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4017070 con 0x7fa7a81031c0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa79e7fc640 1 -- 172.21.15.5:0/4044874456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a4009c30 con 0x7fa7a81031c0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a81a75a0 con 0x7fa7a81031c0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa79e7fc640 1 -- 172.21.15.5:0/4044874456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400f620 con 0x7fa7a81031c0 2024-04-22T13:31:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.129+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81a7ac0 con 0x7fa7a81031c0 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.131+0000 7fa79e7fc640 1 -- 172.21.15.5:0/4044874456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa7a4010460 con 0x7fa7a81031c0 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 Processor -- start 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 -- start start 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469846e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698477430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698479e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5646983d31f0 con 0x56469846e4d0 2024-04-22T13:31:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5646983704b0 con 0x56469846ee80 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5646983cf1b0 con 0x564698477a00 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa79e7fc640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7940a3d50 0x7fa7940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7adcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469846e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7adcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469846e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38270/0 (socket says 172.21.15.5:38270) 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7adcc8640 1 -- 172.21.15.5:0/4209979102 learned_addr learned my addr 172.21.15.5:0/4209979102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.132+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7940a3d50 0x7fa7940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa79e7fc640 1 -- 172.21.15.5:0/4044874456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa7a40d0490 con 0x7fa7a81031c0 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698477430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7accc6640 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698479e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 -- 172.21.15.5:0/4209979102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 msgr2=0x564698479e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698479e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 -- 172.21.15.5:0/4209979102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 msgr2=0x564698477430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698477430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 -- 172.21.15.5:0/4209979102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646983621b0 con 0x56469846e4d0 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7accc6640 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698479e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469846e8b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0012cf0 tx=0x7fa7a0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b401e160fef4825 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7940a3d50 0x7fa7940a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0010c40 tx=0x7fa7a0017000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7937fe640 1 -- 172.21.15.5:0/4209979102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0022070 con 0x56469846e4d0 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7937fe640 1 -- 172.21.15.5:0/4209979102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a0009cb0 con 0x56469846e4d0 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.133+0000 7fa7937fe640 1 -- 172.21.15.5:0/4209979102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000cdb0 con 0x56469846e4d0 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 -- 172.21.15.5:0/4209979102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 msgr2=0x56469846e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469846e8b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0012cf0 tx=0x7fa7a0010740 comp rx=0 tx=0).stop 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 -- 172.21.15.5:0/4209979102 shutdown_connections 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698479e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698477430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/4209979102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469846e8b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 -- 172.21.15.5:0/4209979102 >> 172.21.15.5:0/4209979102 conn(0x564698468ba0 msgr2=0x564698468f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 -- 172.21.15.5:0/4209979102 shutdown_connections 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 -- 172.21.15.5:0/4209979102 wait complete. 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 -- start start 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.134+0000 7fa7b23fc740 1 Processor -- start 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x564698477390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56469846ee80 0x564698474720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 0x564698474d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56469847a910 con 0x564698477a00 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56469838c500 con 0x56469846e4d0 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56469847aa90 con 0x56469846ee80 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 10 client.0 _ll_register_callbacks cb 0x56469835ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 0x564698474d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 0x564698474d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38274/0 (socket says 172.21.15.5:38274) 2024-04-22T13:31:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 -- 172.21.15.5:0/1527680151 learned_addr learned my addr 172.21.15.5:0/1527680151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x564698477390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7accc6640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56469846ee80 0x564698474720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 -- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56469846ee80 msgr2=0x564698474720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56469846ee80 0x564698474720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 -- 172.21.15.5:0/1527680151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 msgr2=0x564698477390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x564698477390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 -- 172.21.15.5:0/1527680151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646980e53f0 con 0x564698477a00 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7accc6640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56469846ee80 0x564698474720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 0x564698474d30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0012a30 tx=0x7fa7a0012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c200f34e91f078 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa790ff9640 1 -- 172.21.15.5:0/1527680151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a002a030 con 0x564698477a00 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa790ff9640 1 -- 172.21.15.5:0/1527680151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa7a001b040 con 0x564698477a00 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.135+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a0009cb0 con 0x564698477a00 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa790ff9640 1 -- 172.21.15.5:0/1527680151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7a0022040 con 0x564698477a00 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 10 client.17328 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646981af7d0 con 0x564698477a00 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa790ff9640 1 -- 172.21.15.5:0/1527680151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a0009cb0 con 0x564698477a00 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa790ff9640 1 -- 172.21.15.5:0/1527680151 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa7a00314d0 con 0x564698477a00 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 10 client.17328 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 10 client.17328 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 20 client.17328 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x564698489fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 4 client.17328 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564698372930 con 0x564698487bd0 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.136+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x564698489fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.137+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x564698489fc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c841841be98de1ff server_cookie=1c2cbf820aef4021 in_seq=0 out_seq=0 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.137+0000 7fa790ff9640 10 client.17328 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.137+0000 7fa790ff9640 1 -- 172.21.15.5:0/1527680151 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564698372930 con 0x564698487bd0 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.137+0000 7fa790ff9640 10 client.17328 handle_command_reply: tid=0 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 client.17328 shutdown 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 msgr2=0x564698474d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 0x564698474d30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0012a30 tx=0x7fa7a0012ea0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 shutdown_connections 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x564698489fc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564698477a00 0x564698474d30 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56469846ee80 0x564698474720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa790ff9640 0 client.17328 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1527680151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x564698477390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 >> 172.21.15.5:0/1527680151 conn(0x564698468ba0 msgr2=0x56469846a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 shutdown_connections 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1527680151 wait complete. 2024-04-22T13:31:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.138+0000 7fa7b23fc740 20 client.17328 trim_cache size 0 max 16384 2024-04-22T13:31:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 Processor -- start 2024-04-22T13:31:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 -- start start 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x5646984837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 0x564698476e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698477390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81afda0 con 0x56469846ee80 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b18e0 con 0x56469846e4d0 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b0470 con 0x564698477a00 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 0x564698476e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 0x564698476e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38288/0 (socket says 172.21.15.5:38288) 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 -- 172.21.15.5:0/1596994663 learned_addr learned my addr 172.21.15.5:0/1596994663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698477390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 -- 172.21.15.5:0/1596994663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 msgr2=0x564698477390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x5646984837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698477390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 -- 172.21.15.5:0/1596994663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 msgr2=0x5646984837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x5646984837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 -- 172.21.15.5:0/1596994663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469846ec20 con 0x56469846ee80 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698477390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x5646984837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.152+0000 7fa7accc6640 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 0x564698476e50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806a2d0 tx=0x7fa79802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f750a1043fb7a89f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa792ffd640 1 -- 172.21.15.5:0/1596994663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798004340 con 0x56469846ee80 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa792ffd640 1 -- 172.21.15.5:0/1596994663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa79802cc90 con 0x56469846ee80 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa792ffd640 1 -- 172.21.15.5:0/1596994663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798004920 con 0x56469846ee80 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1596994663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 msgr2=0x564698476e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 0x564698476e50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806a2d0 tx=0x7fa79802c650 comp rx=0 tx=0).stop 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1596994663 shutdown_connections 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846ee80 0x564698476e50 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x564698477390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/1596994663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846e4d0 0x5646984837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1596994663 >> 172.21.15.5:0/1596994663 conn(0x564698468ba0 msgr2=0x564698467c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1596994663 shutdown_connections 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.153+0000 7fa7b23fc740 1 -- 172.21.15.5:0/1596994663 wait complete. 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 -- start start 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 Processor -- start 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469847fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698470f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x5646984714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b1210 con 0x56469846e4d0 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b0b40 con 0x56469846ee80 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81adda0 con 0x564698477a00 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7b23fc740 10 client.0 _ll_register_callbacks cb 0x7fa7a0005a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469847fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469847fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38290/0 (socket says 172.21.15.5:38290) 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/695377310 learned_addr learned my addr 172.21.15.5:0/695377310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7accc6640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698470f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x5646984714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 msgr2=0x5646984714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x5646984714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/695377310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 msgr2=0x564698470f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698470f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7ad4c7640 1 -- 172.21.15.5:0/695377310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564698471a70 con 0x56469846e4d0 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7adcc8640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x5646984714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.154+0000 7fa7accc6640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698470f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7ad4c7640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469847fed0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa7a409c160 tx=0x7fa7a400b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1d9b573b0fd00f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa790ff9640 1 -- 172.21.15.5:0/695377310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40b5060 con 0x56469846e4d0 2024-04-22T13:31:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa790ff9640 1 -- 172.21.15.5:0/695377310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa7a4098ad0 con 0x56469846e4d0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79802cc90 con 0x56469846e4d0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa790ff9640 1 -- 172.21.15.5:0/695377310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7a4085070 con 0x56469846e4d0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 10 client.17334 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646984edc20 con 0x56469846e4d0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa790ff9640 1 -- 172.21.15.5:0/695377310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa79802cc90 con 0x56469846e4d0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa790ff9640 1 -- 172.21.15.5:0/695377310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa7a40bb170 con 0x56469846e4d0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 10 client.17334 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 10 client.17334 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 20 client.17334 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x5646984f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 4 client.17334 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.155+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646983e0c10 con 0x564698487bd0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7accc6640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x5646984f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa790ff9640 10 client.17334 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7accc6640 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x5646984f02b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8bcd209061322e1 server_cookie=de2d813f824e4f3e in_seq=0 out_seq=0 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa790ff9640 1 -- 172.21.15.5:0/695377310 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646983e0c10 con 0x564698487bd0 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa790ff9640 10 client.17334 handle_command_reply: tid=0 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 client.17334 shutdown 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 msgr2=0x56469847fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469847fed0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa7a409c160 tx=0x7fa7a400b530 comp rx=0 tx=0).stop 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 shutdown_connections 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564698477a00 0x5646984714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56469846ee80 0x564698470f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x564698487bd0 0x5646984f02b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 --2- 172.21.15.5:0/695377310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469846e4d0 0x56469847fed0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa790ff9640 0 client.17334 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.156+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 >> 172.21.15.5:0/695377310 conn(0x564698468ba0 msgr2=0x564698469f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 shutdown_connections 2024-04-22T13:31:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7b23fc740 1 -- 172.21.15.5:0/695377310 wait complete. 2024-04-22T13:31:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7b23fc740 20 client.17334 trim_cache size 0 max 16384 2024-04-22T13:31:36.160 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7940a3d50 msgr2=0x7fa7940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7940a3d50 0x7fa7940a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0010c40 tx=0x7fa7a0017000 comp rx=0 tx=0).stop 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 msgr2=0x7fa7a8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 0x7fa7a8100e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).stop 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 shutdown_connections 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa7940a3d50 0x7fa7940a6210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7a8101890 0x7fa7a81a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a810c6d0 0x7fa7a8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 --2- 172.21.15.5:0/4044874456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7a81031c0 0x7fa7a8100e10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 >> 172.21.15.5:0/4044874456 conn(0x7fa7a8077f70 msgr2=0x7fa7a80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 shutdown_connections 2024-04-22T13:31:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.157+0000 7fa7af72b640 1 -- 172.21.15.5:0/4044874456 wait complete. 2024-04-22T13:31:36.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:31:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 -- 172.21.15.5:0/2446565678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 msgr2=0x7f1620103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 --2- 172.21.15.5:0/2446565678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f1620103fb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1614009930 tx=0x7f161402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 -- 172.21.15.5:0/2446565678 shutdown_connections 2024-04-22T13:31:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 --2- 172.21.15.5:0/2446565678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f1620103fb0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 --2- 172.21.15.5:0/2446565678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16201044f0 0x7f162010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 --2- 172.21.15.5:0/2446565678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1620102270 0x7f1620102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 -- 172.21.15.5:0/2446565678 >> 172.21.15.5:0/2446565678 conn(0x7f16200fda10 msgr2=0x7f16200ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 -- 172.21.15.5:0/2446565678 shutdown_connections 2024-04-22T13:31:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.271+0000 7f16251c7640 1 -- 172.21.15.5:0/2446565678 wait complete. 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 Processor -- start 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 -- start start 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1620102270 0x7f162019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f162019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16201044f0 0x7f16201985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16201122e0 con 0x7f1620103b50 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1620112160 con 0x7f16201044f0 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.272+0000 7f16251c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1620112460 con 0x7f1620102270 2024-04-22T13:31:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f162019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f162019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38302/0 (socket says 172.21.15.5:38302) 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 -- 172.21.15.5:0/2281529768 learned_addr learned my addr 172.21.15.5:0/2281529768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161ed9d640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1620102270 0x7f162019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161f59e640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16201044f0 0x7f16201985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 -- 172.21.15.5:0/2281529768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1620102270 msgr2=0x7f162019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1620102270 0x7f162019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 -- 172.21.15.5:0/2281529768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16201044f0 msgr2=0x7f16201985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16201044f0 0x7f16201985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 -- 172.21.15.5:0/2281529768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1620198d60 con 0x7f1620103b50 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161f59e640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16201044f0 0x7f16201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f161e59c640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f162019e970 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1614000c00 tx=0x7f16140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f1607fff640 1 -- 172.21.15.5:0/2281529768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614043020 con 0x7f1620103b50 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f1607fff640 1 -- 172.21.15.5:0/2281529768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161402cb70 con 0x7f1620103b50 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1620198ff0 con 0x7f1620103b50 2024-04-22T13:31:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f1607fff640 1 -- 172.21.15.5:0/2281529768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614003b80 con 0x7f1620103b50 2024-04-22T13:31:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.273+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16201a52f0 con 0x7f1620103b50 2024-04-22T13:31:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.275+0000 7f1607fff640 1 -- 172.21.15.5:0/2281529768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1614046020 con 0x7f1620103b50 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 Processor -- start 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 -- start start 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c609781f0 con 0x562c60a134d0 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562c609154b0 con 0x562c60a13e80 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1627db9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562c609741b0 con 0x562c60a1ca00 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1607fff640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16080a3d20 0x7f16080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38312/0 (socket says 172.21.15.5:38312) 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161f59e640 1 -- 172.21.15.5:0/2787829114 learned_addr learned my addr 172.21.15.5:0/2787829114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161f59e640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16080a3d20 0x7f16080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f1607fff640 1 -- 172.21.15.5:0/2281529768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f16140f2870 con 0x7f1620103b50 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161e59c640 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161ed9d640 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.276+0000 7f161f59e640 1 -- 172.21.15.5:0/2787829114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 msgr2=0x562c60a1ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161f59e640 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161f59e640 1 -- 172.21.15.5:0/2787829114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 msgr2=0x562c60a1c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161f59e640 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161f59e640 1 -- 172.21.15.5:0/2787829114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c609071b0 con 0x562c60a134d0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161e59c640 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161f59e640 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a138b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1618012dc0 tx=0x7f1618010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d70f3f56f7dd760 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f161f59e640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16080a3d20 0x7f16080a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1618010c40 tx=0x7f161801d000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f16057fa640 1 -- 172.21.15.5:0/2787829114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618022070 con 0x562c60a134d0 2024-04-22T13:31:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f16057fa640 1 -- 172.21.15.5:0/2787829114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1618009dc0 con 0x562c60a134d0 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.277+0000 7f16057fa640 1 -- 172.21.15.5:0/2787829114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161801eac0 con 0x562c60a134d0 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- 172.21.15.5:0/2787829114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 msgr2=0x562c60a138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a138b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1618012dc0 tx=0x7f1618010740 comp rx=0 tx=0).stop 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- 172.21.15.5:0/2787829114 shutdown_connections 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- 172.21.15.5:0/2787829114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a138b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- 172.21.15.5:0/2787829114 >> 172.21.15.5:0/2787829114 conn(0x562c60a0dba0 msgr2=0x562c60a0df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- 172.21.15.5:0/2787829114 shutdown_connections 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- 172.21.15.5:0/2787829114 wait complete. 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- start start 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 Processor -- start 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a1c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c60a1f8b0 con 0x562c60a13e80 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562c60931500 con 0x562c60a134d0 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562c60a1fa30 con 0x562c60a1ca00 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f1627db9740 10 client.0 _ll_register_callbacks cb 0x562c608ffb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f161e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.278+0000 7f161e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a196c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38328/0 (socket says 172.21.15.5:38328) 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 -- 172.21.15.5:0/1282059504 learned_addr learned my addr 172.21.15.5:0/1282059504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161ed9d640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a1c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161f59e640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 -- 172.21.15.5:0/1282059504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 msgr2=0x562c60a19cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a19cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 -- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 msgr2=0x562c60a1c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a1c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 -- 172.21.15.5:0/1282059504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6068a3f0 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f161e59c640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a196c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1614006010 tx=0x7f1614003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be462bf2eb1f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1602ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1602ff5640 1 -- 172.21.15.5:0/1282059504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161403c350 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1602ff5640 1 -- 172.21.15.5:0/1282059504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1614005490 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1618009dc0 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1602ff5640 1 -- 172.21.15.5:0/1282059504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1614005690 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1627db9740 10 client.17346 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c607547d0 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.279+0000 7f1602ff5640 1 -- 172.21.15.5:0/1282059504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1618009dc0 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1602ff5640 1 -- 172.21.15.5:0/1282059504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16140a90b0 con 0x562c60a13e80 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1627db9740 10 client.17346 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1627db9740 10 client.17346 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1627db9740 20 client.17346 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1627db9740 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a2ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1627db9740 4 client.17346 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c60917930 con 0x562c60a2caf0 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f161ed9d640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a2ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f161ed9d640 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a2ef00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26dd0c1c59187f41 server_cookie=c390c7ee1e2bede3 in_seq=0 out_seq=0 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.280+0000 7f1602ff5640 10 client.17346 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1602ff5640 1 -- 172.21.15.5:0/1282059504 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c60917930 con 0x562c60a2caf0 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1602ff5640 10 client.17346 handle_command_reply: tid=0 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 client.17346 shutdown 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 msgr2=0x562c60a196c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a196c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1614006010 tx=0x7f1614003680 comp rx=0 tx=0).stop 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 shutdown_connections 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a2ef00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a196c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1602ff5640 0 client.17346 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 --2- 172.21.15.5:0/1282059504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a1c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 >> 172.21.15.5:0/1282059504 conn(0x562c60a0dba0 msgr2=0x562c60a0f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 shutdown_connections 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 1 -- 172.21.15.5:0/1282059504 wait complete. 2024-04-22T13:31:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.281+0000 7f1627db9740 20 client.17346 trim_cache size 0 max 16384 2024-04-22T13:31:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 Processor -- start 2024-04-22T13:31:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 -- start start 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16201ada40 con 0x562c60a134d0 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f16201af580 con 0x562c60a13e80 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f1627db9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16201ae110 con 0x562c60a1ca00 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38332/0 (socket says 172.21.15.5:38332) 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161ed9d640 1 -- 172.21.15.5:0/1471787778 learned_addr learned my addr 172.21.15.5:0/1471787778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161ed9d640 1 -- 172.21.15.5:0/1471787778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 msgr2=0x562c60a1c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161ed9d640 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161e59c640 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.295+0000 7f161ed9d640 1 -- 172.21.15.5:0/1471787778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 msgr2=0x562c60a1bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f161ed9d640 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f161ed9d640 1 -- 172.21.15.5:0/1471787778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c60a13be0 con 0x562c60a134d0 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f161ed9d640 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a287a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x562c60a14280 tx=0x7f16100139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963298134e7455bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1604ff9640 1 -- 172.21.15.5:0/1471787778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610023070 con 0x562c60a134d0 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1604ff9640 1 -- 172.21.15.5:0/1471787778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161001d470 con 0x562c60a134d0 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1604ff9640 1 -- 172.21.15.5:0/1471787778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610003a10 con 0x562c60a134d0 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 -- 172.21.15.5:0/1471787778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 msgr2=0x562c60a287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a287a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x562c60a14280 tx=0x7f16100139f0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 -- 172.21.15.5:0/1471787778 shutdown_connections 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a1c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a13e80 0x562c60a1bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 --2- 172.21.15.5:0/1471787778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a134d0 0x562c60a287a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 -- 172.21.15.5:0/1471787778 >> 172.21.15.5:0/1471787778 conn(0x562c60a0dba0 msgr2=0x562c60a0cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.296+0000 7f1627db9740 1 -- 172.21.15.5:0/1471787778 shutdown_connections 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 -- 172.21.15.5:0/1471787778 wait complete. 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 -- start start 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 Processor -- start 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a20ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a21220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16201aeeb0 con 0x562c60a13e80 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f16201ae7e0 con 0x562c60a134d0 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16201aba40 con 0x562c60a1ca00 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f1627db9740 10 client.0 _ll_register_callbacks cb 0x7f1614009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a20ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a20ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38342/0 (socket says 172.21.15.5:38342) 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 -- 172.21.15.5:0/3623215927 learned_addr learned my addr 172.21.15.5:0/3623215927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161f59e640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a21220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161ed9d640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 -- 172.21.15.5:0/3623215927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 msgr2=0x562c60a21220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a21220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 -- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 msgr2=0x562c60a207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.297+0000 7f161e59c640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f161e59c640 1 -- 172.21.15.5:0/3623215927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c60a920d0 con 0x562c60a13e80 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f161ed9d640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f161e59c640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a20ce0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1614005ab0 tx=0x7f1614009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f658fbe8bebb2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1602ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1602ff5640 1 -- 172.21.15.5:0/3623215927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140be2b0 con 0x562c60a13e80 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1602ff5640 1 -- 172.21.15.5:0/3623215927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f16140892b0 con 0x562c60a13e80 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f161001d470 con 0x562c60a13e80 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1602ff5640 1 -- 172.21.15.5:0/3623215927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16140c75c0 con 0x562c60a13e80 2024-04-22T13:31:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 10 client.17352 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c60a933b0 con 0x562c60a13e80 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1602ff5640 1 -- 172.21.15.5:0/3623215927 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161001d470 con 0x562c60a13e80 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1602ff5640 1 -- 172.21.15.5:0/3623215927 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16140ddb60 con 0x562c60a13e80 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 10 client.17352 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 10 client.17352 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 20 client.17352 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a95990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 4 client.17352 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.298+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c60985c10 con 0x562c60a2caf0 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f161ed9d640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a95990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f161ed9d640 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a95990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1900d8b829256a22 server_cookie=6e8af8031116f380 in_seq=0 out_seq=0 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1602ff5640 10 client.17352 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1602ff5640 1 -- 172.21.15.5:0/3623215927 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c60985c10 con 0x562c60a2caf0 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1602ff5640 10 client.17352 handle_command_reply: tid=0 2024-04-22T13:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 client.17352 shutdown 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 msgr2=0x562c60a20ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a20ce0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1614005ab0 tx=0x7f1614009260 comp rx=0 tx=0).stop 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 shutdown_connections 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x562c60a2caf0 0x562c60a95990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c60a13e80 0x562c60a20ce0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c60a1ca00 0x562c60a21220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 --2- 172.21.15.5:0/3623215927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562c60a134d0 0x562c60a207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1602ff5640 0 client.17352 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 >> 172.21.15.5:0/3623215927 conn(0x562c60a0dba0 msgr2=0x562c60a0c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 shutdown_connections 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 1 -- 172.21.15.5:0/3623215927 wait complete. 2024-04-22T13:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.299+0000 7f1627db9740 20 client.17352 trim_cache size 0 max 16384 2024-04-22T13:31:36.304 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16080a3d20 msgr2=0x7f16080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16080a3d20 0x7f16080a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1618010c40 tx=0x7f161801d000 comp rx=0 tx=0).stop 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 msgr2=0x7f162019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f162019e970 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1614000c00 tx=0x7f16140045e0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 shutdown_connections 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1620103b50 0x7f162019e970 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16080a3d20 0x7f16080a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16201044f0 0x7f16201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 --2- 172.21.15.5:0/2281529768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1620102270 0x7f162019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 >> 172.21.15.5:0/2281529768 conn(0x7f16200fda10 msgr2=0x7f16200fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 shutdown_connections 2024-04-22T13:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.300+0000 7f16251c7640 1 -- 172.21.15.5:0/2281529768 wait complete. 2024-04-22T13:31:36.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 -- 172.21.15.5:0/4264577228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 msgr2=0x7fbfb010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 --2- 172.21.15.5:0/4264577228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb010eb60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ca0 tx=0x7fbfa802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 -- 172.21.15.5:0/4264577228 shutdown_connections 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 --2- 172.21.15.5:0/4264577228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb010eb60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 --2- 172.21.15.5:0/4264577228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0103b70 0x7fbfb010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 --2- 172.21.15.5:0/4264577228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb01031c0 0x7fbfb01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 -- 172.21.15.5:0/4264577228 >> 172.21.15.5:0/4264577228 conn(0x7fbfb0077f70 msgr2=0x7fbfb0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 -- 172.21.15.5:0/4264577228 shutdown_connections 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.415+0000 7fbfb5470640 1 -- 172.21.15.5:0/4264577228 wait complete. 2024-04-22T13:31:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 Processor -- start 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 -- start start 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb01031c0 0x7fbfb0100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb0101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0101890 0x7fbfb01a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0112380 con 0x7fbfb010c6d0 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0112200 con 0x7fbfb01031c0 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfb5470640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0112500 con 0x7fbfb0101890 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb0101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb0101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38350/0 (socket says 172.21.15.5:38350) 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 -- 172.21.15.5:0/213254631 learned_addr learned my addr 172.21.15.5:0/213254631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0101890 0x7fbfb01a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb01031c0 0x7fbfb0100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 -- 172.21.15.5:0/213254631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0101890 msgr2=0x7fbfb01a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0101890 0x7fbfb01a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 -- 172.21.15.5:0/213254631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb01031c0 msgr2=0x7fbfb0100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb01031c0 0x7fbfb0100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 -- 172.21.15.5:0/213254631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb01a7190 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.416+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb0101350 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c970 tx=0x7fbf9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.417+0000 7fbfacff9640 1 -- 172.21.15.5:0/213254631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98007bb0 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.417+0000 7fbfacff9640 1 -- 172.21.15.5:0/213254631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf98007d50 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.417+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb01a7480 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.417+0000 7fbfacff9640 1 -- 172.21.15.5:0/213254631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800f450 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.417+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01a79d0 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.419+0000 7fbfacff9640 1 -- 172.21.15.5:0/213254631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbf98004eb0 con 0x7fbfb010c6d0 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.419+0000 7fbfb8059740 1 Processor -- start 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.419+0000 7fbfb8059740 1 -- start start 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca34d0 0x55f527ca38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca3e80 0x55f527cac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527caee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfb8059740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f527c081f0 con 0x55f527caca00 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfb8059740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f527ba54b0 con 0x55f527ca34d0 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfb8059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f527c041b0 con 0x55f527ca3e80 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527caee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527caee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38352/0 (socket says 172.21.15.5:38352) 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 -- 172.21.15.5:0/2683425562 learned_addr learned my addr 172.21.15.5:0/2683425562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca34d0 0x55f527ca38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfacff9640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca3e80 0x55f527cac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 -- 172.21.15.5:0/2683425562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca3e80 msgr2=0x55f527cac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca3e80 0x55f527cac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 -- 172.21.15.5:0/2683425562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca34d0 msgr2=0x55f527ca38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca34d0 0x55f527ca38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 -- 172.21.15.5:0/2683425562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f527b971b0 con 0x55f527caca00 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca34d0 0x55f527ca38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfacff9640 1 -- 172.21.15.5:0/213254631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbf980d0470 con 0x7fbfb010c6d0 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.420+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527caee50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbf980cdad0 tx=0x7fbf9801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268c08294ca801b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfa57fa640 1 -- 172.21.15.5:0/2683425562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9809bfd0 con 0x55f527caca00 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e30 tx=0x7fbfa802e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfa57fa640 1 -- 172.21.15.5:0/2683425562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf98046350 con 0x55f527caca00 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfa57fa640 1 -- 172.21.15.5:0/2683425562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9809bdf0 con 0x55f527caca00 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 -- 172.21.15.5:0/2683425562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 msgr2=0x55f527caee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527caee50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbf980cdad0 tx=0x7fbf9801f000 comp rx=0 tx=0).stop 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 -- 172.21.15.5:0/2683425562 shutdown_connections 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527caee50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca3e80 0x55f527cac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 --2- 172.21.15.5:0/2683425562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca34d0 0x55f527ca38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.421+0000 7fbfb8059740 1 -- 172.21.15.5:0/2683425562 >> 172.21.15.5:0/2683425562 conn(0x55f527c9dba0 msgr2=0x55f527c9df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 -- 172.21.15.5:0/2683425562 shutdown_connections 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 -- 172.21.15.5:0/2683425562 wait complete. 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 -- start start 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 Processor -- start 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527ca96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527ca9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f527caf910 con 0x55f527caca00 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f527bc1500 con 0x55f527ca3e80 2024-04-22T13:31:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f527cafa90 con 0x55f527ca34d0 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfb8059740 10 client.0 _ll_register_callbacks cb 0x55f527b8fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527ca9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527ca9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38354/0 (socket says 172.21.15.5:38354) 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/106814912 learned_addr learned my addr 172.21.15.5:0/106814912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.422+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527ca96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 msgr2=0x55f527cac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/106814912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 msgr2=0x55f527ca96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527ca96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/106814912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52791a3f0 con 0x55f527caca00 2024-04-22T13:31:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527ca96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527ca9cf0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8036000 tx=0x7fbfa8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54884b77ae0595ff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbf967fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbf967fc640 1 -- 172.21.15.5:0/106814912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8052060 con 0x55f527caca00 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbf967fc640 1 -- 172.21.15.5:0/106814912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbfa8004050 con 0x55f527caca00 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f527ba7930 con 0x55f527caca00 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbf967fc640 1 -- 172.21.15.5:0/106814912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfa8046070 con 0x55f527caca00 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfb8059740 10 client.17364 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5279e47d0 con 0x55f527caca00 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbf967fc640 1 -- 172.21.15.5:0/106814912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f527ba7930 con 0x55f527caca00 2024-04-22T13:31:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbf967fc640 1 -- 172.21.15.5:0/106814912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbfa803f040 con 0x55f527caca00 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfb8059740 10 client.17364 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.423+0000 7fbfb8059740 10 client.17364 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbfb8059740 20 client.17364 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbfb8059740 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527cbcb90 0x55f527cbefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbfb8059740 4 client.17364 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f527c15c10 con 0x55f527cbcb90 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527cbcb90 0x55f527cbefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbf967fc640 10 client.17364 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.424+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527cbcb90 0x55f527cbefa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=665cc96711e39e04 server_cookie=a46005d926c27186 in_seq=0 out_seq=0 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbf967fc640 1 -- 172.21.15.5:0/106814912 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f527c15c10 con 0x55f527cbcb90 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbf967fc640 10 client.17364 handle_command_reply: tid=0 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 client.17364 shutdown 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 msgr2=0x55f527ca9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527ca9cf0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8036000 tx=0x7fbfa8005d80 comp rx=0 tx=0).stop 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 shutdown_connections 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527cbcb90 0x55f527cbefa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527ca9cf0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527ca96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 --2- 172.21.15.5:0/106814912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbf967fc640 0 client.17364 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 >> 172.21.15.5:0/106814912 conn(0x55f527c9dba0 msgr2=0x55f527c9f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 shutdown_connections 2024-04-22T13:31:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 1 -- 172.21.15.5:0/106814912 wait complete. 2024-04-22T13:31:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.425+0000 7fbfb8059740 20 client.17364 trim_cache size 0 max 16384 2024-04-22T13:31:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.438+0000 7fbfb8059740 1 Processor -- start 2024-04-22T13:31:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.438+0000 7fbfb8059740 1 -- start start 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cb87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527c9ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527cabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfb8059740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01afe10 con 0x55f527caca00 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfb8059740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b1950 con 0x55f527ca3e80 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfb8059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b04e0 con 0x55f527ca34d0 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527cabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527cabe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38362/0 (socket says 172.21.15.5:38362) 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/470229157 learned_addr learned my addr 172.21.15.5:0/470229157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cb87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527c9ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/470229157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 msgr2=0x55f527cb87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cb87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/470229157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 msgr2=0x55f527c9ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527c9ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 -- 172.21.15.5:0/470229157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f527caf850 con 0x55f527caca00 2024-04-22T13:31:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.439+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527cabe10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fbfa802c630 tx=0x7fbfa802c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23670231f1e69e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfa4ff9640 1 -- 172.21.15.5:0/470229157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8052070 con 0x55f527caca00 2024-04-22T13:31:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfa4ff9640 1 -- 172.21.15.5:0/470229157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa8041730 con 0x55f527caca00 2024-04-22T13:31:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfa4ff9640 1 -- 172.21.15.5:0/470229157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8039040 con 0x55f527caca00 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 -- 172.21.15.5:0/470229157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 msgr2=0x55f527cabe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527cabe10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fbfa802c630 tx=0x7fbfa802c810 comp rx=0 tx=0).stop 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 -- 172.21.15.5:0/470229157 shutdown_connections 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527caca00 0x55f527cabe10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527ca3e80 0x55f527c9ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 --2- 172.21.15.5:0/470229157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527cb87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 -- 172.21.15.5:0/470229157 >> 172.21.15.5:0/470229157 conn(0x55f527c9dba0 msgr2=0x55f527c9c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 -- 172.21.15.5:0/470229157 shutdown_connections 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.440+0000 7fbfb8059740 1 -- 172.21.15.5:0/470229157 wait complete. 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 -- start start 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 Processor -- start 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527ca6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 0x55f527ca6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527caca00 0x55f527ca6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b1280 con 0x55f527ca3e80 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b0bb0 con 0x55f527caca00 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01ade10 con 0x55f527ca34d0 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfb8059740 10 client.0 _ll_register_callbacks cb 0x7fbfa8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 0x55f527ca6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 0x55f527ca6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38374/0 (socket says 172.21.15.5:38374) 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 -- 172.21.15.5:0/267212774 learned_addr learned my addr 172.21.15.5:0/267212774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527ca6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527caca00 0x55f527ca6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 -- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 msgr2=0x55f527ca6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527ca6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 -- 172.21.15.5:0/267212774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527caca00 msgr2=0x55f527ca6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527caca00 0x55f527ca6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfa7fff640 1 -- 172.21.15.5:0/267212774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f527d220e0 con 0x55f527ca3e80 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.441+0000 7fbfaf7fe640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527caca00 0x55f527ca6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfa7fff640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 0x55f527ca6780 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c940 tx=0x7fbf9809bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2eb0edf409a071 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbf967fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbf967fc640 1 -- 172.21.15.5:0/267212774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9801d030 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbf967fc640 1 -- 172.21.15.5:0/267212774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbf98005560 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa8041730 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbf967fc640 1 -- 172.21.15.5:0/267212774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf9809c490 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 10 client.17370 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f527d23280 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbf967fc640 1 -- 172.21.15.5:0/267212774 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa8041730 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbf967fc640 1 -- 172.21.15.5:0/267212774 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbf98098b00 con 0x55f527ca3e80 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 10 client.17370 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 10 client.17370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 20 client.17370 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527d23620 0x55f527d25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 4 client.17370 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.442+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f527c0fa80 con 0x55f527d23620 2024-04-22T13:31:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527d23620 0x55f527d25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbf967fc640 10 client.17370 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfaeffd640 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527d23620 0x55f527d25a10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54e3f8c7b2359f7d server_cookie=268457fadb4748b7 in_seq=0 out_seq=0 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbf967fc640 1 -- 172.21.15.5:0/267212774 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f527c0fa80 con 0x55f527d23620 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbf967fc640 10 client.17370 handle_command_reply: tid=0 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 client.17370 shutdown 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 msgr2=0x55f527ca6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 0x55f527ca6780 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c940 tx=0x7fbf9809bdf0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 shutdown_connections 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f527d23620 0x55f527d25a10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f527ca3e80 0x55f527ca6780 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f527caca00 0x55f527ca6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbf967fc640 0 client.17370 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 --2- 172.21.15.5:0/267212774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f527ca34d0 0x55f527ca6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.443+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 >> 172.21.15.5:0/267212774 conn(0x55f527c9dba0 msgr2=0x55f527ca4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 shutdown_connections 2024-04-22T13:31:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb8059740 1 -- 172.21.15.5:0/267212774 wait complete. 2024-04-22T13:31:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb8059740 20 client.17370 trim_cache size 0 max 16384 2024-04-22T13:31:36.449 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf9c0a3d20 msgr2=0x7fbf9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e30 tx=0x7fbfa802e040 comp rx=0 tx=0).stop 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 msgr2=0x7fbfb0101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb0101350 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c970 tx=0x7fbf9800ce30 comp rx=0 tx=0).stop 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 shutdown_connections 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb010c6d0 0x7fbfb0101350 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0101890 0x7fbfb01a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 --2- 172.21.15.5:0/213254631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfb01031c0 0x7fbfb0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 >> 172.21.15.5:0/213254631 conn(0x7fbfb0077f70 msgr2=0x7fbfb00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 shutdown_connections 2024-04-22T13:31:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.444+0000 7fbfb5470640 1 -- 172.21.15.5:0/213254631 wait complete. 2024-04-22T13:31:36.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:31:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 -- 172.21.15.5:0/3638138469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc074101090 msgr2=0x7fc074101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/3638138469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc074101090 0x7fc074101470 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 -- 172.21.15.5:0/3638138469 shutdown_connections 2024-04-22T13:31:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/3638138469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc074106160 0x7fc07410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/3638138469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc074101a40 0x7fc074105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/3638138469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc074101090 0x7fc074101470 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 -- 172.21.15.5:0/3638138469 >> 172.21.15.5:0/3638138469 conn(0x7fc074078000 msgr2=0x7fc074078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 -- 172.21.15.5:0/3638138469 shutdown_connections 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 -- 172.21.15.5:0/3638138469 wait complete. 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.560+0000 7fc07b1e0640 1 Processor -- start 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 -- start start 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc074101090 0x7fc074198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc074106160 0x7fc074198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 0x7fc07419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741121e0 con 0x7fc07419d000 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc074112060 con 0x7fc074106160 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07b1e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc074112360 con 0x7fc074101090 2024-04-22T13:31:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 0x7fc07419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 0x7fc07419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38390/0 (socket says 172.21.15.5:38390) 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 -- 172.21.15.5:0/2081351352 learned_addr learned my addr 172.21.15.5:0/2081351352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc078f7c640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc074101090 0x7fc074198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc073fff640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc074106160 0x7fc074198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 -- 172.21.15.5:0/2081351352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc074101090 msgr2=0x7fc074198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc074101090 0x7fc074198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 -- 172.21.15.5:0/2081351352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc074106160 msgr2=0x7fc074198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc074106160 0x7fc074198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 -- 172.21.15.5:0/2081351352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419daf0 con 0x7fc07419d000 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.561+0000 7fc07977d640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 0x7fc07419d4b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00da40 tx=0x7fc06c00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.562+0000 7fc071ffb640 1 -- 172.21.15.5:0/2081351352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c017070 con 0x7fc07419d000 2024-04-22T13:31:36.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.562+0000 7fc071ffb640 1 -- 172.21.15.5:0/2081351352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06c003e20 con 0x7fc07419d000 2024-04-22T13:31:36.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.562+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07419dde0 con 0x7fc07419d000 2024-04-22T13:31:36.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.562+0000 7fc071ffb640 1 -- 172.21.15.5:0/2081351352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c010460 con 0x7fc07419d000 2024-04-22T13:31:36.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.562+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a55d0 con 0x7fc07419d000 2024-04-22T13:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.564+0000 7fc07ddcd740 1 Processor -- start 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.564+0000 7fc071ffb640 1 -- 172.21.15.5:0/2081351352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc06c004000 con 0x7fc07419d000 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.564+0000 7fc07ddcd740 1 -- start start 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad3b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3be80 0x5649aad44430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 0x5649aad46e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649aaca01f0 con 0x5649aad44a00 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5649aac3d4b0 con 0x5649aad3b4d0 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5649aac9c1b0 con 0x5649aad3be80 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 0x5649aad46e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 0x5649aad46e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38398/0 (socket says 172.21.15.5:38398) 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 -- 172.21.15.5:0/1452823984 learned_addr learned my addr 172.21.15.5:0/1452823984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc078f7c640 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3be80 0x5649aad44430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc071ffb640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0640a3da0 0x7fc0640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad3b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 -- 172.21.15.5:0/1452823984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3be80 msgr2=0x5649aad44430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3be80 0x5649aad44430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 -- 172.21.15.5:0/1452823984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 msgr2=0x5649aad3b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad3b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 -- 172.21.15.5:0/1452823984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649aac2f1b0 con 0x5649aad44a00 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc07977d640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0640a3da0 0x7fc0640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc071ffb640 1 -- 172.21.15.5:0/2081351352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc06c0d0360 con 0x7fc07419d000 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc073fff640 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 0x5649aad46e50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cf50 tx=0x7fc06000c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee74cc45324e21ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.565+0000 7fc05effd640 1 -- 172.21.15.5:0/1452823984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060012650 con 0x5649aad44a00 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc05effd640 1 -- 172.21.15.5:0/1452823984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc060017030 con 0x5649aad44a00 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc05effd640 1 -- 172.21.15.5:0/1452823984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000fa30 con 0x5649aad44a00 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 -- 172.21.15.5:0/1452823984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 msgr2=0x5649aad46e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 0x5649aad46e50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cf50 tx=0x7fc06000c670 comp rx=0 tx=0).stop 2024-04-22T13:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07977d640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0640a3da0 0x7fc0640a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc06c09c2c0 tx=0x7fc06c06ddf0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 -- 172.21.15.5:0/1452823984 shutdown_connections 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad44a00 0x5649aad46e50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3be80 0x5649aad44430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/1452823984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad3b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 -- 172.21.15.5:0/1452823984 >> 172.21.15.5:0/1452823984 conn(0x5649aad35ba0 msgr2=0x5649aad35f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 -- 172.21.15.5:0/1452823984 shutdown_connections 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.566+0000 7fc07ddcd740 1 -- 172.21.15.5:0/1452823984 wait complete. 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 -- start start 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 Processor -- start 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad44300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad41690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad44a00 0x5649aad41c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649aad47970 con 0x5649aad3be80 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5649aac59500 con 0x5649aad3b4d0 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5649aad47af0 con 0x5649aad44a00 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc07ddcd740 10 client.0 _ll_register_callbacks cb 0x5649aac27b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc078f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad41690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc078f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad41690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38410/0 (socket says 172.21.15.5:38410) 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc078f7c640 1 -- 172.21.15.5:0/501984068 learned_addr learned my addr 172.21.15.5:0/501984068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc073fff640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad44a00 0x5649aad41c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.567+0000 7fc073fff640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad44300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc078f7c640 1 -- 172.21.15.5:0/501984068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad44a00 msgr2=0x5649aad41c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc078f7c640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad44a00 0x5649aad41c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc078f7c640 1 -- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 msgr2=0x5649aad44300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc078f7c640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad44300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc078f7c640 1 -- 172.21.15.5:0/501984068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649aa9b23f0 con 0x5649aad3be80 2024-04-22T13:31:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc073fff640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad44300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc078f7c640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad41690 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc06802c170 tx=0x7fc06802c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5bc69ce92d9560 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc053fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc053fff640 1 -- 172.21.15.5:0/501984068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06802c610 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc053fff640 1 -- 172.21.15.5:0/501984068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc068005030 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc060017030 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc053fff640 1 -- 172.21.15.5:0/501984068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc068005230 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 10 client.17382 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649aaa7c7d0 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc053fff640 1 -- 172.21.15.5:0/501984068 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc060017030 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc053fff640 1 -- 172.21.15.5:0/501984068 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc068003680 con 0x5649aad3be80 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 10 client.17382 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 10 client.17382 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 20 client.17382 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.568+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aad54be0 0x5649aad57040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.569+0000 7fc07ddcd740 4 client.17382 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.569+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649aac3f930 con 0x5649aad54be0 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.569+0000 7fc073fff640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aad54be0 0x5649aad57040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.569+0000 7fc073fff640 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aad54be0 0x5649aad57040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78a62a6d67c01027 server_cookie=41b2d06b7e588b90 in_seq=0 out_seq=0 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.569+0000 7fc053fff640 10 client.17382 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc053fff640 1 -- 172.21.15.5:0/501984068 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649aac3f930 con 0x5649aad54be0 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc053fff640 10 client.17382 handle_command_reply: tid=0 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 client.17382 shutdown 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 msgr2=0x5649aad41690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad41690 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc06802c170 tx=0x7fc06802c9f0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 shutdown_connections 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aad54be0 0x5649aad57040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad41690 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad44a00 0x5649aad41c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/501984068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad3b4d0 0x5649aad44300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc053fff640 0 client.17382 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 >> 172.21.15.5:0/501984068 conn(0x5649aad35ba0 msgr2=0x5649aad378f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 shutdown_connections 2024-04-22T13:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 1 -- 172.21.15.5:0/501984068 wait complete. 2024-04-22T13:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.570+0000 7fc07ddcd740 20 client.17382 trim_cache size 0 max 16384 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 Processor -- start 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 -- start start 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aad50830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aad43dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741adae0 con 0x5649aad3be80 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741af620 con 0x5649aad44a00 2024-04-22T13:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741ae1b0 con 0x5649aad3b4d0 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad4d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38424/0 (socket says 172.21.15.5:38424) 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 -- 172.21.15.5:0/4041264113 learned_addr learned my addr 172.21.15.5:0/4041264113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc073fff640 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aad50830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 -- 172.21.15.5:0/4041264113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 msgr2=0x5649aad50830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aad50830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 -- 172.21.15.5:0/4041264113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 msgr2=0x5649aad43dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aad43dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 -- 172.21.15.5:0/4041264113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649aad3d270 con 0x5649aad3be80 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc078f7c640 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad4d120 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb88513ad70c97b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc05e7fc640 1 -- 172.21.15.5:0/4041264113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068000e60 con 0x5649aad3be80 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.584+0000 7fc05e7fc640 1 -- 172.21.15.5:0/4041264113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc068057020 con 0x5649aad3be80 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc05e7fc640 1 -- 172.21.15.5:0/4041264113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680046c0 con 0x5649aad3be80 2024-04-22T13:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4041264113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 msgr2=0x5649aad4d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad4d120 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802ca60 comp rx=0 tx=0).stop 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4041264113 shutdown_connections 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aad4d120 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aad43dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4041264113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aad50830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4041264113 >> 172.21.15.5:0/4041264113 conn(0x5649aad35ba0 msgr2=0x5649aad34c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4041264113 shutdown_connections 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4041264113 wait complete. 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 -- start start 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.585+0000 7fc07ddcd740 1 Processor -- start 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aadb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aadb3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aadbe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741aef50 con 0x5649aad3be80 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741ae880 con 0x5649aad44a00 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741abae0 con 0x5649aad3b4d0 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc07ddcd740 10 client.0 _ll_register_callbacks cb 0x7fc068004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aadb3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc073fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aadb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aadb3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38434/0 (socket says 172.21.15.5:38434) 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 -- 172.21.15.5:0/4057354498 learned_addr learned my addr 172.21.15.5:0/4057354498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc073fff640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aadbe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 -- 172.21.15.5:0/4057354498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 msgr2=0x5649aadb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aadb3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 -- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 msgr2=0x5649aadbe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aadbe1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc078f7c640 1 -- 172.21.15.5:0/4057354498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649aadb3f10 con 0x5649aad3be80 2024-04-22T13:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.586+0000 7fc073fff640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aadbe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc078f7c640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aadb3680 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc068009870 tx=0x7fc068005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926e227eaf7f15cf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc053fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc053fff640 1 -- 172.21.15.5:0/4057354498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068060020 con 0x5649aad3be80 2024-04-22T13:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc053fff640 1 -- 172.21.15.5:0/4057354498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc068003800 con 0x5649aad3be80 2024-04-22T13:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc068057020 con 0x5649aad3be80 2024-04-22T13:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc053fff640 1 -- 172.21.15.5:0/4057354498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0680053d0 con 0x5649aad3be80 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 10 client.17388 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649aadbf7d0 con 0x5649aad3be80 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc053fff640 1 -- 172.21.15.5:0/4057354498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc068057020 con 0x5649aad3be80 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc053fff640 1 -- 172.21.15.5:0/4057354498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc068052450 con 0x5649aad3be80 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 10 client.17388 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 10 client.17388 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 20 client.17388 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aadbfdd0 0x5649aadc21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 4 client.17388 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.587+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649aacadc10 con 0x5649aadbfdd0 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc073fff640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aadbfdd0 0x5649aadc21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc073fff640 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aadbfdd0 0x5649aadc21f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daff56c377389023 server_cookie=4b755762dba5c99 in_seq=0 out_seq=0 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc053fff640 10 client.17388 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc053fff640 1 -- 172.21.15.5:0/4057354498 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649aacadc10 con 0x5649aadbfdd0 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc053fff640 10 client.17388 handle_command_reply: tid=0 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 client.17388 shutdown 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 msgr2=0x5649aadb3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aadb3680 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc068009870 tx=0x7fc068005c00 comp rx=0 tx=0).stop 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 shutdown_connections 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5649aadbfdd0 0x5649aadc21f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aad3be80 0x5649aadb3680 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649aad44a00 0x5649aadbe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc053fff640 0 client.17388 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 --2- 172.21.15.5:0/4057354498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649aad3b4d0 0x5649aadb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 >> 172.21.15.5:0/4057354498 conn(0x5649aad35ba0 msgr2=0x5649aad3c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 shutdown_connections 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.588+0000 7fc07ddcd740 1 -- 172.21.15.5:0/4057354498 wait complete. 2024-04-22T13:31:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07ddcd740 20 client.17388 trim_cache size 0 max 16384 2024-04-22T13:31:36.593 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0640a3da0 msgr2=0x7fc0640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0640a3da0 0x7fc0640a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc06c09c2c0 tx=0x7fc06c06ddf0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 msgr2=0x7fc07419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 0x7fc07419d4b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00da40 tx=0x7fc06c00df00 comp rx=0 tx=0).stop 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 shutdown_connections 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc07419d000 0x7fc07419d4b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0640a3da0 0x7fc0640a6260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc074106160 0x7fc074198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 --2- 172.21.15.5:0/2081351352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc074101090 0x7fc074198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 >> 172.21.15.5:0/2081351352 conn(0x7fc074078000 msgr2=0x7fc0740ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 shutdown_connections 2024-04-22T13:31:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.589+0000 7fc07b1e0640 1 -- 172.21.15.5:0/2081351352 wait complete. 2024-04-22T13:31:36.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 -- 172.21.15.5:0/991223033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 msgr2=0x7fb804100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 --2- 172.21.15.5:0/991223033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb804100a80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009930 tx=0x7fb7f402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 -- 172.21.15.5:0/991223033 shutdown_connections 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 --2- 172.21.15.5:0/991223033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb804100a80 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 --2- 172.21.15.5:0/991223033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804100fc0 0x7fb80410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 --2- 172.21.15.5:0/991223033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb804106da0 0x7fb804107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 -- 172.21.15.5:0/991223033 >> 172.21.15.5:0/991223033 conn(0x7fb804075f80 msgr2=0x7fb804076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 -- 172.21.15.5:0/991223033 shutdown_connections 2024-04-22T13:31:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.705+0000 7fb80979f640 1 -- 172.21.15.5:0/991223033 wait complete. 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 Processor -- start 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 -- start start 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb8041985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb804100fc0 0x7fb804198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804106da0 0x7fb80419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb804112030 con 0x7fb804100600 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb804111eb0 con 0x7fb804100fc0 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.706+0000 7fb80979f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041121b0 con 0x7fb804106da0 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb8041985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb8041985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38444/0 (socket says 172.21.15.5:38444) 2024-04-22T13:31:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 -- 172.21.15.5:0/1896277928 learned_addr learned my addr 172.21.15.5:0/1896277928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb8037fe640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb804100fc0 0x7fb804198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb808f9e640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804106da0 0x7fb80419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 -- 172.21.15.5:0/1896277928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804106da0 msgr2=0x7fb80419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804106da0 0x7fb80419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 -- 172.21.15.5:0/1896277928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb804100fc0 msgr2=0x7fb804198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb804100fc0 0x7fb804198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 -- 172.21.15.5:0/1896277928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80419d5a0 con 0x7fb804100600 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb808f9e640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804106da0 0x7fb80419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb803fff640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb8041985b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9f0 tx=0x7fb7ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb8017fa640 1 -- 172.21.15.5:0/1896277928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00cd80 con 0x7fb804100600 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb8017fa640 1 -- 172.21.15.5:0/1896277928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec0108e0 con 0x7fb804100600 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80419d890 con 0x7fb804100600 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb8017fa640 1 -- 172.21.15.5:0/1896277928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec018980 con 0x7fb804100600 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.707+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8041a5120 con 0x7fb804100600 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 Processor -- start 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 -- start start 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e2719d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e271021f0 con 0x564e2719df60 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e2709f4b0 con 0x564e2719d5b0 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564e270fe1b0 con 0x564e271a7060 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb8017fa640 1 -- 172.21.15.5:0/1896277928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb7ec010430 con 0x7fb804100600 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb808f9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e2719d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb808f9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e2719d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:33404/0 (socket says 172.21.15.5:33404) 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb808f9e640 1 -- 172.21.15.5:0/597125988 learned_addr learned my addr 172.21.15.5:0/597125988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.710+0000 7fb803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38446/0 (socket says 172.21.15.5:38446) 2024-04-22T13:31:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8037fe640 1 -- 172.21.15.5:0/597125988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 msgr2=0x564e2719d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8037fe640 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e2719d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8037fe640 1 -- 172.21.15.5:0/597125988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 msgr2=0x564e271a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8037fe640 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8037fe640 1 -- 172.21.15.5:0/597125988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e270911b0 con 0x564e271a7060 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb803fff640 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8017fa640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7f00a3d20 0x7fb7f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8037fe640 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a74c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009230 tx=0x7fb7f4009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fdcb2cee54d683c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb808f9e640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7f00a3d20 0x7fb7f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb8017fa640 1 -- 172.21.15.5:0/1896277928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb7ec0d0b40 con 0x7fb804100600 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb7fa7fc640 1 -- 172.21.15.5:0/597125988 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4045020 con 0x564e271a7060 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb7fa7fc640 1 -- 172.21.15.5:0/597125988 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f402c7e0 con 0x564e271a7060 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.711+0000 7fb7fa7fc640 1 -- 172.21.15.5:0/597125988 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4005400 con 0x564e271a7060 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 -- 172.21.15.5:0/597125988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 msgr2=0x564e271a74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb808f9e640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7f00a3d20 0x7fb7f00a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc012640 tx=0x7fb7fc009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a74c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009230 tx=0x7fb7f4009560 comp rx=0 tx=0).stop 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 -- 172.21.15.5:0/597125988 shutdown_connections 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a74c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/597125988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e2719d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 -- 172.21.15.5:0/597125988 >> 172.21.15.5:0/597125988 conn(0x564e27197da0 msgr2=0x564e27198180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 -- 172.21.15.5:0/597125988 shutdown_connections 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 -- 172.21.15.5:0/597125988 wait complete. 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.712+0000 7fb80d5ea740 1 -- start start 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 Processor -- start 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e271ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719df60 0x564e271aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 0x564e271aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e271adf40 con 0x564e271a7060 2024-04-22T13:31:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564e270bb500 con 0x564e2719d5b0 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564e271ae0c0 con 0x564e2719df60 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb80d5ea740 10 client.0 _ll_register_callbacks cb 0x564e27089b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 0x564e271aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719df60 0x564e271aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 0x564e271aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38456/0 (socket says 172.21.15.5:38456) 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 -- 172.21.15.5:0/3813374631 learned_addr learned my addr 172.21.15.5:0/3813374631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb803fff640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e271ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 -- 172.21.15.5:0/3813374631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719df60 msgr2=0x564e271aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719df60 0x564e271aa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 -- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 msgr2=0x564e271ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e271ad3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb808f9e640 1 -- 172.21.15.5:0/3813374631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e26e143f0 con 0x564e271a7060 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.713+0000 7fb8037fe640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719df60 0x564e271aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb808f9e640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 0x564e271aad00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0109f0 tx=0x7fb7fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f81a2b5d1d2cba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb7e37fe640 1 -- 172.21.15.5:0/3813374631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc003d80 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb7e37fe640 1 -- 172.21.15.5:0/3813374631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb7fc01e930 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f402c7e0 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb7e37fe640 1 -- 172.21.15.5:0/3813374631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc01eb30 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 10 client.17400 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e26ede7d0 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb7e37fe640 1 -- 172.21.15.5:0/3813374631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f402c7e0 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb7e37fe640 1 -- 172.21.15.5:0/3813374631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb7fc028490 con 0x564e271a7060 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 10 client.17400 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 20 client.17400 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e271bb190 0x564e271bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 4 client.17400 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.714+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e270a1930 con 0x564e271bb190 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.715+0000 7fb803fff640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e271bb190 0x564e271bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.715+0000 7fb7e37fe640 10 client.17400 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.715+0000 7fb803fff640 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e271bb190 0x564e271bd5a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e249ee3d3f2817 server_cookie=9cf98d97939a2280 in_seq=0 out_seq=0 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb7e37fe640 1 -- 172.21.15.5:0/3813374631 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e270a1930 con 0x564e271bb190 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb7e37fe640 10 client.17400 handle_command_reply: tid=0 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 client.17400 shutdown 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 msgr2=0x564e271aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 0x564e271aad00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0109f0 tx=0x7fb7fc004a40 comp rx=0 tx=0).stop 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 shutdown_connections 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e271bb190 0x564e271bd5a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e271a7060 0x564e271aad00 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719df60 0x564e271aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/3813374631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719d5b0 0x564e271ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 >> 172.21.15.5:0/3813374631 conn(0x564e27197da0 msgr2=0x564e27199af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb7e37fe640 0 client.17400 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 shutdown_connections 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 1 -- 172.21.15.5:0/3813374631 wait complete. 2024-04-22T13:31:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.716+0000 7fb80d5ea740 20 client.17400 trim_cache size 0 max 16384 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 Processor -- start 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 -- start start 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 0x564e271b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719df60 0x564e271aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041ab6b0 con 0x564e2719d5b0 2024-04-22T13:31:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041ad1f0 con 0x564e2719df60 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041abd80 con 0x564e271a7060 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 0x564e271b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 0x564e271b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38464/0 (socket says 172.21.15.5:38464) 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.729+0000 7fb803fff640 1 -- 172.21.15.5:0/2948583345 learned_addr learned my addr 172.21.15.5:0/2948583345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb8037fe640 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719df60 0x564e271aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb803fff640 1 -- 172.21.15.5:0/2948583345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 msgr2=0x564e271a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb803fff640 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb803fff640 1 -- 172.21.15.5:0/2948583345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719df60 msgr2=0x564e271aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb803fff640 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719df60 0x564e271aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb803fff640 1 -- 172.21.15.5:0/2948583345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2719d1e0 con 0x564e2719d5b0 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb8037fe640 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719df60 0x564e271aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb803fff640 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 0x564e271b6e40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c6a0 tx=0x7fb7ec0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd1421d33529544 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb7f9ffb640 1 -- 172.21.15.5:0/2948583345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec010720 con 0x564e2719d5b0 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb7f9ffb640 1 -- 172.21.15.5:0/2948583345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec09d210 con 0x564e2719d5b0 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.730+0000 7fb7f9ffb640 1 -- 172.21.15.5:0/2948583345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec09aea0 con 0x564e2719d5b0 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2948583345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 msgr2=0x564e271b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 0x564e271b6e40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c6a0 tx=0x7fb7ec0bd040 comp rx=0 tx=0).stop 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2948583345 shutdown_connections 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e271a7060 0x564e271a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e2719df60 0x564e271aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2948583345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719d5b0 0x564e271b6e40 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2948583345 >> 172.21.15.5:0/2948583345 conn(0x564e27197da0 msgr2=0x564e27199a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2948583345 shutdown_connections 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2948583345 wait complete. 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- start start 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 Processor -- start 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719d5b0 0x564e271b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e271a7060 0x564e271abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041acb20 con 0x564e2719df60 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041ac450 con 0x564e271a7060 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041a9690 con 0x564e2719d5b0 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb80d5ea740 10 client.0 _ll_register_callbacks cb 0x7fb7fc0106f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.731+0000 7fb8037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38468/0 (socket says 172.21.15.5:38468) 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 -- 172.21.15.5:0/2155374592 learned_addr learned my addr 172.21.15.5:0/2155374592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb808f9e640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e271a7060 0x564e271abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb803fff640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719d5b0 0x564e271b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 -- 172.21.15.5:0/2155374592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719d5b0 msgr2=0x564e271b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719d5b0 0x564e271b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 -- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e271a7060 msgr2=0x564e271abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e271a7060 0x564e271abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 -- 172.21.15.5:0/2155374592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e271ac360 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb803fff640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719d5b0 0x564e271b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb8037fe640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271b3890 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009230 tx=0x7fb7f4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304ef7a9df9049fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb7e37fe640 1 -- 172.21.15.5:0/2155374592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4045030 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb7e37fe640 1 -- 172.21.15.5:0/2155374592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb7f40048c0 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec09d210 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb7e37fe640 1 -- 172.21.15.5:0/2155374592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7f405a050 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb80d5ea740 10 client.17406 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e27220de0 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.732+0000 7fb7e37fe640 1 -- 172.21.15.5:0/2155374592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec09d210 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb7e37fe640 1 -- 172.21.15.5:0/2155374592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb7f40035f0 con 0x564e2719df60 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 10 client.17406 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 20 client.17406 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e27221160 0x564e27223580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 4 client.17406 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e2710fc10 con 0x564e27221160 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb803fff640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e27221160 0x564e27223580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb7e37fe640 10 client.17406 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb803fff640 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e27221160 0x564e27223580 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c6ee6f3586c575 server_cookie=57591d43d9bda1da in_seq=0 out_seq=0 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb7e37fe640 1 -- 172.21.15.5:0/2155374592 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e2710fc10 con 0x564e27221160 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb7e37fe640 10 client.17406 handle_command_reply: tid=0 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.733+0000 7fb80d5ea740 1 client.17406 shutdown 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 msgr2=0x564e271b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271b3890 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009230 tx=0x7fb7f4005270 comp rx=0 tx=0).stop 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 shutdown_connections 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564e27221160 0x564e27223580 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e2719df60 0x564e271b3890 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564e271a7060 0x564e271abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 --2- 172.21.15.5:0/2155374592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e2719d5b0 0x564e271b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb7e37fe640 0 client.17406 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 >> 172.21.15.5:0/2155374592 conn(0x564e27197da0 msgr2=0x564e27196590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 shutdown_connections 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 1 -- 172.21.15.5:0/2155374592 wait complete. 2024-04-22T13:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80d5ea740 20 client.17406 trim_cache size 0 max 16384 2024-04-22T13:31:36.744 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7f00a3d20 msgr2=0x7fb7f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7f00a3d20 0x7fb7f00a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc012640 tx=0x7fb7fc009500 comp rx=0 tx=0).stop 2024-04-22T13:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 msgr2=0x7fb8041985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb8041985b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9f0 tx=0x7fb7ec00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 shutdown_connections 2024-04-22T13:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7f00a3d20 0x7fb7f00a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb804106da0 0x7fb80419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb804100fc0 0x7fb804198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 --2- 172.21.15.5:0/1896277928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804100600 0x7fb8041985b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.734+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 >> 172.21.15.5:0/1896277928 conn(0x7fb804075f80 msgr2=0x7fb8040fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.735+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 shutdown_connections 2024-04-22T13:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:36.735+0000 7fb80979f640 1 -- 172.21.15.5:0/1896277928 wait complete. 2024-04-22T13:31:37.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:36 smithi049 ceph-mon[31379]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:37.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:36 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/468350946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:36 smithi107 ceph-mon[31859]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:36 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/468350946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:36 smithi005 ceph-mon[28006]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:36 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/468350946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:37.747 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:31:37.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:31:37.748 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:31:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 -- 172.21.15.5:0/550739788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 msgr2=0x7fe414110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 --2- 172.21.15.5:0/550739788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe414110ce0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009ca0 tx=0x7fe40c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 -- 172.21.15.5:0/550739788 shutdown_connections 2024-04-22T13:31:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 --2- 172.21.15.5:0/550739788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe414110ce0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 --2- 172.21.15.5:0/550739788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 0x7fe41410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 --2- 172.21.15.5:0/550739788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe414100dd0 0x7fe4141011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 -- 172.21.15.5:0/550739788 >> 172.21.15.5:0/550739788 conn(0x7fe414077f70 msgr2=0x7fe414078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 -- 172.21.15.5:0/550739788 shutdown_connections 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 -- 172.21.15.5:0/550739788 wait complete. 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 Processor -- start 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.871+0000 7fe41b56d640 1 -- start start 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe41b56d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe414100dd0 0x7fe4141a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe41b56d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 0x7fe4141a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe41b56d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe41419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe41b56d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe414114490 con 0x7fe41410e8f0 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe41b56d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe414114310 con 0x7fe414101780 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe41b56d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe414114610 con 0x7fe414100dd0 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe41419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe41419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38482/0 (socket says 172.21.15.5:38482) 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 -- 172.21.15.5:0/3057441974 learned_addr learned my addr 172.21.15.5:0/3057441974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe418b08640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 0x7fe4141a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419309640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe414100dd0 0x7fe4141a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 -- 172.21.15.5:0/3057441974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe414100dd0 msgr2=0x7fe4141a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe414100dd0 0x7fe4141a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 -- 172.21.15.5:0/3057441974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 msgr2=0x7fe4141a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 0x7fe4141a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 -- 172.21.15.5:0/3057441974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41419b040 con 0x7fe41410e8f0 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe418b08640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 0x7fe4141a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe419b0a640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe41419a870 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe40c004af0 tx=0x7fe40c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.872+0000 7fe40a7fc640 1 -- 172.21.15.5:0/3057441974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c039040 con 0x7fe41410e8f0 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.873+0000 7fe40a7fc640 1 -- 172.21.15.5:0/3057441974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe40c044070 con 0x7fe41410e8f0 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.873+0000 7fe40a7fc640 1 -- 172.21.15.5:0/3057441974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c003e60 con 0x7fe41410e8f0 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.873+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41419b2d0 con 0x7fe41410e8f0 2024-04-22T13:31:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.873+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141a7500 con 0x7fe41410e8f0 2024-04-22T13:31:37.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.875+0000 7fe40a7fc640 1 -- 172.21.15.5:0/3057441974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe40c0324c0 con 0x7fe41410e8f0 2024-04-22T13:31:37.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.875+0000 7fe41e234740 1 Processor -- start 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.875+0000 7fe41e234740 1 -- start start 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.875+0000 7fe41e234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.875+0000 7fe41e234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.875+0000 7fe41e234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe41e234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbe7450f00 con 0x55dbe74ec8e0 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe41e234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dbe73ee590 con 0x55dbe74ed290 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe41e234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dbe744cec0 con 0x55dbe74f5e10 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74eccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38484/0 (socket says 172.21.15.5:38484) 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 -- 172.21.15.5:0/3671505726 learned_addr learned my addr 172.21.15.5:0/3671505726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe40a7fc640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4000a3d20 0x7fe4000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe419309640 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe419b0a640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4000a3d20 0x7fe4000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 -- 172.21.15.5:0/3671505726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 msgr2=0x55dbe74f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 -- 172.21.15.5:0/3671505726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 msgr2=0x55dbe74f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 -- 172.21.15.5:0/3671505726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe722b7e0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe40a7fc640 1 -- 172.21.15.5:0/3057441974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe40c0f2b20 con 0x7fe41410e8f0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe418b08640 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74eccc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe404013550 tx=0x7fe40400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb5296c2092fb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe419b0a640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4000a3d20 0x7fe4000a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0beb30 tx=0x7fe40c0df040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.876+0000 7fe3ff7fe640 1 -- 172.21.15.5:0/3671505726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404019030 con 0x55dbe74ec8e0 2024-04-22T13:31:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe3ff7fe640 1 -- 172.21.15.5:0/3671505726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4040035f0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe3ff7fe640 1 -- 172.21.15.5:0/3671505726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404011720 con 0x55dbe74ec8e0 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 -- 172.21.15.5:0/3671505726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 msgr2=0x55dbe74eccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74eccc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe404013550 tx=0x7fe40400b670 comp rx=0 tx=0).stop 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 -- 172.21.15.5:0/3671505726 shutdown_connections 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 --2- 172.21.15.5:0/3671505726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74eccc0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 -- 172.21.15.5:0/3671505726 >> 172.21.15.5:0/3671505726 conn(0x55dbe74e6fb0 msgr2=0x55dbe74e7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 -- 172.21.15.5:0/3671505726 shutdown_connections 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.877+0000 7fe41e234740 1 -- 172.21.15.5:0/3671505726 wait complete. 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 -- start start 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 Processor -- start 2024-04-22T13:31:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbe74f8d80 con 0x55dbe74ec8e0 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dbe740a580 con 0x55dbe74ed290 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dbe74f8f00 con 0x55dbe74f5e10 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe41e234740 10 client.0 _ll_register_callbacks cb 0x55dbe7410650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38488/0 (socket says 172.21.15.5:38488) 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe418b08640 1 -- 172.21.15.5:0/1553663271 learned_addr learned my addr 172.21.15.5:0/1553663271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.878+0000 7fe419309640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 -- 172.21.15.5:0/1553663271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 msgr2=0x55dbe74f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 -- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 msgr2=0x55dbe74f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 -- 172.21.15.5:0/1553663271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe728bf70 con 0x55dbe74ec8e0 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe418b08640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f2e70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe40400b9a0 tx=0x7fe40400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a2ce88bec34583 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe3fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe3fcff9640 1 -- 172.21.15.5:0/1553663271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404022030 con 0x55dbe74ec8e0 2024-04-22T13:31:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe3fcff9640 1 -- 172.21.15.5:0/1553663271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe404003bb0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4040035f0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe3fcff9640 1 -- 172.21.15.5:0/1553663271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe404019050 con 0x55dbe74ec8e0 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe41e234740 10 client.17418 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbe7123170 con 0x55dbe74ec8e0 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.879+0000 7fe3fcff9640 1 -- 172.21.15.5:0/1553663271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4040035f0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe3fcff9640 1 -- 172.21.15.5:0/1553663271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe404004b80 con 0x55dbe74ec8e0 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe41e234740 10 client.17418 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe41e234740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe41e234740 20 client.17418 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe41e234740 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe7506310 0x55dbe7508700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe41e234740 4 client.17418 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe745e920 con 0x55dbe7506310 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe419309640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe7506310 0x55dbe7508700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe419309640 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe7506310 0x55dbe7508700 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91b443eb5d16666 server_cookie=593f05a40c1bef8c in_seq=0 out_seq=0 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.880+0000 7fe3fcff9640 10 client.17418 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe3fcff9640 1 -- 172.21.15.5:0/1553663271 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbe745e920 con 0x55dbe7506310 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe3fcff9640 10 client.17418 handle_command_reply: tid=0 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 client.17418 shutdown 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 msgr2=0x55dbe74f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f2e70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe40400b9a0 tx=0x7fe40400b8f0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 shutdown_connections 2024-04-22T13:31:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe7506310 0x55dbe7508700 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe3fcff9640 0 client.17418 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 --2- 172.21.15.5:0/1553663271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f2e70 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.881+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 >> 172.21.15.5:0/1553663271 conn(0x55dbe74e6fb0 msgr2=0x55dbe74e8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.882+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 shutdown_connections 2024-04-22T13:31:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.882+0000 7fe41e234740 1 -- 172.21.15.5:0/1553663271 wait complete. 2024-04-22T13:31:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.882+0000 7fe41e234740 20 client.17418 trim_cache size 0 max 16384 2024-04-22T13:31:37.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 Processor -- start 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 -- start start 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74ed290 0x55dbe7501c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 0x55dbe74e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141afce0 con 0x55dbe74f5e10 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b1820 con 0x55dbe74ec8e0 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe41e234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b03b0 con 0x55dbe74ed290 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 0x55dbe74e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 0x55dbe74e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38494/0 (socket says 172.21.15.5:38494) 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 -- 172.21.15.5:0/2051563253 learned_addr learned my addr 172.21.15.5:0/2051563253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe419309640 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74ed290 0x55dbe7501c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 -- 172.21.15.5:0/2051563253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74ed290 msgr2=0x55dbe7501c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74ed290 0x55dbe7501c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 -- 172.21.15.5:0/2051563253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ec8e0 msgr2=0x55dbe74f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe418b08640 1 -- 172.21.15.5:0/2051563253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe74f13c0 con 0x55dbe74f5e10 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.900+0000 7fe419309640 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74ed290 0x55dbe7501c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe418b08640 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 0x55dbe74e8af0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe4040059f0 tx=0x7fe404005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbea4bf15699e9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe3feffd640 1 -- 172.21.15.5:0/2051563253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404022040 con 0x55dbe74f5e10 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe3feffd640 1 -- 172.21.15.5:0/2051563253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe40401a040 con 0x55dbe74f5e10 2024-04-22T13:31:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe3feffd640 1 -- 172.21.15.5:0/2051563253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404011a60 con 0x55dbe74f5e10 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 -- 172.21.15.5:0/2051563253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 msgr2=0x55dbe74e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 0x55dbe74e8af0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe4040059f0 tx=0x7fe404005fd0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 -- 172.21.15.5:0/2051563253 shutdown_connections 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74f5e10 0x55dbe74e8af0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74ed290 0x55dbe7501c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 --2- 172.21.15.5:0/2051563253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ec8e0 0x55dbe74f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.901+0000 7fe41e234740 1 -- 172.21.15.5:0/2051563253 >> 172.21.15.5:0/2051563253 conn(0x55dbe74e6fb0 msgr2=0x55dbe74e83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 -- 172.21.15.5:0/2051563253 shutdown_connections 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 -- 172.21.15.5:0/2051563253 wait complete. 2024-04-22T13:31:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 -- start start 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 Processor -- start 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b1150 con 0x55dbe74ec8e0 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b0a80 con 0x55dbe74ed290 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141adcc0 con 0x55dbe74f5e10 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe41e234740 10 client.0 _ll_register_callbacks cb 0x7fe41000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.902+0000 7fe418b08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38504/0 (socket says 172.21.15.5:38504) 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 -- 172.21.15.5:0/2594334116 learned_addr learned my addr 172.21.15.5:0/2594334116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe419309640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 -- 172.21.15.5:0/2594334116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 msgr2=0x55dbe74f07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 -- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 msgr2=0x55dbe74f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 -- 172.21.15.5:0/2594334116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe756b8e0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe418b08640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74efd40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe40400b950 tx=0x7fe404004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11d531bd419b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe3fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe3fcff9640 1 -- 172.21.15.5:0/2594334116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404022070 con 0x55dbe74ec8e0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe3fcff9640 1 -- 172.21.15.5:0/2594334116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fe404013050 con 0x55dbe74ec8e0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40401a040 con 0x55dbe74ec8e0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe3fcff9640 1 -- 172.21.15.5:0/2594334116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4040035f0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe41e234740 10 client.17424 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.903+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbe756cc00 con 0x55dbe74ec8e0 2024-04-22T13:31:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe3fcff9640 1 -- 172.21.15.5:0/2594334116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe40401a040 con 0x55dbe74ec8e0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe3fcff9640 1 -- 172.21.15.5:0/2594334116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe4040051a0 con 0x55dbe74ec8e0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe41e234740 10 client.17424 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe41e234740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe41e234740 20 client.17424 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe41e234740 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe756d0b0 0x55dbe756f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe41e234740 4 client.17424 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dbe7458790 con 0x55dbe756d0b0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe419309640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe756d0b0 0x55dbe756f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe3fcff9640 10 client.17424 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.904+0000 7fe419309640 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe756d0b0 0x55dbe756f4d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed28efe0809a047 server_cookie=cb50a68a8a8c6493 in_seq=0 out_seq=0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe3fcff9640 1 -- 172.21.15.5:0/2594334116 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dbe7458790 con 0x55dbe756d0b0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe3fcff9640 10 client.17424 handle_command_reply: tid=0 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 client.17424 shutdown 2024-04-22T13:31:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 msgr2=0x55dbe74efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74efd40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe40400b950 tx=0x7fe404004170 comp rx=0 tx=0).stop 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 shutdown_connections 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55dbe756d0b0 0x55dbe756f4d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbe74f5e10 0x55dbe74f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dbe74ed290 0x55dbe74f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 --2- 172.21.15.5:0/2594334116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe74ec8e0 0x55dbe74efd40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 >> 172.21.15.5:0/2594334116 conn(0x55dbe74e6fb0 msgr2=0x55dbe74e6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe3fcff9640 0 client.17424 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 shutdown_connections 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 1 -- 172.21.15.5:0/2594334116 wait complete. 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.906+0000 7fe41e234740 20 client.17424 trim_cache size 0 max 16384 2024-04-22T13:31:37.908 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:31:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:31:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "ef009ffb-1895-4a59-b3cb-c29f543284b6", 2024-04-22T13:31:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:31:37.909 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4000a3d20 msgr2=0x7fe4000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4000a3d20 0x7fe4000a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0beb30 tx=0x7fe40c0df040 comp rx=0 tx=0).stop 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 msgr2=0x7fe41419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe41419a870 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe40c004af0 tx=0x7fe40c034760 comp rx=0 tx=0).stop 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 shutdown_connections 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe41410e8f0 0x7fe41419a870 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4000a3d20 0x7fe4000a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe414101780 0x7fe4141a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 --2- 172.21.15.5:0/3057441974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe414100dd0 0x7fe4141a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 >> 172.21.15.5:0/3057441974 conn(0x7fe414077f70 msgr2=0x7fe41410fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 shutdown_connections 2024-04-22T13:31:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:37.907+0000 7fe41b56d640 1 -- 172.21.15.5:0/3057441974 wait complete. 2024-04-22T13:31:37.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:37.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:38.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 -- 172.21.15.5:0/3201315245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa81031c0 msgr2=0x7fcaa81035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 --2- 172.21.15.5:0/3201315245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa81031c0 0x7fcaa81035a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 -- 172.21.15.5:0/3201315245 shutdown_connections 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 --2- 172.21.15.5:0/3201315245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa810c590 0x7fcaa810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 --2- 172.21.15.5:0/3201315245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa8103ae0 0x7fcaa810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 --2- 172.21.15.5:0/3201315245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa81031c0 0x7fcaa81035a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 -- 172.21.15.5:0/3201315245 >> 172.21.15.5:0/3201315245 conn(0x7fcaa8077f40 msgr2=0x7fcaa8078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 -- 172.21.15.5:0/3201315245 shutdown_connections 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 -- 172.21.15.5:0/3201315245 wait complete. 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 Processor -- start 2024-04-22T13:31:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.023+0000 7fcaaf277640 1 -- start start 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa81031c0 0x7fcaa8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 0x7fcaa8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa810c590 0x7fcaa819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81121c0 con 0x7fcaa8103ae0 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8112040 con 0x7fcaa81031c0 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8112340 con 0x7fcaa810c590 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 0x7fcaa8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 0x7fcaa8198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38522/0 (socket says 172.21.15.5:38522) 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 -- 172.21.15.5:0/1936973938 learned_addr learned my addr 172.21.15.5:0/1936973938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaad013640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa81031c0 0x7fcaa8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaad814640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa810c590 0x7fcaa819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 -- 172.21.15.5:0/1936973938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa810c590 msgr2=0x7fcaa819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa810c590 0x7fcaa819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 -- 172.21.15.5:0/1936973938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa81031c0 msgr2=0x7fcaa8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa81031c0 0x7fcaa8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 -- 172.21.15.5:0/1936973938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa819d6f0 con 0x7fcaa8103ae0 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaad013640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa81031c0 0x7fcaa8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaac812640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 0x7fcaa8198c40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d950 tx=0x7fca9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fca9e7fc640 1 -- 172.21.15.5:0/1936973938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98017070 con 0x7fcaa8103ae0 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fca9e7fc640 1 -- 172.21.15.5:0/1936973938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98009c30 con 0x7fcaa8103ae0 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa819d9e0 con 0x7fcaa8103ae0 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fca9e7fc640 1 -- 172.21.15.5:0/1936973938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9800f620 con 0x7fcaa8103ae0 2024-04-22T13:31:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.024+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a5270 con 0x7fcaa8103ae0 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.027+0000 7fca9e7fc640 1 -- 172.21.15.5:0/1936973938 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fca98010460 con 0x7fcaa8103ae0 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.027+0000 7fcab1e60740 1 Processor -- start 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.027+0000 7fcab1e60740 1 -- start start 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0844b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08454540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08456e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcab1e60740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc083b01e0 con 0x55bc0844bea0 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcab1e60740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bc0834d390 con 0x55bc0844b580 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcab1e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bc083ac1a0 con 0x55bc08454a80 2024-04-22T13:31:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fca9e7fc640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fca940a3d20 0x7fca940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08454540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08454540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38536/0 (socket says 172.21.15.5:38536) 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 -- 172.21.15.5:0/629066944 learned_addr learned my addr 172.21.15.5:0/629066944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad814640 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0844b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad814640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fca940a3d20 0x7fca940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaac812640 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08456e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fca9e7fc640 1 -- 172.21.15.5:0/1936973938 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fca980d0540 con 0x7fcaa8103ae0 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 -- 172.21.15.5:0/629066944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 msgr2=0x55bc08456e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08456e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 -- 172.21.15.5:0/629066944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 msgr2=0x55bc0844b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0844b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 -- 172.21.15.5:0/629066944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0818c7d0 con 0x55bc0844bea0 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaac812640 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08456e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad013640 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08454540 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8ca5a76d705fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.028+0000 7fcaad814640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fca940a3d20 0x7fca940a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009370 tx=0x7fcaa0009240 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fca937fe640 1 -- 172.21.15.5:0/629066944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40042c0 con 0x55bc0844bea0 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fca937fe640 1 -- 172.21.15.5:0/629066944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa4004460 con 0x55bc0844bea0 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fca937fe640 1 -- 172.21.15.5:0/629066944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4004920 con 0x55bc0844bea0 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 -- 172.21.15.5:0/629066944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 msgr2=0x55bc08454540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08454540 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa4005e80 comp rx=0 tx=0).stop 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 -- 172.21.15.5:0/629066944 shutdown_connections 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08454540 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08456e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 --2- 172.21.15.5:0/629066944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0844b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 -- 172.21.15.5:0/629066944 >> 172.21.15.5:0/629066944 conn(0x55bc08445cb0 msgr2=0x55bc08446090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 -- 172.21.15.5:0/629066944 shutdown_connections 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.029+0000 7fcab1e60740 1 -- 172.21.15.5:0/629066944 wait complete. 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 -- start start 2024-04-22T13:31:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 Processor -- start 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc084519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844bea0 0x55bc08451f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 0x55bc08452470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc08457810 con 0x55bc08454a80 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bc083692f0 con 0x55bc0844b580 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bc08457990 con 0x55bc0844bea0 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcab1e60740 10 client.0 _ll_register_callbacks cb 0x55bc08376580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcaad814640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 0x55bc08452470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcaad814640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 0x55bc08452470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38538/0 (socket says 172.21.15.5:38538) 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcaad814640 1 -- 172.21.15.5:0/2893329851 learned_addr learned my addr 172.21.15.5:0/2893329851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcaac812640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844bea0 0x55bc08451f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.030+0000 7fcaad013640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc084519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad814640 1 -- 172.21.15.5:0/2893329851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844bea0 msgr2=0x55bc08451f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad814640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844bea0 0x55bc08451f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad814640 1 -- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 msgr2=0x55bc084519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad814640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc084519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad814640 1 -- 172.21.15.5:0/2893329851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc080c23f0 con 0x55bc08454a80 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad013640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc084519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcaad814640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 0x55bc08452470 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0011fd0 tx=0x7fcaa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9a905d868ee9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fca90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fca90ff9640 1 -- 172.21.15.5:0/2893329851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0026020 con 0x55bc08454a80 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa4004460 con 0x55bc08454a80 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fca90ff9640 1 -- 172.21.15.5:0/2893329851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fcaa0022870 con 0x55bc08454a80 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fca90ff9640 1 -- 172.21.15.5:0/2893329851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcaa0031050 con 0x55bc08454a80 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcab1e60740 10 client.17436 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc080820f0 con 0x55bc08454a80 2024-04-22T13:31:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fca90ff9640 1 -- 172.21.15.5:0/2893329851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa4004460 con 0x55bc08454a80 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fca90ff9640 1 -- 172.21.15.5:0/2893329851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcaa000f930 con 0x55bc08454a80 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcab1e60740 10 client.17436 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.031+0000 7fcab1e60740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fcab1e60740 20 client.17436 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc08464a20 0x55bc08466ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fcab1e60740 4 client.17436 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc083bdc00 con 0x55bc08464a20 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fcaad013640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc08464a20 0x55bc08466ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fcaad013640 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc08464a20 0x55bc08466ea0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77377191d550197 server_cookie=11ce7a36ce39ccd1 in_seq=0 out_seq=0 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.032+0000 7fca90ff9640 10 client.17436 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fca90ff9640 1 -- 172.21.15.5:0/2893329851 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc083bdc00 con 0x55bc08464a20 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fca90ff9640 10 client.17436 handle_command_reply: tid=0 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 client.17436 shutdown 2024-04-22T13:31:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 msgr2=0x55bc08452470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 0x55bc08452470 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0011fd0 tx=0x7fcaa0004a40 comp rx=0 tx=0).stop 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 shutdown_connections 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc08464a20 0x55bc08466ea0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc08454a80 0x55bc08452470 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844bea0 0x55bc08451f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fca90ff9640 0 client.17436 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2893329851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc084519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 >> 172.21.15.5:0/2893329851 conn(0x55bc08445cb0 msgr2=0x55bc08447a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 shutdown_connections 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.033+0000 7fcab1e60740 1 -- 172.21.15.5:0/2893329851 wait complete. 2024-04-22T13:31:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.034+0000 7fcab1e60740 20 client.17436 trim_cache size 0 max 16384 2024-04-22T13:31:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 Processor -- start 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 -- start start 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844b580 0x55bc08454120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08460700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc08454a80 0x55bc08450930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81adb00 con 0x55bc0844bea0 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81af640 con 0x55bc08454a80 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.051+0000 7fcab1e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ae1d0 con 0x55bc0844b580 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08460700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08460700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38554/0 (socket says 172.21.15.5:38554) 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 -- 172.21.15.5:0/2178722062 learned_addr learned my addr 172.21.15.5:0/2178722062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaad814640 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc08454a80 0x55bc08450930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 -- 172.21.15.5:0/2178722062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844b580 msgr2=0x55bc08454120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaad013640 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844b580 0x55bc08454120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844b580 0x55bc08454120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 -- 172.21.15.5:0/2178722062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc08454a80 msgr2=0x55bc08450930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc08454a80 0x55bc08450930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 -- 172.21.15.5:0/2178722062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc084457e0 con 0x55bc0844bea0 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fcaac812640 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08460700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fca98098e80 tx=0x7fca9801e310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166cea0f3741efa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fca92ffd640 1 -- 172.21.15.5:0/2178722062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98020040 con 0x55bc0844bea0 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fca92ffd640 1 -- 172.21.15.5:0/2178722062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98009de0 con 0x55bc0844bea0 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.052+0000 7fca92ffd640 1 -- 172.21.15.5:0/2178722062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98018d10 con 0x55bc0844bea0 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 -- 172.21.15.5:0/2178722062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 msgr2=0x55bc08460700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08460700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fca98098e80 tx=0x7fca9801e310 comp rx=0 tx=0).stop 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 -- 172.21.15.5:0/2178722062 shutdown_connections 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc08460700 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc08454a80 0x55bc08450930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 --2- 172.21.15.5:0/2178722062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc0844b580 0x55bc08454120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 -- 172.21.15.5:0/2178722062 >> 172.21.15.5:0/2178722062 conn(0x55bc08445cb0 msgr2=0x55bc08452240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 -- 172.21.15.5:0/2178722062 shutdown_connections 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 -- 172.21.15.5:0/2178722062 wait complete. 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 -- start start 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.053+0000 7fcab1e60740 1 Processor -- start 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0845cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc0845d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08459be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81aef70 con 0x55bc0844bea0 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ae8a0 con 0x55bc0844b580 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81abae0 con 0x55bc08454a80 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 10 client.0 _ll_register_callbacks cb 0x7fcaa0014790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc0845d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc0845d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38558/0 (socket says 172.21.15.5:38558) 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 -- 172.21.15.5:0/637496266 learned_addr learned my addr 172.21.15.5:0/637496266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaad013640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0845cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaad814640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08459be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 -- 172.21.15.5:0/637496266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 msgr2=0x55bc08459be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08459be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 -- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 msgr2=0x55bc0845cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0845cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 -- 172.21.15.5:0/637496266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc0845a2a0 con 0x55bc0844bea0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcaac812640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc0845d150 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fca9809c1b0 tx=0x7fca980cdc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d419bce47b8e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fca90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fca90ff9640 1 -- 172.21.15.5:0/637496266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98020070 con 0x55bc0844bea0 2024-04-22T13:31:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fca90ff9640 1 -- 172.21.15.5:0/637496266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fca98016170 con 0x55bc0844bea0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca98009de0 con 0x55bc0844bea0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.054+0000 7fca90ff9640 1 -- 172.21.15.5:0/637496266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca980a0860 con 0x55bc0844bea0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 10 client.17442 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc084cb9c0 con 0x55bc0844bea0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fca90ff9640 1 -- 172.21.15.5:0/637496266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98009de0 con 0x55bc0844bea0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fca90ff9640 1 -- 172.21.15.5:0/637496266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fca980d0040 con 0x55bc0844bea0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 10 client.17442 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 20 client.17442 populate_metadata read hostname 'smithi005' 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc084cc080 0x55bc084ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 4 client.17442 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc083b7a70 con 0x55bc084cc080 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.055+0000 7fcaad013640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc084cc080 0x55bc084ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcaad013640 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc084cc080 0x55bc084ce470 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447cd5ad2d6e97db server_cookie=cd43cbed06f69c8c in_seq=0 out_seq=0 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fca90ff9640 10 client.17442 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fca90ff9640 1 -- 172.21.15.5:0/637496266 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bc083b7a70 con 0x55bc084cc080 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fca90ff9640 10 client.17442 handle_command_reply: tid=0 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 client.17442 shutdown 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 msgr2=0x55bc0845d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc0845d150 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fca9809c1b0 tx=0x7fca980cdc80 comp rx=0 tx=0).stop 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 shutdown_connections 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc084cc080 0x55bc084ce470 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc0844bea0 0x55bc0845d150 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc08454a80 0x55bc08459be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 --2- 172.21.15.5:0/637496266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc0844b580 0x55bc0845cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 >> 172.21.15.5:0/637496266 conn(0x55bc08445cb0 msgr2=0x55bc08446fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fca90ff9640 0 client.17442 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 shutdown_connections 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 1 -- 172.21.15.5:0/637496266 wait complete. 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.056+0000 7fcab1e60740 20 client.17442 trim_cache size 0 max 16384 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:31:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "ef009ffb-1895-4a59-b3cb-c29f543284b6": { 2024-04-22T13:31:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:31:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "ef009ffb-1895-4a59-b3cb-c29f543284b6", 2024-04-22T13:31:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:31:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:31:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:31:38.061 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fca940a3d20 msgr2=0x7fca940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fca940a3d20 0x7fca940a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009370 tx=0x7fcaa0009240 comp rx=0 tx=0).stop 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 msgr2=0x7fcaa8198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 0x7fcaa8198c40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d950 tx=0x7fca9800de10 comp rx=0 tx=0).stop 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 shutdown_connections 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaa8103ae0 0x7fcaa8198c40 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fca940a3d20 0x7fca940a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa810c590 0x7fcaa819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 --2- 172.21.15.5:0/1936973938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcaa81031c0 0x7fcaa8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 >> 172.21.15.5:0/1936973938 conn(0x7fcaa8077f40 msgr2=0x7fcaa80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 shutdown_connections 2024-04-22T13:31:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:31:38.057+0000 7fcaaf277640 1 -- 172.21.15.5:0/1936973938 wait complete. 2024-04-22T13:31:38.064 INFO:tasks.cephfs.filesystem:scrub status for tag:ef009ffb-1895-4a59-b3cb-c29f543284b6 - {'path': '/', 'tag': 'ef009ffb-1895-4a59-b3cb-c29f543284b6', 'options': 'recursive,force'} 2024-04-22T13:31:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:38 smithi049 ceph-mon[31379]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:38 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:31:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:38 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:38 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:31:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:38 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:38 smithi107 ceph-mon[31859]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:38 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:31:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:38 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:38 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:31:39.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:38 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:39.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:38 smithi005 ceph-mon[28006]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:38 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:31:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:38 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:38 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:31:39.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:38 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:41.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:40 smithi049 ceph-mon[31379]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:41.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:40 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:31:41.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:40 smithi107 ceph-mon[31859]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:41.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:40 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:31:41.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:40 smithi005 ceph-mon[28006]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:41.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:40 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:31:43.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:42 smithi049 ceph-mon[31379]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:43.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:42 smithi107 ceph-mon[31859]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:43.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:42 smithi005 ceph-mon[28006]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:45.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:44 smithi049 ceph-mon[31379]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:45.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:44 smithi107 ceph-mon[31859]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:45.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:44 smithi005 ceph-mon[28006]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:47.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:46 smithi049 ceph-mon[31379]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:47.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:46 smithi107 ceph-mon[31859]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:47.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:46 smithi005 ceph-mon[28006]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:49.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:48 smithi049 ceph-mon[31379]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:49.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:48 smithi107 ceph-mon[31859]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:49.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:48 smithi005 ceph-mon[28006]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:51.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:50 smithi049 ceph-mon[31379]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:51.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:50 smithi107 ceph-mon[31859]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:51.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:50 smithi005 ceph-mon[28006]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:53.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:52 smithi049 ceph-mon[31379]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:53.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:52 smithi107 ceph-mon[31859]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:53.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:52 smithi005 ceph-mon[28006]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:55.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:54 smithi049 ceph-mon[31379]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:55.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:54 smithi107 ceph-mon[31859]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:55.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:54 smithi005 ceph-mon[28006]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:57.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:56 smithi049 ceph-mon[31379]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:57.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:56 smithi107 ceph-mon[31859]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:57.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:56 smithi005 ceph-mon[28006]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:59.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:31:58 smithi049 ceph-mon[31379]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:59.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:31:58 smithi107 ceph-mon[31859]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:31:59.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:31:58 smithi005 ceph-mon[28006]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:01.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:00 smithi049 ceph-mon[31379]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:01.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:00 smithi107 ceph-mon[31859]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:01.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:00 smithi005 ceph-mon[28006]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:03.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:02 smithi049 ceph-mon[31379]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:03.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:02 smithi107 ceph-mon[31859]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:03.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:02 smithi005 ceph-mon[28006]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:05.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:04 smithi049 ceph-mon[31379]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:05.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:04 smithi107 ceph-mon[31859]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:05.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:04 smithi005 ceph-mon[28006]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:07.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:06 smithi049 ceph-mon[31379]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:32:07.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:06 smithi107 ceph-mon[31859]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:32:07.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:06 smithi005 ceph-mon[28006]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:32:08.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:08.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 -- 172.21.15.5:0/3648335633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 msgr2=0x7efdac1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/3648335633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac1035a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 -- 172.21.15.5:0/3648335633 shutdown_connections 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/3648335633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdac10c590 0x7efdac10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/3648335633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdac103ae0 0x7efdac10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/3648335633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac1035a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 -- 172.21.15.5:0/3648335633 >> 172.21.15.5:0/3648335633 conn(0x7efdac077f40 msgr2=0x7efdac078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 -- 172.21.15.5:0/3648335633 shutdown_connections 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.187+0000 7efdb1ff4640 1 -- 172.21.15.5:0/3648335633 wait complete. 2024-04-22T13:32:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.188+0000 7efdb1ff4640 1 Processor -- start 2024-04-22T13:32:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.188+0000 7efdb1ff4640 1 -- start start 2024-04-22T13:32:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdac103ae0 0x7efdac198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdac10c590 0x7efdac19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1121e0 con 0x7efdac1031c0 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdac112060 con 0x7efdac10c590 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdac112360 con 0x7efdac103ae0 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58806/0 (socket says 172.21.15.5:58806) 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 -- 172.21.15.5:0/2918216276 learned_addr learned my addr 172.21.15.5:0/2918216276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdac103ae0 0x7efdac198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdabfff640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdac10c590 0x7efdac19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 -- 172.21.15.5:0/2918216276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdac103ae0 msgr2=0x7efdac198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdac103ae0 0x7efdac198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 -- 172.21.15.5:0/2918216276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdac10c590 msgr2=0x7efdac19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdac10c590 0x7efdac19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 -- 172.21.15.5:0/2918216276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac19d710 con 0x7efdac1031c0 2024-04-22T13:32:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdab7fe640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac198720 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005940 tx=0x7efd9c02c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efda8ff9640 1 -- 172.21.15.5:0/2918216276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c043020 con 0x7efdac1031c0 2024-04-22T13:32:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efda8ff9640 1 -- 172.21.15.5:0/2918216276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c004e00 con 0x7efdac1031c0 2024-04-22T13:32:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac19d9a0 con 0x7efdac1031c0 2024-04-22T13:32:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.189+0000 7efda8ff9640 1 -- 172.21.15.5:0/2918216276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003970 con 0x7efdac1031c0 2024-04-22T13:32:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.190+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1a5290 con 0x7efdac1031c0 2024-04-22T13:32:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efda8ff9640 1 -- 172.21.15.5:0/2918216276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efd9c046020 con 0x7efdac1031c0 2024-04-22T13:32:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 Processor -- start 2024-04-22T13:32:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 -- start start 2024-04-22T13:32:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5580 0x558fa37b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efda8ff9640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37be540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 0x558fa37c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558fa371a1e0 con 0x558fa37bea80 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558fa36b7390 con 0x558fa37b5580 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.192+0000 7efdb4beb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558fa37161a0 con 0x558fa37b5ea0 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdab7fe640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 0x558fa37c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 0x558fa37c0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58818/0 (socket says 172.21.15.5:58818) 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 -- 172.21.15.5:0/3770361384 learned_addr learned my addr 172.21.15.5:0/3770361384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5580 0x558fa37b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdaaffd640 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37be540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efda8ff9640 1 -- 172.21.15.5:0/2918216276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efd9c0f2610 con 0x7efdac1031c0 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 -- 172.21.15.5:0/3770361384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 msgr2=0x558fa37be540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37be540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 -- 172.21.15.5:0/3770361384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5580 msgr2=0x558fa37b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5580 0x558fa37b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 -- 172.21.15.5:0/3770361384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa34f67d0 con 0x558fa37bea80 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdab7fe640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0f0390 tx=0x7efd9c089480 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efdabfff640 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 0x558fa37c0e70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efda4011aa0 tx=0x7efda4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca999ca92b952428 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efd91ffb640 1 -- 172.21.15.5:0/3770361384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4004b20 con 0x558fa37bea80 2024-04-22T13:32:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efd91ffb640 1 -- 172.21.15.5:0/3770361384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda400fd30 con 0x558fa37bea80 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.193+0000 7efd91ffb640 1 -- 172.21.15.5:0/3770361384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40055f0 con 0x558fa37bea80 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 -- 172.21.15.5:0/3770361384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 msgr2=0x558fa37c0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 0x558fa37c0e70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efda4011aa0 tx=0x7efda4004ab0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 -- 172.21.15.5:0/3770361384 shutdown_connections 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37bea80 0x558fa37c0e70 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37be540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 --2- 172.21.15.5:0/3770361384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5580 0x558fa37b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 -- 172.21.15.5:0/3770361384 >> 172.21.15.5:0/3770361384 conn(0x558fa37afcb0 msgr2=0x558fa37b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 -- 172.21.15.5:0/3770361384 shutdown_connections 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 -- 172.21.15.5:0/3770361384 wait complete. 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.194+0000 7efdb4beb740 1 -- start start 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 Processor -- start 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 0x558fa37bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558fa37c1870 con 0x558fa37b5580 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558fa36d32f0 con 0x558fa37bea80 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558fa37c19f0 con 0x558fa37b5ea0 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdb4beb740 10 client.0 _ll_register_callbacks cb 0x558fa36e0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37bbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58820/0 (socket says 172.21.15.5:58820) 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 -- 172.21.15.5:0/2848873523 learned_addr learned my addr 172.21.15.5:0/2848873523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 -- 172.21.15.5:0/2848873523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 msgr2=0x558fa37bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37bc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 -- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 msgr2=0x558fa37bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 0x558fa37bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.195+0000 7efdabfff640 1 -- 172.21.15.5:0/2848873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa342c3f0 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdabfff640 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37bbb10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7efda4009270 tx=0x7efda4011610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501b4e8aa116ec4d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efd8affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efd8affd640 1 -- 172.21.15.5:0/2848873523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda400fa90 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efd8affd640 1 -- 172.21.15.5:0/2848873523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efda4005370 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda400fd30 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efd8affd640 1 -- 172.21.15.5:0/2848873523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efda4005570 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 10 client.17454 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fa33ec0f0 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efd8affd640 1 -- 172.21.15.5:0/2848873523 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda400fd30 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efd8affd640 1 -- 172.21.15.5:0/2848873523 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efda401d030 con 0x558fa37b5580 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 10 client.17454 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 20 client.17454 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa37d2dd0 0x558fa37d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 4 client.17454 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.196+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa3727c00 con 0x558fa37d2dd0 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.197+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa37d2dd0 0x558fa37d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.197+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa37d2dd0 0x558fa37d51c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5250b0fd16129a0 server_cookie=7095dd323b84422e in_seq=0 out_seq=0 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.197+0000 7efd8affd640 10 client.17454 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efd8affd640 1 -- 172.21.15.5:0/2848873523 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fa3727c00 con 0x558fa37d2dd0 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efd8affd640 10 client.17454 handle_command_reply: tid=0 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 client.17454 shutdown 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 msgr2=0x558fa37bbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37bbb10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7efda4009270 tx=0x7efda4011610 comp rx=0 tx=0).stop 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 shutdown_connections 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa37d2dd0 0x558fa37d51c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 0x558fa37bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2848873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37bbb10 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 >> 172.21.15.5:0/2848873523 conn(0x558fa37afcb0 msgr2=0x558fa37b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efd8affd640 0 client.17454 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:08.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 shutdown_connections 2024-04-22T13:32:08.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 1 -- 172.21.15.5:0/2848873523 wait complete. 2024-04-22T13:32:08.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.198+0000 7efdb4beb740 20 client.17454 trim_cache size 0 max 16384 2024-04-22T13:32:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 Processor -- start 2024-04-22T13:32:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 -- start start 2024-04-22T13:32:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5ea0 0x558fa37cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37bea80 0x558fa37d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ab940 con 0x558fa37b5580 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ad480 con 0x558fa37b5ea0 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.216+0000 7efdb4beb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ac010 con 0x558fa37bea80 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58822/0 (socket says 172.21.15.5:58822) 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 -- 172.21.15.5:0/1992125798 learned_addr learned my addr 172.21.15.5:0/1992125798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 -- 172.21.15.5:0/1992125798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37bea80 msgr2=0x558fa37d5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdaaffd640 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5ea0 0x558fa37cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37bea80 0x558fa37d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 -- 172.21.15.5:0/1992125798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5ea0 msgr2=0x558fa37cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5ea0 0x558fa37cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 -- 172.21.15.5:0/1992125798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa37d5670 con 0x558fa37b5580 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdaaffd640 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5ea0 0x558fa37cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efdabfff640 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37be240 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7efda4004a30 tx=0x7efda4032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5deb0fc66f0689d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efd917fa640 1 -- 172.21.15.5:0/1992125798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4022020 con 0x558fa37b5580 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efd917fa640 1 -- 172.21.15.5:0/1992125798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda4024070 con 0x558fa37b5580 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.217+0000 7efd917fa640 1 -- 172.21.15.5:0/1992125798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4011020 con 0x558fa37b5580 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 -- 172.21.15.5:0/1992125798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 msgr2=0x558fa37be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37be240 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7efda4004a30 tx=0x7efda4032cb0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 -- 172.21.15.5:0/1992125798 shutdown_connections 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37bea80 0x558fa37d5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37b5ea0 0x558fa37cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 --2- 172.21.15.5:0/1992125798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37be240 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 -- 172.21.15.5:0/1992125798 >> 172.21.15.5:0/1992125798 conn(0x558fa37afcb0 msgr2=0x558fa37b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 -- 172.21.15.5:0/1992125798 shutdown_connections 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 -- 172.21.15.5:0/1992125798 wait complete. 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 -- start start 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.218+0000 7efdb4beb740 1 Processor -- start 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 0x558fa37b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1acdb0 con 0x558fa37b5580 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ac6e0 con 0x558fa37bea80 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a9920 con 0x558fa37b5ea0 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdb4beb740 10 client.0 _ll_register_callbacks cb 0x7efda4011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58830/0 (socket says 172.21.15.5:58830) 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 -- 172.21.15.5:0/2446347096 learned_addr learned my addr 172.21.15.5:0/2446347096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 -- 172.21.15.5:0/2446347096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 msgr2=0x558fa37b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 -- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 msgr2=0x558fa37b88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 0x558fa37b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 -- 172.21.15.5:0/2446347096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa37b5cd0 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efdabfff640 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37cb320 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7efda4009270 tx=0x7efda4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec58849c9d762f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.219+0000 7efd8affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efd8affd640 1 -- 172.21.15.5:0/2446347096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4022070 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efd8affd640 1 -- 172.21.15.5:0/2446347096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efda4011030 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4024070 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efd8affd640 1 -- 172.21.15.5:0/2446347096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efda4009e60 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 10 client.17460 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fa3838d50 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efd8affd640 1 -- 172.21.15.5:0/2446347096 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda4024070 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efd8affd640 1 -- 172.21.15.5:0/2446347096 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efda401d070 con 0x558fa37b5580 2024-04-22T13:32:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 10 client.17460 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 20 client.17460 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa3839300 0x558fa383b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 4 client.17460 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.220+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fa3721a70 con 0x558fa3839300 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa3839300 0x558fa383b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdaaffd640 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa3839300 0x558fa383b720 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a817d660c9a2062 server_cookie=c5413bace667e02f in_seq=0 out_seq=0 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efd8affd640 10 client.17460 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efd8affd640 1 -- 172.21.15.5:0/2446347096 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558fa3721a70 con 0x558fa3839300 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efd8affd640 10 client.17460 handle_command_reply: tid=0 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 client.17460 shutdown 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 msgr2=0x558fa37cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37cb320 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7efda4009270 tx=0x7efda4011970 comp rx=0 tx=0).stop 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 shutdown_connections 2024-04-22T13:32:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558fa3839300 0x558fa383b720 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558fa37bea80 0x558fa37b88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efd8affd640 0 client.17460 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fa37b5ea0 0x558fa37b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 --2- 172.21.15.5:0/2446347096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fa37b5580 0x558fa37cb320 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.221+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 >> 172.21.15.5:0/2446347096 conn(0x558fa37afcb0 msgr2=0x558fa37b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 shutdown_connections 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb4beb740 1 -- 172.21.15.5:0/2446347096 wait complete. 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb4beb740 20 client.17460 trim_cache size 0 max 16384 2024-04-22T13:32:08.226 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:08.227 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:32:08.227 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:32:08.227 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:08.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 msgr2=0x7efd940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0f0390 tx=0x7efd9c089480 comp rx=0 tx=0).stop 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 msgr2=0x7efdac198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac198720 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005940 tx=0x7efd9c02c690 comp rx=0 tx=0).stop 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 shutdown_connections 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdac10c590 0x7efdac19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdac103ae0 0x7efdac198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 --2- 172.21.15.5:0/2918216276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdac1031c0 0x7efdac198720 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 >> 172.21.15.5:0/2918216276 conn(0x7efdac077f40 msgr2=0x7efdac0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 shutdown_connections 2024-04-22T13:32:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.222+0000 7efdb1ff4640 1 -- 172.21.15.5:0/2918216276 wait complete. 2024-04-22T13:32:08.229 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:32:08.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:32:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 -- 172.21.15.5:0/3882965353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 msgr2=0x7f084c10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 --2- 172.21.15.5:0/3882965353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c10bcf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f083c009900 tx=0x7f083c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 -- 172.21.15.5:0/3882965353 shutdown_connections 2024-04-22T13:32:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 --2- 172.21.15.5:0/3882965353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c10bcf0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 --2- 172.21.15.5:0/3882965353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f084c10c310 0x7f084c10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 --2- 172.21.15.5:0/3882965353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f084c102ef0 0x7f084c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.336+0000 7f085279f640 1 -- 172.21.15.5:0/3882965353 >> 172.21.15.5:0/3882965353 conn(0x7f084c0fc730 msgr2=0x7f084c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 -- 172.21.15.5:0/3882965353 shutdown_connections 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 -- 172.21.15.5:0/3882965353 wait complete. 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 Processor -- start 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 -- start start 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f084c102ef0 0x7f084c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f084c10c310 0x7f084c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f084c112020 con 0x7f084c103810 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f084c111ea0 con 0x7f084c102ef0 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.337+0000 7f085279f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1121a0 con 0x7f084c10c310 2024-04-22T13:32:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58838/0 (socket says 172.21.15.5:58838) 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f085179d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f084c102ef0 0x7f084c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 -- 172.21.15.5:0/2773767398 learned_addr learned my addr 172.21.15.5:0/2773767398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0851f9e640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f084c10c310 0x7f084c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 -- 172.21.15.5:0/2773767398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f084c10c310 msgr2=0x7f084c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f084c10c310 0x7f084c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 -- 172.21.15.5:0/2773767398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f084c102ef0 msgr2=0x7f084c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f084c102ef0 0x7f084c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 -- 172.21.15.5:0/2773767398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19d4e0 con 0x7f084c103810 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f0850f9c640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c198a80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f083c005940 tx=0x7f083c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c043020 con 0x7f084c103810 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19d770 con 0x7f084c103810 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c02dba0 con 0x7f084c103810 2024-04-22T13:32:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c003850 con 0x7f084c103810 2024-04-22T13:32:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.338+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a5060 con 0x7f084c103810 2024-04-22T13:32:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.339+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c04fce0 con 0x7f084c103810 2024-04-22T13:32:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.340+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f083c046020 con 0x7f084c103810 2024-04-22T13:32:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.341+0000 7f08427fc640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08380a3dc0 0x7f08380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.341+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f083c0f2680 con 0x7f084c103810 2024-04-22T13:32:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.341+0000 7f085179d640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08380a3dc0 0x7f08380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.341+0000 7f085179d640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08380a3dc0 0x7f08380a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0848009990 tx=0x7f0848009110 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.346+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f083c04d020 con 0x7f084c103810 2024-04-22T13:32:08.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.574+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f084c199840 con 0x7f084c103810 2024-04-22T13:32:08.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.575+0000 7f08427fc640 1 -- 172.21.15.5:0/2773767398 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f083c0bee60 con 0x7f084c103810 2024-04-22T13:32:08.571 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:32:08.571 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:32:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:32:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08380a3dc0 msgr2=0x7f08380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08380a3dc0 0x7f08380a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0848009990 tx=0x7f0848009110 comp rx=0 tx=0).stop 2024-04-22T13:32:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 msgr2=0x7f084c198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c198a80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f083c005940 tx=0x7f083c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 shutdown_connections 2024-04-22T13:32:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f084c103810 0x7f084c198a80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08380a3dc0 0x7f08380a6280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f084c10c310 0x7f084c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 --2- 172.21.15.5:0/2773767398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f084c102ef0 0x7f084c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 >> 172.21.15.5:0/2773767398 conn(0x7f084c0fc730 msgr2=0x7f084c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 shutdown_connections 2024-04-22T13:32:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.577+0000 7f085279f640 1 -- 172.21.15.5:0/2773767398 wait complete. 2024-04-22T13:32:08.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:32:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.692+0000 7f1aece06640 1 -- 172.21.15.5:0/3632076357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae810c6d0 msgr2=0x7f1ae810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.692+0000 7f1aece06640 1 --2- 172.21.15.5:0/3632076357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae810c6d0 0x7f1ae810eb60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009ca0 tx=0x7f1ae002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.692+0000 7f1aece06640 1 -- 172.21.15.5:0/3632076357 shutdown_connections 2024-04-22T13:32:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.692+0000 7f1aece06640 1 --2- 172.21.15.5:0/3632076357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae810c6d0 0x7f1ae810eb60 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.692+0000 7f1aece06640 1 --2- 172.21.15.5:0/3632076357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8103b70 0x7f1ae810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.692+0000 7f1aece06640 1 --2- 172.21.15.5:0/3632076357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ae81031c0 0x7f1ae81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 -- 172.21.15.5:0/3632076357 >> 172.21.15.5:0/3632076357 conn(0x7f1ae8077f70 msgr2=0x7f1ae8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 -- 172.21.15.5:0/3632076357 shutdown_connections 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 -- 172.21.15.5:0/3632076357 wait complete. 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 Processor -- start 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 -- start start 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 0x7f1ae819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ae810c6d0 0x7f1ae819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8198640 0x7f1ae8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8112380 con 0x7f1ae81031c0 2024-04-22T13:32:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.693+0000 7f1aece06640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8112200 con 0x7f1ae810c6d0 2024-04-22T13:32:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1aece06640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8112500 con 0x7f1ae8198640 2024-04-22T13:32:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 0x7f1ae819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 0x7f1ae819e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58846/0 (socket says 172.21.15.5:58846) 2024-04-22T13:32:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 -- 172.21.15.5:0/4177985789 learned_addr learned my addr 172.21.15.5:0/4177985789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ae810c6d0 0x7f1ae819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8198640 0x7f1ae8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 -- 172.21.15.5:0/4177985789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8198640 msgr2=0x7f1ae8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8198640 0x7f1ae8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 -- 172.21.15.5:0/4177985789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ae810c6d0 msgr2=0x7f1ae819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ae810c6d0 0x7f1ae819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 -- 172.21.15.5:0/4177985789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae8199160 con 0x7f1ae81031c0 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8198640 0x7f1ae8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ae659c640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 0x7f1ae819e440 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c920 tx=0x7f1ad800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ad77fe640 1 -- 172.21.15.5:0/4177985789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8007bb0 con 0x7f1ae81031c0 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ad77fe640 1 -- 172.21.15.5:0/4177985789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad8007d50 con 0x7f1ae81031c0 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae8199450 con 0x7f1ae81031c0 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1ad77fe640 1 -- 172.21.15.5:0/4177985789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8018730 con 0x7f1ae81031c0 2024-04-22T13:32:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.694+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae81a5650 con 0x7f1ae81031c0 2024-04-22T13:32:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1ad77fe640 1 -- 172.21.15.5:0/4177985789 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1ad800f450 con 0x7f1ae81031c0 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 Processor -- start 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 -- start start 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1ad77fe640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1ad00a3d20 0x7f1ad00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b4578198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b457822430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b457824e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b45777e1f0 con 0x55b457819e80 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b45771b4b0 con 0x55b457822a00 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.697+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b45777a1b0 con 0x55b4578194d0 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b457822430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1ad00a3d20 0x7f1ad00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b457822430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58858/0 (socket says 172.21.15.5:58858) 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/1068518021 learned_addr learned my addr 172.21.15.5:0/1068518021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ad77fe640 1 -- 172.21.15.5:0/4177985789 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1ad80d0910 con 0x7f1ae81031c0 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae659c640 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b4578198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b457824e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/1068518021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 msgr2=0x55b4578198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b4578198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/1068518021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 msgr2=0x55b457824e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b457824e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/1068518021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45770d1b0 con 0x55b457819e80 2024-04-22T13:32:08.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b457822430 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00ab70 tx=0x7f1adc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d120be34356858 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1ad00a3d20 0x7f1ad00a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0006010 tx=0x7f1ae0032d20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ad4ff9640 1 -- 172.21.15.5:0/1068518021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc005220 con 0x55b457819e80 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ad4ff9640 1 -- 172.21.15.5:0/1068518021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1adc004c50 con 0x55b457819e80 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.698+0000 7f1ad4ff9640 1 -- 172.21.15.5:0/1068518021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0186a0 con 0x55b457819e80 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 -- 172.21.15.5:0/1068518021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 msgr2=0x55b457822430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b457822430 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00ab70 tx=0x7f1adc00f790 comp rx=0 tx=0).stop 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 -- 172.21.15.5:0/1068518021 shutdown_connections 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b457822430 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b457824e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/1068518021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b4578198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 -- 172.21.15.5:0/1068518021 >> 172.21.15.5:0/1068518021 conn(0x55b457813ba0 msgr2=0x55b457813f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 -- 172.21.15.5:0/1068518021 shutdown_connections 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 -- 172.21.15.5:0/1068518021 wait complete. 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.699+0000 7f1aef9f7740 1 -- start start 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 Processor -- start 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4578194d0 0x55b4578222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b45781f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457822a00 0x55b45781fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b457825970 con 0x55b457819e80 2024-04-22T13:32:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b457737500 con 0x55b4578194d0 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b457825af0 con 0x55b457822a00 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1aef9f7740 10 client.0 _ll_register_callbacks cb 0x55b457705b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b45781f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b45781f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58866/0 (socket says 172.21.15.5:58866) 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/514681022 learned_addr learned my addr 172.21.15.5:0/514681022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae659c640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4578194d0 0x55b4578222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457822a00 0x55b45781fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457822a00 msgr2=0x55b45781fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457822a00 0x55b45781fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/514681022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4578194d0 msgr2=0x55b4578222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4578194d0 0x55b4578222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 -- 172.21.15.5:0/514681022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4574903f0 con 0x55b457819e80 2024-04-22T13:32:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b45781f640 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00ab40 tx=0x7f1adc004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdeb3030dfe5f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1acdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1acdffb640 1 -- 172.21.15.5:0/514681022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc027070 con 0x55b457819e80 2024-04-22T13:32:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.700+0000 7f1acdffb640 1 -- 172.21.15.5:0/514681022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1adc018650 con 0x55b457819e80 2024-04-22T13:32:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1adc004c50 con 0x55b457819e80 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1acdffb640 1 -- 172.21.15.5:0/514681022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1adc033050 con 0x55b457819e80 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 10 client.17478 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b45755a7d0 con 0x55b457819e80 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1acdffb640 1 -- 172.21.15.5:0/514681022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1adc004c50 con 0x55b457819e80 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1acdffb640 1 -- 172.21.15.5:0/514681022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1adc023430 con 0x55b457819e80 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 10 client.17478 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 10 client.17478 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 20 client.17478 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b457832aa0 0x55b457834e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 4 client.17478 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b45771d930 con 0x55b457832aa0 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.701+0000 7f1ae659c640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b457832aa0 0x55b457834e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.702+0000 7f1ae659c640 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b457832aa0 0x55b457834e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eacde3233ac2541a server_cookie=e7450b12aea36a35 in_seq=0 out_seq=0 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.702+0000 7f1acdffb640 10 client.17478 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.702+0000 7f1acdffb640 1 -- 172.21.15.5:0/514681022 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b45771d930 con 0x55b457832aa0 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.702+0000 7f1acdffb640 10 client.17478 handle_command_reply: tid=0 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.702+0000 7f1aef9f7740 1 client.17478 shutdown 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 msgr2=0x55b45781f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b45781f640 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00ab40 tx=0x7f1adc004970 comp rx=0 tx=0).stop 2024-04-22T13:32:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 shutdown_connections 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b457832aa0 0x55b457834e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457819e80 0x55b45781f640 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457822a00 0x55b45781fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1acdffb640 0 client.17478 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/514681022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b4578194d0 0x55b4578222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 >> 172.21.15.5:0/514681022 conn(0x55b457813ba0 msgr2=0x55b4578158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 shutdown_connections 2024-04-22T13:32:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 1 -- 172.21.15.5:0/514681022 wait complete. 2024-04-22T13:32:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.703+0000 7f1aef9f7740 20 client.17478 trim_cache size 0 max 16384 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 Processor -- start 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 -- start start 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b45782e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457819e80 0x55b45782b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 0x55b457821d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ada00 con 0x55b457822a00 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81af540 con 0x55b457819e80 2024-04-22T13:32:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ae0d0 con 0x55b4578194d0 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1ae6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 0x55b457821d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1ae6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 0x55b457821d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58872/0 (socket says 172.21.15.5:58872) 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.716+0000 7f1ae6d9d640 1 -- 172.21.15.5:0/91930492 learned_addr learned my addr 172.21.15.5:0/91930492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae659c640 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b45782e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae6d9d640 1 -- 172.21.15.5:0/91930492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 msgr2=0x55b45782e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b45782e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae6d9d640 1 -- 172.21.15.5:0/91930492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457819e80 msgr2=0x55b45782b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457819e80 0x55b45782b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457819e80 0x55b45782b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae6d9d640 1 -- 172.21.15.5:0/91930492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4578352f0 con 0x55b457822a00 2024-04-22T13:32:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 0x55b457821d70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009ca0 tx=0x7f1ae0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6fe38aafc94b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1acffff640 1 -- 172.21.15.5:0/91930492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0052070 con 0x55b457822a00 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1acffff640 1 -- 172.21.15.5:0/91930492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ae003a050 con 0x55b457822a00 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.717+0000 7f1acffff640 1 -- 172.21.15.5:0/91930492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae004d480 con 0x55b457822a00 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- 172.21.15.5:0/91930492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 msgr2=0x55b457821d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 0x55b457821d70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009ca0 tx=0x7f1ae0047000 comp rx=0 tx=0).stop 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- 172.21.15.5:0/91930492 shutdown_connections 2024-04-22T13:32:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b457822a00 0x55b457821d70 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457819e80 0x55b45782b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/91930492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4578194d0 0x55b45782e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- 172.21.15.5:0/91930492 >> 172.21.15.5:0/91930492 conn(0x55b457813ba0 msgr2=0x55b457812c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- 172.21.15.5:0/91930492 shutdown_connections 2024-04-22T13:32:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- 172.21.15.5:0/91930492 wait complete. 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- start start 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 Processor -- start 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 0x55b457890d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457819e80 0x55b457891290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b4578917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81aee70 con 0x55b4578194d0 2024-04-22T13:32:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ae7a0 con 0x55b457822a00 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ab9e0 con 0x55b457819e80 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1aef9f7740 10 client.0 _ll_register_callbacks cb 0x7f1adc005640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1ae659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 0x55b457890d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1ae659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 0x55b457890d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58878/0 (socket says 172.21.15.5:58878) 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.718+0000 7f1ae659c640 1 -- 172.21.15.5:0/2586297745 learned_addr learned my addr 172.21.15.5:0/2586297745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457819e80 0x55b457891290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b4578917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae659c640 1 -- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457819e80 msgr2=0x55b457891290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae659c640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457819e80 0x55b457891290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae659c640 1 -- 172.21.15.5:0/2586297745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 msgr2=0x55b4578917d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae659c640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b4578917d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae659c640 1 -- 172.21.15.5:0/2586297745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b45789c020 con 0x55b4578194d0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae6d9d640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b4578917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1ae659c640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 0x55b457890d50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8007590 tx=0x7f1ad80d0b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c643e09e0bb6871 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1acdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1acdffb640 1 -- 172.21.15.5:0/2586297745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80a5550 con 0x55b4578194d0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1acdffb640 1 -- 172.21.15.5:0/2586297745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1ad809abe0 con 0x55b4578194d0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae003a050 con 0x55b4578194d0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1acdffb640 1 -- 172.21.15.5:0/2586297745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ad800f990 con 0x55b4578194d0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1aef9f7740 10 client.17484 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b45789d340 con 0x55b4578194d0 2024-04-22T13:32:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1acdffb640 1 -- 172.21.15.5:0/2586297745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ae003a050 con 0x55b4578194d0 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.719+0000 7f1acdffb640 1 -- 172.21.15.5:0/2586297745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1ad80ca9e0 con 0x55b4578194d0 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 10 client.17484 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 10 client.17484 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 20 client.17484 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b45789da10 0x55b45789fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 4 client.17484 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b45778bc10 con 0x55b45789da10 2024-04-22T13:32:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b45789da10 0x55b45789fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1ae5d9b640 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b45789da10 0x55b45789fe00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1805cad86879c660 server_cookie=9e22cad983dfe71 in_seq=0 out_seq=0 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1acdffb640 10 client.17484 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1acdffb640 1 -- 172.21.15.5:0/2586297745 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b45778bc10 con 0x55b45789da10 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1acdffb640 10 client.17484 handle_command_reply: tid=0 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 client.17484 shutdown 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 msgr2=0x55b457890d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 0x55b457890d50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8007590 tx=0x7f1ad80d0b10 comp rx=0 tx=0).stop 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 shutdown_connections 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b457822a00 0x55b4578917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b457819e80 0x55b457891290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55b45789da10 0x55b45789fe00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.720+0000 7f1aef9f7740 1 --2- 172.21.15.5:0/2586297745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4578194d0 0x55b457890d50 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1acdffb640 0 client.17484 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 >> 172.21.15.5:0/2586297745 conn(0x55b457813ba0 msgr2=0x55b45781a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 shutdown_connections 2024-04-22T13:32:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aef9f7740 1 -- 172.21.15.5:0/2586297745 wait complete. 2024-04-22T13:32:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aef9f7740 20 client.17484 trim_cache size 0 max 16384 2024-04-22T13:32:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:08.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1ad00a3d20 msgr2=0x7f1ad00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1ad00a3d20 0x7f1ad00a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0006010 tx=0x7f1ae0032d20 comp rx=0 tx=0).stop 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 msgr2=0x7f1ae819e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 0x7f1ae819e440 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800c920 tx=0x7f1ad800cd20 comp rx=0 tx=0).stop 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 shutdown_connections 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1ad00a3d20 0x7f1ad00a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ae8198640 0x7f1ae8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ae810c6d0 0x7f1ae819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 --2- 172.21.15.5:0/4177985789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae81031c0 0x7f1ae819e440 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 >> 172.21.15.5:0/4177985789 conn(0x7f1ae8077f70 msgr2=0x7f1ae80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 shutdown_connections 2024-04-22T13:32:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.721+0000 7f1aece06640 1 -- 172.21.15.5:0/4177985789 wait complete. 2024-04-22T13:32:08.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:32:08.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 -- 172.21.15.5:0/3444353889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78341031c0 msgr2=0x7f78341035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 --2- 172.21.15.5:0/3444353889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78341031c0 0x7f78341035a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7824009900 tx=0x7f782402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 -- 172.21.15.5:0/3444353889 shutdown_connections 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 --2- 172.21.15.5:0/3444353889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f783410c500 0x7f783410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 --2- 172.21.15.5:0/3444353889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7834103ae0 0x7f783410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 --2- 172.21.15.5:0/3444353889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78341031c0 0x7f78341035a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 -- 172.21.15.5:0/3444353889 >> 172.21.15.5:0/3444353889 conn(0x7f7834078000 msgr2=0x7f7834078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 -- 172.21.15.5:0/3444353889 shutdown_connections 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.850+0000 7f783a043640 1 -- 172.21.15.5:0/3444353889 wait complete. 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 Processor -- start 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 -- start start 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78341031c0 0x7f7834198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7834103ae0 0x7f7834198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 0x7f783419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7834112170 con 0x7f783410c500 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7834111ff0 con 0x7f78341031c0 2024-04-22T13:32:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f783a043640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78341122f0 con 0x7f7834103ae0 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f7833fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 0x7f783419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f7833fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 0x7f783419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58196/0 (socket says 172.21.15.5:58196) 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.851+0000 7f7833fff640 1 -- 172.21.15.5:0/3251482750 learned_addr learned my addr 172.21.15.5:0/3251482750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7832ffd640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7834103ae0 0x7f7834198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f78337fe640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78341031c0 0x7f7834198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7833fff640 1 -- 172.21.15.5:0/3251482750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7834103ae0 msgr2=0x7f7834198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7833fff640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7834103ae0 0x7f7834198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7833fff640 1 -- 172.21.15.5:0/3251482750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78341031c0 msgr2=0x7f7834198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7833fff640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78341031c0 0x7f7834198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7833fff640 1 -- 172.21.15.5:0/3251482750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783419d680 con 0x7f783410c500 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7832ffd640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7834103ae0 0x7f7834198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7833fff640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 0x7f783419cf50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f782c00ea60 tx=0x7f782c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7830ff9640 1 -- 172.21.15.5:0/3251482750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c00ce40 con 0x7f783410c500 2024-04-22T13:32:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783419d970 con 0x7f783410c500 2024-04-22T13:32:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7830ff9640 1 -- 172.21.15.5:0/3251482750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782c005020 con 0x7f783410c500 2024-04-22T13:32:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f7830ff9640 1 -- 172.21.15.5:0/3251482750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c010670 con 0x7f783410c500 2024-04-22T13:32:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.852+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78341a5200 con 0x7f783410c500 2024-04-22T13:32:08.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.854+0000 7f7830ff9640 1 -- 172.21.15.5:0/3251482750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f782c0054b0 con 0x7f783410c500 2024-04-22T13:32:08.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.854+0000 7f783cc33740 1 Processor -- start 2024-04-22T13:32:08.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 -- start start 2024-04-22T13:32:08.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559defea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559def4f1f0 con 0x5559defead60 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5559deeec4b0 con 0x5559deff3e60 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f783cc33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5559def4b1b0 con 0x5559defea3b0 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58204/0 (socket says 172.21.15.5:58204) 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 -- 172.21.15.5:0/2880175974 learned_addr learned my addr 172.21.15.5:0/2880175974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559defea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f7830ff9640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f781c0a3d20 0x7f781c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 -- 172.21.15.5:0/2880175974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 msgr2=0x5559defea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559defea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 -- 172.21.15.5:0/2880175974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 msgr2=0x5559deff42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f78337fe640 1 -- 172.21.15.5:0/2880175974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559deede1b0 con 0x5559defead60 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f7833fff640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f781c0a3d20 0x7f781c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.855+0000 7f7830ff9640 1 -- 172.21.15.5:0/3251482750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f782c0d0d10 con 0x7f783410c500 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f78337fe640 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff3890 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7824009900 tx=0x7f7824005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc3d06bfa70ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f7819ffb640 1 -- 172.21.15.5:0/2880175974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824004240 con 0x5559defead60 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f7819ffb640 1 -- 172.21.15.5:0/2880175974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78240043e0 con 0x5559defead60 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f7819ffb640 1 -- 172.21.15.5:0/2880175974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240046c0 con 0x5559defead60 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f7833fff640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f781c0a3d20 0x7f781c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f782c0ce540 tx=0x7f782c06b690 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 -- 172.21.15.5:0/2880175974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 msgr2=0x5559deff3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff3890 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7824009900 tx=0x7f7824005e60 comp rx=0 tx=0).stop 2024-04-22T13:32:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 -- 172.21.15.5:0/2880175974 shutdown_connections 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff3890 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 --2- 172.21.15.5:0/2880175974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559defea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 -- 172.21.15.5:0/2880175974 >> 172.21.15.5:0/2880175974 conn(0x5559defe4ba0 msgr2=0x5559defe4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 -- 172.21.15.5:0/2880175974 shutdown_connections 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.856+0000 7f783cc33740 1 -- 172.21.15.5:0/2880175974 wait complete. 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 -- start start 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 Processor -- start 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559deffa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559deffad40 con 0x5559defead60 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5559def08500 con 0x5559deff3e60 2024-04-22T13:32:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5559deffaec0 con 0x5559defea3b0 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f783cc33740 10 client.0 _ll_register_callbacks cb 0x5559deed6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f7832ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559deffa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f7832ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559deffa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:36172/0 (socket says 172.21.15.5:36172) 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58220/0 (socket says 172.21.15.5:58220) 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.857+0000 7f7832ffd640 1 -- 172.21.15.5:0/3748217494 learned_addr learned my addr 172.21.15.5:0/3748217494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 -- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 msgr2=0x5559deff7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 -- 172.21.15.5:0/3748217494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 msgr2=0x5559deff7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 -- 172.21.15.5:0/3748217494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559dec613f0 con 0x5559defea3b0 2024-04-22T13:32:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7832ffd640 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559deffa200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7828012ed0 tx=0x7f78280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3b6968241cfc34a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7812ffd640 1 -- 172.21.15.5:0/3748217494 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782800ee00 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78240043e0 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7812ffd640 1 -- 172.21.15.5:0/3748217494 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f7828004260 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7812ffd640 1 -- 172.21.15.5:0/3748217494 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7828026050 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f783cc33740 10 client.26899 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559ded2b7d0 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.858+0000 7f7812ffd640 1 -- 172.21.15.5:0/3748217494 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78240043e0 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f7812ffd640 1 -- 172.21.15.5:0/3748217494 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7828010460 con 0x5559defea3b0 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f783cc33740 10 client.26899 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f783cc33740 10 client.26899 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f783cc33740 20 client.26899 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f783cc33740 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df007e90 0x5559df00a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f783cc33740 4 client.26899 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559deeee930 con 0x5559df007e90 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.859+0000 7f78337fe640 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df007e90 0x5559df00a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f78337fe640 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df007e90 0x5559df00a2c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e9ea82a41ea7717b server_cookie=1e548a2e0726a11 in_seq=0 out_seq=0 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f7812ffd640 10 client.26899 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f7812ffd640 1 -- 172.21.15.5:0/3748217494 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559deeee930 con 0x5559df007e90 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f7812ffd640 10 client.26899 handle_command_reply: tid=0 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 client.26899 shutdown 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 msgr2=0x5559deffa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559deffa200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7828012ed0 tx=0x7f78280046c0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 shutdown_connections 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df007e90 0x5559df00a2c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 --2- 172.21.15.5:0/3748217494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559deffa200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f7812ffd640 0 client.26899 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:08.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.860+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 >> 172.21.15.5:0/3748217494 conn(0x5559defe4ba0 msgr2=0x5559defe68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.861+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 shutdown_connections 2024-04-22T13:32:08.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.861+0000 7f783cc33740 1 -- 172.21.15.5:0/3748217494 wait complete. 2024-04-22T13:32:08.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.861+0000 7f783cc33740 20 client.26899 trim_cache size 0 max 16384 2024-04-22T13:32:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 Processor -- start 2024-04-22T13:32:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 -- start start 2024-04-22T13:32:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559df003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ab930 con 0x5559defead60 2024-04-22T13:32:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ad470 con 0x5559deff3e60 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f783cc33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ac000 con 0x5559defea3b0 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58226/0 (socket says 172.21.15.5:58226) 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.874+0000 7f78337fe640 1 -- 172.21.15.5:0/2091795464 learned_addr learned my addr 172.21.15.5:0/2091795464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559df003c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78337fe640 1 -- 172.21.15.5:0/2091795464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 msgr2=0x5559df003c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78337fe640 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559df003c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78337fe640 1 -- 172.21.15.5:0/2091795464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 msgr2=0x5559deff6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78337fe640 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78337fe640 1 -- 172.21.15.5:0/2091795464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559deff3be0 con 0x5559defead60 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78337fe640 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff2800 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f782402cc60 tx=0x7f782402cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffab3ec888a4d97e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78197fa640 1 -- 172.21.15.5:0/2091795464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824052080 con 0x5559defead60 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78197fa640 1 -- 172.21.15.5:0/2091795464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78240526d0 con 0x5559defead60 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f78197fa640 1 -- 172.21.15.5:0/2091795464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824034960 con 0x5559defead60 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f783cc33740 1 -- 172.21.15.5:0/2091795464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 msgr2=0x5559deff2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.875+0000 7f783cc33740 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff2800 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f782402cc60 tx=0x7f782402cc90 comp rx=0 tx=0).stop 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- 172.21.15.5:0/2091795464 shutdown_connections 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff2800 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559deff3e60 0x5559deff6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 --2- 172.21.15.5:0/2091795464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559defea3b0 0x5559df003c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- 172.21.15.5:0/2091795464 >> 172.21.15.5:0/2091795464 conn(0x5559defe4ba0 msgr2=0x5559defe5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- 172.21.15.5:0/2091795464 shutdown_connections 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- 172.21.15.5:0/2091795464 wait complete. 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- start start 2024-04-22T13:32:08.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 Processor -- start 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559defea3b0 0x5559df000370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559deff3e60 0x5559deff9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78341acda0 con 0x5559defead60 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ac6d0 con 0x5559defea3b0 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78341a9910 con 0x5559deff3e60 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f783cc33740 10 client.0 _ll_register_callbacks cb 0x7f78280127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.876+0000 7f78337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58242/0 (socket says 172.21.15.5:58242) 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 -- 172.21.15.5:0/2068788083 learned_addr learned my addr 172.21.15.5:0/2068788083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559defea3b0 0x5559df000370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559deff3e60 0x5559deff9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 -- 172.21.15.5:0/2068788083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559deff3e60 msgr2=0x5559deff9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559deff3e60 0x5559deff9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 -- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559defea3b0 msgr2=0x5559df000370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559defea3b0 0x5559df000370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 -- 172.21.15.5:0/2068788083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559deff96c0 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f78337fe640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff8c40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f78240059d0 tx=0x7f782402cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8171110c46a100f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7812ffd640 1 -- 172.21.15.5:0/2068788083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824004030 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7812ffd640 1 -- 172.21.15.5:0/2068788083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f78240041d0 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78240526d0 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7812ffd640 1 -- 172.21.15.5:0/2068788083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78240043d0 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f783cc33740 10 client.17502 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559df06e090 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.877+0000 7f7812ffd640 1 -- 172.21.15.5:0/2068788083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78240526d0 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f7812ffd640 1 -- 172.21.15.5:0/2068788083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7824034b80 con 0x5559defead60 2024-04-22T13:32:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f783cc33740 10 client.17502 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f783cc33740 10 client.17502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f783cc33740 20 client.17502 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f783cc33740 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df06e360 0x5559df070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f783cc33740 4 client.17502 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559def5cc10 con 0x5559df06e360 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df06e360 0x5559df070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f7812ffd640 10 client.17502 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.878+0000 7f7832ffd640 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df06e360 0x5559df070750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=edde1f886c5bbd16 server_cookie=b61bc4fcd9d7301f in_seq=0 out_seq=0 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f7812ffd640 1 -- 172.21.15.5:0/2068788083 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559def5cc10 con 0x5559df06e360 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f7812ffd640 10 client.17502 handle_command_reply: tid=0 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 client.17502 shutdown 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 msgr2=0x5559deff8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff8c40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f78240059d0 tx=0x7f782402cd00 comp rx=0 tx=0).stop 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 shutdown_connections 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5559df06e360 0x5559df070750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559defead60 0x5559deff8c40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f7812ffd640 0 client.17502 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559deff3e60 0x5559deff9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 --2- 172.21.15.5:0/2068788083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5559defea3b0 0x5559df000370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 >> 172.21.15.5:0/2068788083 conn(0x5559defe4ba0 msgr2=0x5559defe3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 shutdown_connections 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 1 -- 172.21.15.5:0/2068788083 wait complete. 2024-04-22T13:32:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783cc33740 20 client.17502 trim_cache size 0 max 16384 2024-04-22T13:32:08.884 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f781c0a3d20 msgr2=0x7f781c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f781c0a3d20 0x7f781c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f782c0ce540 tx=0x7f782c06b690 comp rx=0 tx=0).stop 2024-04-22T13:32:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 msgr2=0x7f783419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 0x7f783419cf50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f782c00ea60 tx=0x7f782c00ef20 comp rx=0 tx=0).stop 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 shutdown_connections 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f783410c500 0x7f783419cf50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f781c0a3d20 0x7f781c0a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7834103ae0 0x7f7834198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 --2- 172.21.15.5:0/3251482750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78341031c0 0x7f7834198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.879+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 >> 172.21.15.5:0/3251482750 conn(0x7f7834078000 msgr2=0x7f78340fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.880+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 shutdown_connections 2024-04-22T13:32:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.880+0000 7f783a043640 1 -- 172.21.15.5:0/3251482750 wait complete. 2024-04-22T13:32:08.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:32:08.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 -- 172.21.15.5:0/1138396395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8410c6d0 msgr2=0x7f5c8410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 --2- 172.21.15.5:0/1138396395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8410c6d0 0x7f5c8410eb60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009ca0 tx=0x7f5c7c02c120 comp rx=0 tx=0).stop 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 -- 172.21.15.5:0/1138396395 shutdown_connections 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 --2- 172.21.15.5:0/1138396395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8410c6d0 0x7f5c8410eb60 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 --2- 172.21.15.5:0/1138396395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c84103b70 0x7f5c8410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 --2- 172.21.15.5:0/1138396395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c841031c0 0x7f5c841035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 -- 172.21.15.5:0/1138396395 >> 172.21.15.5:0/1138396395 conn(0x7f5c84077f70 msgr2=0x7f5c84078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 -- 172.21.15.5:0/1138396395 shutdown_connections 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.995+0000 7f5c88a55640 1 -- 172.21.15.5:0/1138396395 wait complete. 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 Processor -- start 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 -- start start 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c841031c0 0x7f5c8419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c8410c6d0 0x7f5c8419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 0x7f5c84198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c84112450 con 0x7f5c841987b0 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841122d0 con 0x7f5c8410c6d0 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.996+0000 7f5c88a55640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841125d0 con 0x7f5c841031c0 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 0x7f5c84198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 0x7f5c84198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58262/0 (socket says 172.21.15.5:58262) 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 -- 172.21.15.5:0/2735005306 learned_addr learned my addr 172.21.15.5:0/2735005306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c8410c6d0 0x7f5c8419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c8259c640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c841031c0 0x7f5c8419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 -- 172.21.15.5:0/2735005306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c841031c0 msgr2=0x7f5c8419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c841031c0 0x7f5c8419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 -- 172.21.15.5:0/2735005306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c8410c6d0 msgr2=0x7f5c8419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c8410c6d0 0x7f5c8419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 -- 172.21.15.5:0/2735005306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c84199340 con 0x7f5c841987b0 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c8410c6d0 0x7f5c8419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c8259c640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c841031c0 0x7f5c8419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 0x7f5c84198c60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c037040 tx=0x7f5c7c035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c737fe640 1 -- 172.21.15.5:0/2735005306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c039040 con 0x7f5c841987b0 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c841995d0 con 0x7f5c841987b0 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c737fe640 1 -- 172.21.15.5:0/2735005306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c7c044070 con 0x7f5c841987b0 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c737fe640 1 -- 172.21.15.5:0/2735005306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004440 con 0x7f5c841987b0 2024-04-22T13:32:08.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.997+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c841a5760 con 0x7f5c841987b0 2024-04-22T13:32:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:08.999+0000 7f5c737fe640 1 -- 172.21.15.5:0/2735005306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5c7c032780 con 0x7f5c841987b0 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 Processor -- start 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 -- start start 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2efe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c737fe640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c6c0a3d20 0x7f5c6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efed60 0x5631a2f07890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 0x5631a2f082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5631a2e631f0 con 0x5631a2f07e60 2024-04-22T13:32:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5631a2e004b0 con 0x5631a2efe3b0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8b648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631a2e5f1b0 con 0x5631a2efed60 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 0x5631a2f082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c8259c640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c6c0a3d20 0x7f5c6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 0x5631a2f082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58278/0 (socket says 172.21.15.5:58278) 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c82d9d640 1 -- 172.21.15.5:0/3233484764 learned_addr learned my addr 172.21.15.5:0/3233484764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.000+0000 7f5c737fe640 1 -- 172.21.15.5:0/2735005306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5c7c0f2fd0 con 0x7f5c841987b0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2efe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efed60 0x5631a2f07890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c82d9d640 1 -- 172.21.15.5:0/3233484764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efed60 msgr2=0x5631a2f07890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efed60 0x5631a2f07890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c82d9d640 1 -- 172.21.15.5:0/3233484764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 msgr2=0x5631a2efe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2efe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c82d9d640 1 -- 172.21.15.5:0/3233484764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a2df21b0 con 0x5631a2f07e60 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c8259c640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c6c0a3d20 0x7f5c6c0a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006fd0 tx=0x7f5c74008040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 0x5631a2f082c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0bba20 tx=0x7f5c7c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dc4d64c14a7f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c70ff9640 1 -- 172.21.15.5:0/3233484764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0c8b80 con 0x5631a2f07e60 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c70ff9640 1 -- 172.21.15.5:0/3233484764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c7c0bf050 con 0x5631a2f07e60 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.001+0000 7f5c70ff9640 1 -- 172.21.15.5:0/3233484764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c09b520 con 0x5631a2f07e60 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 -- 172.21.15.5:0/3233484764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 msgr2=0x5631a2f082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 0x5631a2f082c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0bba20 tx=0x7f5c7c003f90 comp rx=0 tx=0).stop 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 -- 172.21.15.5:0/3233484764 shutdown_connections 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2f07e60 0x5631a2f082c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efed60 0x5631a2f07890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 --2- 172.21.15.5:0/3233484764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2efe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 -- 172.21.15.5:0/3233484764 >> 172.21.15.5:0/3233484764 conn(0x5631a2ef8ba0 msgr2=0x5631a2ef8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 -- 172.21.15.5:0/3233484764 shutdown_connections 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 -- 172.21.15.5:0/3233484764 wait complete. 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 -- start start 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.002+0000 7f5c8b648740 1 Processor -- start 2024-04-22T13:32:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 0x5631a2f0e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efed60 0x5631a2f0b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5631a2f0eb60 con 0x5631a2efe3b0 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5631a2e1c500 con 0x5631a2efed60 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631a2f0ece0 con 0x5631a2f07e60 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 10 client.0 _ll_register_callbacks cb 0x5631a2deab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 0x5631a2f0e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 0x5631a2f0e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58292/0 (socket says 172.21.15.5:58292) 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 -- 172.21.15.5:0/867025764 learned_addr learned my addr 172.21.15.5:0/867025764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efed60 0x5631a2f0b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 -- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 msgr2=0x5631a2f0bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 -- 172.21.15.5:0/867025764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efed60 msgr2=0x5631a2f0b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efed60 0x5631a2f0b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 -- 172.21.15.5:0/867025764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a2b753f0 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efed60 0x5631a2f0b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8259c640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 0x5631a2f0e200 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400ced0 tx=0x7f5c740047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98711f7073cc0fee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c69ffb640 1 -- 172.21.15.5:0/867025764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74023060 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c69ffb640 1 -- 172.21.15.5:0/867025764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5c740117e0 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7c0bf050 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.003+0000 7f5c69ffb640 1 -- 172.21.15.5:0/867025764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c74011a20 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 10 client.17514 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631a2c3f7d0 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c69ffb640 1 -- 172.21.15.5:0/867025764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c7c0bf050 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c69ffb640 1 -- 172.21.15.5:0/867025764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5c7401da50 con 0x5631a2efe3b0 2024-04-22T13:32:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 10 client.17514 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 20 client.17514 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f1bcb0 0x5631a2f1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 4 client.17514 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a2e02930 con 0x5631a2f1bcb0 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.004+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f1bcb0 0x5631a2f1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.005+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f1bcb0 0x5631a2f1e150 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c500ec786b6b7a3 server_cookie=232e956401ea024b in_seq=0 out_seq=0 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.005+0000 7f5c69ffb640 10 client.17514 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.005+0000 7f5c69ffb640 1 -- 172.21.15.5:0/867025764 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631a2e02930 con 0x5631a2f1bcb0 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.005+0000 7f5c69ffb640 10 client.17514 handle_command_reply: tid=0 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.005+0000 7f5c8b648740 1 client.17514 shutdown 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 msgr2=0x5631a2f0e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 0x5631a2f0e200 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400ced0 tx=0x7f5c740047d0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 shutdown_connections 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f1bcb0 0x5631a2f1e150 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efed60 0x5631a2f0b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 --2- 172.21.15.5:0/867025764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efe3b0 0x5631a2f0e200 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c69ffb640 0 client.17514 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 >> 172.21.15.5:0/867025764 conn(0x5631a2ef8ba0 msgr2=0x5631a2efa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 shutdown_connections 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 1 -- 172.21.15.5:0/867025764 wait complete. 2024-04-22T13:32:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.006+0000 7f5c8b648740 20 client.17514 trim_cache size 0 max 16384 2024-04-22T13:32:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 Processor -- start 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 -- start start 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2f17990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841aba30 con 0x5631a2efed60 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ad570 con 0x5631a2efe3b0 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8b648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ac100 con 0x5631a2f07e60 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58294/0 (socket says 172.21.15.5:58294) 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c81d9b640 1 -- 172.21.15.5:0/1056130046 learned_addr learned my addr 172.21.15.5:0/1056130046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c8259c640 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2f17990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c82d9d640 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c81d9b640 1 -- 172.21.15.5:0/1056130046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 msgr2=0x5631a2f0e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.019+0000 7f5c81d9b640 1 -- 172.21.15.5:0/1056130046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 msgr2=0x5631a2f17990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2f17990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c81d9b640 1 -- 172.21.15.5:0/1056130046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a2ef4470 con 0x5631a2efed60 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8259c640 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2f17990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0dcc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800aec0 tx=0x7f5c7800a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369f736762260dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c6bfff640 1 -- 172.21.15.5:0/1056130046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78018960 con 0x5631a2efed60 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c6bfff640 1 -- 172.21.15.5:0/1056130046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c7801e250 con 0x5631a2efed60 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c6bfff640 1 -- 172.21.15.5:0/1056130046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780165d0 con 0x5631a2efed60 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 -- 172.21.15.5:0/1056130046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 msgr2=0x5631a2f0dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0dcc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800aec0 tx=0x7f5c7800a2b0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 -- 172.21.15.5:0/1056130046 shutdown_connections 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0dcc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2f07e60 0x5631a2f0e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 --2- 172.21.15.5:0/1056130046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2efe3b0 0x5631a2f17990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 -- 172.21.15.5:0/1056130046 >> 172.21.15.5:0/1056130046 conn(0x5631a2ef8ba0 msgr2=0x5631a2f075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 -- 172.21.15.5:0/1056130046 shutdown_connections 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.020+0000 7f5c8b648740 1 -- 172.21.15.5:0/1056130046 wait complete. 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 -- start start 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 Processor -- start 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efe3b0 0x5631a2f0cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2f07e60 0x5631a2f12d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841acea0 con 0x5631a2efed60 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ac7d0 con 0x5631a2f07e60 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841a9a30 con 0x5631a2efe3b0 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8b648740 10 client.0 _ll_register_callbacks cb 0x7f5c7400c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58300/0 (socket says 172.21.15.5:58300) 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 -- 172.21.15.5:0/330827021 learned_addr learned my addr 172.21.15.5:0/330827021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c8259c640 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efe3b0 0x5631a2f0cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 -- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efe3b0 msgr2=0x5631a2f0cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efe3b0 0x5631a2f0cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 -- 172.21.15.5:0/330827021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2f07e60 msgr2=0x5631a2f12d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2f07e60 0x5631a2f12d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.021+0000 7f5c81d9b640 1 -- 172.21.15.5:0/330827021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a2f13380 con 0x5631a2efed60 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c81d9b640 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0d450 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5c78021040 tx=0x7f5c7801e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e341f4b2714b8cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c69ffb640 1 -- 172.21.15.5:0/330827021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7802b020 con 0x5631a2efed60 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c69ffb640 1 -- 172.21.15.5:0/330827021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5c78018960 con 0x5631a2efed60 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7801e250 con 0x5631a2efed60 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c69ffb640 1 -- 172.21.15.5:0/330827021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c78036050 con 0x5631a2efed60 2024-04-22T13:32:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 10 client.17520 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631a2f14640 con 0x5631a2efed60 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c69ffb640 1 -- 172.21.15.5:0/330827021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c7801e250 con 0x5631a2efed60 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c69ffb640 1 -- 172.21.15.5:0/330827021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5c78015040 con 0x5631a2efed60 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 10 client.17520 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 10 client.17520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 20 client.17520 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f826b0 0x5631a2f84ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 4 client.17520 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.022+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631a2e70c10 con 0x5631a2f826b0 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8259c640 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f826b0 0x5631a2f84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8259c640 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f826b0 0x5631a2f84ad0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28213080156a3176 server_cookie=a4a4eabf7bce8fee in_seq=0 out_seq=0 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c69ffb640 10 client.17520 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c69ffb640 1 -- 172.21.15.5:0/330827021 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631a2e70c10 con 0x5631a2f826b0 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c69ffb640 10 client.17520 handle_command_reply: tid=0 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 client.17520 shutdown 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 msgr2=0x5631a2f0d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0d450 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5c78021040 tx=0x7f5c7801e480 comp rx=0 tx=0).stop 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 shutdown_connections 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5631a2f826b0 0x5631a2f84ad0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631a2efed60 0x5631a2f0d450 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5631a2f07e60 0x5631a2f12d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 --2- 172.21.15.5:0/330827021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631a2efe3b0 0x5631a2f0cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 >> 172.21.15.5:0/330827021 conn(0x5631a2ef8ba0 msgr2=0x5631a2ef7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c69ffb640 0 client.17520 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 shutdown_connections 2024-04-22T13:32:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.023+0000 7f5c8b648740 1 -- 172.21.15.5:0/330827021 wait complete. 2024-04-22T13:32:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c8b648740 20 client.17520 trim_cache size 0 max 16384 2024-04-22T13:32:09.026 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c6c0a3d20 msgr2=0x7f5c6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c6c0a3d20 0x7f5c6c0a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006fd0 tx=0x7f5c74008040 comp rx=0 tx=0).stop 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 msgr2=0x7f5c84198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 0x7f5c84198c60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c037040 tx=0x7f5c7c035ce0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 shutdown_connections 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c841987b0 0x7f5c84198c60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5c6c0a3d20 0x7f5c6c0a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c8410c6d0 0x7f5c8419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 --2- 172.21.15.5:0/2735005306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c841031c0 0x7f5c8419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 >> 172.21.15.5:0/2735005306 conn(0x7f5c84077f70 msgr2=0x7f5c840fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 shutdown_connections 2024-04-22T13:32:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.024+0000 7f5c88a55640 1 -- 172.21.15.5:0/2735005306 wait complete. 2024-04-22T13:32:09.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 -- 172.21.15.5:0/563065457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101a40 msgr2=0x7f2bd4105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 --2- 172.21.15.5:0/563065457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101a40 0x7f2bd4105a80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009900 tx=0x7f2bc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 -- 172.21.15.5:0/563065457 shutdown_connections 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 --2- 172.21.15.5:0/563065457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101a40 0x7f2bd4105a80 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 --2- 172.21.15.5:0/563065457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd4106160 0x7f2bd410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 --2- 172.21.15.5:0/563065457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bd4101090 0x7f2bd4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 -- 172.21.15.5:0/563065457 >> 172.21.15.5:0/563065457 conn(0x7f2bd4078000 msgr2=0x7f2bd4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 -- 172.21.15.5:0/563065457 shutdown_connections 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.139+0000 7f2bdb277640 1 -- 172.21.15.5:0/563065457 wait complete. 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 Processor -- start 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 -- start start 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 0x7f2bd410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bd4106160 0x7f2bd410ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd410f140 0x7f2bd41a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41121e0 con 0x7f2bd4101090 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4112060 con 0x7f2bd4106160 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bdb277640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4112360 con 0x7f2bd410f140 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 0x7f2bd410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 0x7f2bd410e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58330/0 (socket says 172.21.15.5:58330) 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 -- 172.21.15.5:0/3215481830 learned_addr learned my addr 172.21.15.5:0/3215481830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9814640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd410f140 0x7f2bd41a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd8812640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bd4106160 0x7f2bd410ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 -- 172.21.15.5:0/3215481830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd410f140 msgr2=0x7f2bd41a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd410f140 0x7f2bd41a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 -- 172.21.15.5:0/3215481830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bd4106160 msgr2=0x7f2bd410ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bd4106160 0x7f2bd410ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.140+0000 7f2bd9013640 1 -- 172.21.15.5:0/3215481830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd41a6f70 con 0x7f2bd4101090 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bd9814640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd410f140 0x7f2bd41a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bd9013640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 0x7f2bd410e6c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c920 tx=0x7f2bd000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bca7fc640 1 -- 172.21.15.5:0/3215481830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007c70 con 0x7f2bd4101090 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bca7fc640 1 -- 172.21.15.5:0/3215481830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd0007e10 con 0x7f2bd4101090 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd41a7260 con 0x7f2bd4101090 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bca7fc640 1 -- 172.21.15.5:0/3215481830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0018820 con 0x7f2bd4101090 2024-04-22T13:32:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.141+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd41a7780 con 0x7f2bd4101090 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.143+0000 7f2bca7fc640 1 -- 172.21.15.5:0/3215481830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2bd000f450 con 0x7f2bd4101090 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 Processor -- start 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 -- start start 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f420724d0 0x560f420728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 0x560f4207de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560f41fd71f0 con 0x560f4207ba00 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f41f744b0 con 0x560f420724d0 2024-04-22T13:32:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bdde62740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f41fd31b0 con 0x560f42072e80 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bca7fc640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2bc00a3d50 0x7f2bc00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 0x560f4207de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd9013640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 0x560f4207de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58346/0 (socket says 172.21.15.5:58346) 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 -- 172.21.15.5:0/494413546 learned_addr learned my addr 172.21.15.5:0/494413546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd9814640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2bc00a3d50 0x7f2bc00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd9814640 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f420724d0 0x560f420728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bca7fc640 1 -- 172.21.15.5:0/3215481830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2bd00d0980 con 0x7f2bd4101090 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 -- 172.21.15.5:0/494413546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 msgr2=0x560f4207b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 -- 172.21.15.5:0/494413546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f420724d0 msgr2=0x560f420728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f420724d0 0x560f420728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd8812640 1 -- 172.21.15.5:0/494413546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f41f661b0 con 0x560f4207ba00 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.144+0000 7f2bd9814640 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f420724d0 0x560f420728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bd8812640 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 0x560f4207de50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2bd406a320 tx=0x7f2bc402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efddcc45d5bc656 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bd9814640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2bc00a3d50 0x7f2bc00a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005c40 tx=0x7f2bcc0095e0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bbf7fe640 1 -- 172.21.15.5:0/494413546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4044020 con 0x560f4207ba00 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bbf7fe640 1 -- 172.21.15.5:0/494413546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc402cc90 con 0x560f4207ba00 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bbf7fe640 1 -- 172.21.15.5:0/494413546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40046c0 con 0x560f4207ba00 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bdde62740 1 -- 172.21.15.5:0/494413546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 msgr2=0x560f4207de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.145+0000 7f2bdde62740 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 0x560f4207de50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2bd406a320 tx=0x7f2bc402c650 comp rx=0 tx=0).stop 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- 172.21.15.5:0/494413546 shutdown_connections 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f4207ba00 0x560f4207de50 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 --2- 172.21.15.5:0/494413546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f420724d0 0x560f420728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- 172.21.15.5:0/494413546 >> 172.21.15.5:0/494413546 conn(0x560f4206cba0 msgr2=0x560f4206cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- 172.21.15.5:0/494413546 shutdown_connections 2024-04-22T13:32:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- 172.21.15.5:0/494413546 wait complete. 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- start start 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 Processor -- start 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f42078620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42078b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560f4207e910 con 0x560f42072e80 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f41f90500 con 0x560f4207ba00 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.146+0000 7f2bdde62740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f4207ea90 con 0x560f420724d0 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bdde62740 10 client.0 _ll_register_callbacks cb 0x560f41f5eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f42078620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f42078620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58352/0 (socket says 172.21.15.5:58352) 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 -- 172.21.15.5:0/4176624350 learned_addr learned my addr 172.21.15.5:0/4176624350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd9013640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd9814640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42078b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 -- 172.21.15.5:0/4176624350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 msgr2=0x560f4207b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 -- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 msgr2=0x560f42078b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42078b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 -- 172.21.15.5:0/4176624350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f41ce93f0 con 0x560f42072e80 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bd8812640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f42078620 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005720 tx=0x7f2bc402cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9016f035638eac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bbcff9640 1 -- 172.21.15.5:0/4176624350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4044030 con 0x560f42072e80 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bbcff9640 1 -- 172.21.15.5:0/4176624350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2bc40037f0 con 0x560f42072e80 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc402cc90 con 0x560f42072e80 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bbcff9640 1 -- 172.21.15.5:0/4176624350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bc4057050 con 0x560f42072e80 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bdde62740 10 client.17532 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.147+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f41db37d0 con 0x560f42072e80 2024-04-22T13:32:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bbcff9640 1 -- 172.21.15.5:0/4176624350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc402cc90 con 0x560f42072e80 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bbcff9640 1 -- 172.21.15.5:0/4176624350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2bc4040b70 con 0x560f42072e80 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bdde62740 10 client.17532 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bdde62740 10 client.17532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bdde62740 20 client.17532 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bdde62740 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f4208b9b0 0x560f4208dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bdde62740 4 client.17532 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f41f76930 con 0x560f4208b9b0 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.148+0000 7f2bd9013640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f4208b9b0 0x560f4208dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.149+0000 7f2bd9013640 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f4208b9b0 0x560f4208dda0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9812193138220457 server_cookie=7818d4026bac53a8 in_seq=0 out_seq=0 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.149+0000 7f2bbcff9640 10 client.17532 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.149+0000 7f2bbcff9640 1 -- 172.21.15.5:0/4176624350 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f41f76930 con 0x560f4208b9b0 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.149+0000 7f2bbcff9640 10 client.17532 handle_command_reply: tid=0 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.149+0000 7f2bdde62740 1 client.17532 shutdown 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 msgr2=0x560f42078620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f42078620 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005720 tx=0x7f2bc402cbe0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 shutdown_connections 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f4208b9b0 0x560f4208dda0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f42078620 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42078b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bbcff9640 0 client.17532 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 --2- 172.21.15.5:0/4176624350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 >> 172.21.15.5:0/4176624350 conn(0x560f4206cba0 msgr2=0x560f4206e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 shutdown_connections 2024-04-22T13:32:09.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 1 -- 172.21.15.5:0/4176624350 wait complete. 2024-04-22T13:32:09.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.150+0000 7f2bdde62740 20 client.17532 trim_cache size 0 max 16384 2024-04-22T13:32:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 Processor -- start 2024-04-22T13:32:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 -- start start 2024-04-22T13:32:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 0x560f420877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f4207b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41afbd0 con 0x560f420724d0 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b1710 con 0x560f4207ba00 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bdde62740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b02a0 con 0x560f42072e80 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bd9013640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 0x560f420877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bd9013640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 0x560f420877d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58364/0 (socket says 172.21.15.5:58364) 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bd9013640 1 -- 172.21.15.5:0/1389314415 learned_addr learned my addr 172.21.15.5:0/1389314415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.163+0000 7f2bd8812640 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9013640 1 -- 172.21.15.5:0/1389314415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 msgr2=0x560f4207ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9013640 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9814640 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f4207b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9013640 1 -- 172.21.15.5:0/1389314415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 msgr2=0x560f4207b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9013640 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f4207b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9013640 1 -- 172.21.15.5:0/1389314415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f42078ec0 con 0x560f420724d0 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd8812640 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bd9013640 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 0x560f420877d0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2bd009c8e0 tx=0x7f2bd009c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867d24c1e9fe3b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bbeffd640 1 -- 172.21.15.5:0/1389314415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00a0d00 con 0x560f420724d0 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bbeffd640 1 -- 172.21.15.5:0/1389314415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd00a55c0 con 0x560f420724d0 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bbeffd640 1 -- 172.21.15.5:0/1389314415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0098fc0 con 0x560f420724d0 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 -- 172.21.15.5:0/1389314415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 msgr2=0x560f420877d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 0x560f420877d0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2bd009c8e0 tx=0x7f2bd009c850 comp rx=0 tx=0).stop 2024-04-22T13:32:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 -- 172.21.15.5:0/1389314415 shutdown_connections 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f4207b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f42072e80 0x560f4207ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 --2- 172.21.15.5:0/1389314415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f420724d0 0x560f420877d0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.164+0000 7f2bdde62740 1 -- 172.21.15.5:0/1389314415 >> 172.21.15.5:0/1389314415 conn(0x560f4206cba0 msgr2=0x560f4206bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 -- 172.21.15.5:0/1389314415 shutdown_connections 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 -- 172.21.15.5:0/1389314415 wait complete. 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 -- start start 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 Processor -- start 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f4207fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42080160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b1040 con 0x560f42072e80 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b0970 con 0x560f4207ba00 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41adbb0 con 0x560f420724d0 2024-04-22T13:32:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bdde62740 10 client.0 _ll_register_callbacks cb 0x7f2bc402cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bd8812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f4207fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bd8812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f4207fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58366/0 (socket says 172.21.15.5:58366) 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bd8812640 1 -- 172.21.15.5:0/855240731 learned_addr learned my addr 172.21.15.5:0/855240731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.165+0000 7f2bd9814640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42080160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd9013640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd8812640 1 -- 172.21.15.5:0/855240731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 msgr2=0x560f4207f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd8812640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd8812640 1 -- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 msgr2=0x560f42080160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd8812640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42080160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd8812640 1 -- 172.21.15.5:0/855240731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f420f0c40 con 0x560f42072e80 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd9013640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bd8812640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f4207fc20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4004650 tx=0x7f2bc4004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb58178cbb91ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bbcff9640 1 -- 172.21.15.5:0/855240731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4044040 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bbcff9640 1 -- 172.21.15.5:0/855240731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2bc4049d50 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd00a55c0 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bbcff9640 1 -- 172.21.15.5:0/855240731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bc402cc50 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bdde62740 10 client.17538 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f420f1e80 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.166+0000 7f2bbcff9640 1 -- 172.21.15.5:0/855240731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd00a55c0 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bbcff9640 1 -- 172.21.15.5:0/855240731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2bc40035f0 con 0x560f42072e80 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 10 client.17538 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 10 client.17538 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 20 client.17538 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f420f25f0 0x560f420f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 4 client.17538 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f41fe4c10 con 0x560f420f25f0 2024-04-22T13:32:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bd9013640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f420f25f0 0x560f420f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bbcff9640 10 client.17538 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bd9013640 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f420f25f0 0x560f420f4a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9396358086ce9b93 server_cookie=5d108ed25fe2320 in_seq=0 out_seq=0 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bbcff9640 1 -- 172.21.15.5:0/855240731 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f41fe4c10 con 0x560f420f25f0 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bbcff9640 10 client.17538 handle_command_reply: tid=0 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.167+0000 7f2bdde62740 1 client.17538 shutdown 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 msgr2=0x560f4207fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f4207fc20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4004650 tx=0x7f2bc4004140 comp rx=0 tx=0).stop 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 shutdown_connections 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560f420f25f0 0x560f420f4a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f42072e80 0x560f4207fc20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bbcff9640 0 client.17538 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f4207ba00 0x560f42080160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 --2- 172.21.15.5:0/855240731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f420724d0 0x560f4207f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 >> 172.21.15.5:0/855240731 conn(0x560f4206cba0 msgr2=0x560f42077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 shutdown_connections 2024-04-22T13:32:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 1 -- 172.21.15.5:0/855240731 wait complete. 2024-04-22T13:32:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdde62740 20 client.17538 trim_cache size 0 max 16384 2024-04-22T13:32:09.171 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2bc00a3d50 msgr2=0x7f2bc00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2bc00a3d50 0x7f2bc00a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005c40 tx=0x7f2bcc0095e0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 msgr2=0x7f2bd410e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 0x7f2bd410e6c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c920 tx=0x7f2bd000cde0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 shutdown_connections 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2bc00a3d50 0x7f2bc00a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd410f140 0x7f2bd41a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bd4106160 0x7f2bd410ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 --2- 172.21.15.5:0/3215481830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4101090 0x7f2bd410e6c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 >> 172.21.15.5:0/3215481830 conn(0x7f2bd4078000 msgr2=0x7f2bd40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 shutdown_connections 2024-04-22T13:32:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.168+0000 7f2bdb277640 1 -- 172.21.15.5:0/3215481830 wait complete. 2024-04-22T13:32:09.173 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:08 smithi005 ceph-mon[28006]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:32:09.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:08 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2773767398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:09.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:32:09.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:08 smithi049 ceph-mon[31379]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:32:09.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:08 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2773767398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 -- 172.21.15.5:0/11742975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 msgr2=0x7f1f2810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 --2- 172.21.15.5:0/11742975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f2810bfc0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009900 tx=0x7f1f1c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 -- 172.21.15.5:0/11742975 shutdown_connections 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 --2- 172.21.15.5:0/11742975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f2810bfc0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 --2- 172.21.15.5:0/11742975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f2810c500 0x7f1f2810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 --2- 172.21.15.5:0/11742975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f281031c0 0x7f1f281035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 -- 172.21.15.5:0/11742975 >> 172.21.15.5:0/11742975 conn(0x7f1f28078000 msgr2=0x7f1f28078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 -- 172.21.15.5:0/11742975 shutdown_connections 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.284+0000 7f1f2d093640 1 -- 172.21.15.5:0/11742975 wait complete. 2024-04-22T13:32:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 Processor -- start 2024-04-22T13:32:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 -- start start 2024-04-22T13:32:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f281031c0 0x7f1f28100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f28101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f2810c500 0x7f1f28101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28112190 con 0x7f1f28103ae0 2024-04-22T13:32:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28112010 con 0x7f1f281031c0 2024-04-22T13:32:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.285+0000 7f1f2d093640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28112310 con 0x7f1f2810c500 2024-04-22T13:32:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f28101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f28101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58388/0 (socket says 172.21.15.5:58388) 2024-04-22T13:32:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 -- 172.21.15.5:0/1878557041 learned_addr learned my addr 172.21.15.5:0/1878557041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f281031c0 0x7f1f28100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2759e640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f2810c500 0x7f1f28101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 -- 172.21.15.5:0/1878557041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f2810c500 msgr2=0x7f1f28101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f2810c500 0x7f1f28101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 -- 172.21.15.5:0/1878557041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f281031c0 msgr2=0x7f1f28100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f281031c0 0x7f1f28100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 -- 172.21.15.5:0/1878557041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f281a6c50 con 0x7f1f28103ae0 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2759e640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f2810c500 0x7f1f28101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2659c640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f28101350 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c005940 tx=0x7f1f1c02d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f0ffff640 1 -- 172.21.15.5:0/1878557041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c043020 con 0x7f1f28103ae0 2024-04-22T13:32:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f0ffff640 1 -- 172.21.15.5:0/1878557041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c02dc90 con 0x7f1f28103ae0 2024-04-22T13:32:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f281a6ee0 con 0x7f1f28103ae0 2024-04-22T13:32:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f0ffff640 1 -- 172.21.15.5:0/1878557041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c003ab0 con 0x7f1f28103ae0 2024-04-22T13:32:09.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.286+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f281a73a0 con 0x7f1f28103ae0 2024-04-22T13:32:09.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:08 smithi107 ceph-mon[31859]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:32:09.287 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:08 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2773767398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.289+0000 7f1f0ffff640 1 -- 172.21.15.5:0/1878557041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1f1c046020 con 0x7f1f28103ae0 2024-04-22T13:32:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 Processor -- start 2024-04-22T13:32:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 -- start start 2024-04-22T13:32:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b5882c1f0 con 0x555b588d0a00 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b587c94b0 con 0x555b588c7e80 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555b588281b0 con 0x555b588c74d0 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f0ffff640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f100a3d20 0x7f1f100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58398/0 (socket says 172.21.15.5:58398) 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2659c640 1 -- 172.21.15.5:0/193830574 learned_addr learned my addr 172.21.15.5:0/193830574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f100a3d20 0x7f1f100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f0ffff640 1 -- 172.21.15.5:0/1878557041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1f1c0f2840 con 0x7f1f28103ae0 2024-04-22T13:32:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 -- 172.21.15.5:0/193830574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 msgr2=0x555b588d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 -- 172.21.15.5:0/193830574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 msgr2=0x555b588d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2759e640 1 -- 172.21.15.5:0/193830574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b587bb1b0 con 0x555b588c74d0 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.290+0000 7f1f2659c640 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.291+0000 7f1f2759e640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f100a3d20 0x7f1f100a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f20011c50 tx=0x7f1f2000b000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.291+0000 7f1f2759e640 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588c78b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f20005fd0 tx=0x7f1f20011750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49a299087d50e816 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.291+0000 7f1f0d7fa640 1 -- 172.21.15.5:0/193830574 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20022070 con 0x555b588c74d0 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.291+0000 7f1f0d7fa640 1 -- 172.21.15.5:0/193830574 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f20009c50 con 0x555b588c74d0 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.291+0000 7f1f0d7fa640 1 -- 172.21.15.5:0/193830574 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20005490 con 0x555b588c74d0 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- 172.21.15.5:0/193830574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 msgr2=0x555b588c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588c78b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f20005fd0 tx=0x7f1f20011750 comp rx=0 tx=0).stop 2024-04-22T13:32:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- 172.21.15.5:0/193830574 shutdown_connections 2024-04-22T13:32:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/193830574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588c78b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- 172.21.15.5:0/193830574 >> 172.21.15.5:0/193830574 conn(0x555b588c1ba0 msgr2=0x555b588c1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- 172.21.15.5:0/193830574 shutdown_connections 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- 172.21.15.5:0/193830574 wait complete. 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- start start 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 Processor -- start 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 0x555b588d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588d0a00 0x555b588cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b588d3910 con 0x555b588c74d0 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b587e5500 con 0x555b588d0a00 2024-04-22T13:32:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555b588d3a90 con 0x555b588c7e80 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2fd63740 10 client.0 _ll_register_callbacks cb 0x555b587b3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 0x555b588d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 0x555b588d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58404/0 (socket says 172.21.15.5:58404) 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f26d9d640 1 -- 172.21.15.5:0/3664947935 learned_addr learned my addr 172.21.15.5:0/3664947935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.292+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f2759e640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588d0a00 0x555b588cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f26d9d640 1 -- 172.21.15.5:0/3664947935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 msgr2=0x555b588cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f26d9d640 1 -- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588d0a00 msgr2=0x555b588cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588d0a00 0x555b588cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f26d9d640 1 -- 172.21.15.5:0/3664947935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5853e3f0 con 0x555b588c74d0 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f2659c640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 0x555b588d02f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800b920 tx=0x7f1f1800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6ca1cc58343383 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f0aff5640 1 -- 172.21.15.5:0/3664947935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1801b440 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f0aff5640 1 -- 172.21.15.5:0/3664947935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1f18011440 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b587cb930 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f0aff5640 1 -- 172.21.15.5:0/3664947935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f18011680 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f2fd63740 10 client.17550 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b586087d0 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.293+0000 7f1f0aff5640 1 -- 172.21.15.5:0/3664947935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555b587cb930 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f0aff5640 1 -- 172.21.15.5:0/3664947935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1f18019020 con 0x555b588c74d0 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2fd63740 10 client.17550 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:09.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2fd63740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2fd63740 20 client.17550 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b588e0bc0 0x555b588e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2fd63740 4 client.17550 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b58839c10 con 0x555b588e0bc0 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2659c640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b588e0bc0 0x555b588e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f0aff5640 10 client.17550 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.294+0000 7f1f2659c640 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b588e0bc0 0x555b588e2ff0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290ce3ef765e1f23 server_cookie=595555da7d9b02e4 in_seq=0 out_seq=0 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f0aff5640 1 -- 172.21.15.5:0/3664947935 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b58839c10 con 0x555b588e0bc0 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f0aff5640 10 client.17550 handle_command_reply: tid=0 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 client.17550 shutdown 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 msgr2=0x555b588d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 0x555b588d02f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800b920 tx=0x7f1f1800bd20 comp rx=0 tx=0).stop 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 shutdown_connections 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b588e0bc0 0x555b588e2ff0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588d0a00 0x555b588cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f0aff5640 0 client.17550 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/3664947935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588c74d0 0x555b588d02f0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 >> 172.21.15.5:0/3664947935 conn(0x555b588c1ba0 msgr2=0x555b588c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 shutdown_connections 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 1 -- 172.21.15.5:0/3664947935 wait complete. 2024-04-22T13:32:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.295+0000 7f1f2fd63740 20 client.17550 trim_cache size 0 max 16384 2024-04-22T13:32:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.308+0000 7f1f2fd63740 1 Processor -- start 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 -- start start 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281afc70 con 0x555b588d0a00 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b17b0 con 0x555b588c7e80 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b0340 con 0x555b588c74d0 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58420/0 (socket says 172.21.15.5:58420) 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 -- 172.21.15.5:0/4266793873 learned_addr learned my addr 172.21.15.5:0/4266793873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 -- 172.21.15.5:0/4266793873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 msgr2=0x555b588dc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 -- 172.21.15.5:0/4266793873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 msgr2=0x555b588c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 -- 172.21.15.5:0/4266793873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b588cc630 con 0x555b588d0a00 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f2759e640 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588cfdb0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009440 tx=0x7f1f20011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b90424fc9c7a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f0cff9640 1 -- 172.21.15.5:0/4266793873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20035070 con 0x555b588d0a00 2024-04-22T13:32:09.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.309+0000 7f1f0cff9640 1 -- 172.21.15.5:0/4266793873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f20014050 con 0x555b588d0a00 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f0cff9640 1 -- 172.21.15.5:0/4266793873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20022040 con 0x555b588d0a00 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 -- 172.21.15.5:0/4266793873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 msgr2=0x555b588cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588cfdb0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009440 tx=0x7f1f20011f20 comp rx=0 tx=0).stop 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 -- 172.21.15.5:0/4266793873 shutdown_connections 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588cfdb0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c7e80 0x555b588c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/4266793873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c74d0 0x555b588dc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 -- 172.21.15.5:0/4266793873 >> 172.21.15.5:0/4266793873 conn(0x555b588c1ba0 msgr2=0x555b588c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 -- 172.21.15.5:0/4266793873 shutdown_connections 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 -- 172.21.15.5:0/4266793873 wait complete. 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 -- start start 2024-04-22T13:32:09.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.310+0000 7f1f2fd63740 1 Processor -- start 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c74d0 0x555b588d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b10e0 con 0x555b588d0a00 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b0a10 con 0x555b588c74d0 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281adc50 con 0x555b588c7e80 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 10 client.0 _ll_register_callbacks cb 0x7f1f18004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58424/0 (socket says 172.21.15.5:58424) 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 -- 172.21.15.5:0/2564763477 learned_addr learned my addr 172.21.15.5:0/2564763477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2659c640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 -- 172.21.15.5:0/2564763477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 msgr2=0x555b588c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 -- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c74d0 msgr2=0x555b588d8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c74d0 0x555b588d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c74d0 0x555b588d8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 -- 172.21.15.5:0/2564763477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b588cabd0 con 0x555b588d0a00 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2759e640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588ca510 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f1f20033040 tx=0x7f1f20014ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30959f189bea17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f0aff5640 1 -- 172.21.15.5:0/2564763477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20040060 con 0x555b588d0a00 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f0aff5640 1 -- 172.21.15.5:0/2564763477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1f20014200 con 0x555b588d0a00 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.311+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f20014050 con 0x555b588d0a00 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f0aff5640 1 -- 172.21.15.5:0/2564763477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f20035040 con 0x555b588d0a00 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 10 client.17556 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b58947020 con 0x555b588d0a00 2024-04-22T13:32:09.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f0aff5640 1 -- 172.21.15.5:0/2564763477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f20014050 con 0x555b588d0a00 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f0aff5640 1 -- 172.21.15.5:0/2564763477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1f2002f070 con 0x555b588d0a00 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 10 client.17556 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 20 client.17556 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b589474f0 0x555b589498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 4 client.17556 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b58833a80 con 0x555b589474f0 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.312+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b589474f0 0x555b589498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f26d9d640 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b589474f0 0x555b589498e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d046917450d80f4e server_cookie=23259ef0693568b9 in_seq=0 out_seq=0 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f0aff5640 10 client.17556 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:09.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f0aff5640 1 -- 172.21.15.5:0/2564763477 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b58833a80 con 0x555b589474f0 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f0aff5640 10 client.17556 handle_command_reply: tid=0 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 client.17556 shutdown 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 msgr2=0x555b588ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588ca510 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f1f20033040 tx=0x7f1f20014ab0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 shutdown_connections 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x555b589474f0 0x555b589498e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b588d0a00 0x555b588ca510 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f0aff5640 0 client.17556 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b588c7e80 0x555b588c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 --2- 172.21.15.5:0/2564763477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b588c74d0 0x555b588d8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 >> 172.21.15.5:0/2564763477 conn(0x555b588c1ba0 msgr2=0x555b588c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 shutdown_connections 2024-04-22T13:32:09.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 1 -- 172.21.15.5:0/2564763477 wait complete. 2024-04-22T13:32:09.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.313+0000 7f1f2fd63740 20 client.17556 trim_cache size 0 max 16384 2024-04-22T13:32:09.324 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f100a3d20 msgr2=0x7f1f100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f100a3d20 0x7f1f100a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f20011c50 tx=0x7f1f2000b000 comp rx=0 tx=0).stop 2024-04-22T13:32:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 msgr2=0x7f1f28101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:09.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f28101350 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c005940 tx=0x7f1f1c02d890 comp rx=0 tx=0).stop 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 shutdown_connections 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f28103ae0 0x7f1f28101350 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1f100a3d20 0x7f1f100a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f2810c500 0x7f1f28101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 --2- 172.21.15.5:0/1878557041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f281031c0 0x7f1f28100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 >> 172.21.15.5:0/1878557041 conn(0x7f1f28078000 msgr2=0x7f1f280fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 shutdown_connections 2024-04-22T13:32:09.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:09.314+0000 7f1f2d093640 1 -- 172.21.15.5:0/1878557041 wait complete. 2024-04-22T13:32:10.327 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:32:10.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:32:10.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:32:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 -- 172.21.15.5:0/404275862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 msgr2=0x7fec0c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 --2- 172.21.15.5:0/404275862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c1035a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fec08009900 tx=0x7fec0802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 -- 172.21.15.5:0/404275862 shutdown_connections 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 --2- 172.21.15.5:0/404275862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c10c6d0 0x7fec0c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 --2- 172.21.15.5:0/404275862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c103b70 0x7fec0c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 --2- 172.21.15.5:0/404275862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c1035a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 -- 172.21.15.5:0/404275862 >> 172.21.15.5:0/404275862 conn(0x7fec0c077f70 msgr2=0x7fec0c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 -- 172.21.15.5:0/404275862 shutdown_connections 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.453+0000 7fec132d2640 1 -- 172.21.15.5:0/404275862 wait complete. 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 Processor -- start 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 -- start start 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c10c6d0 0x7fec0c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c1988c0 0x7fec0c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c112390 con 0x7fec0c1031c0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c112210 con 0x7fec0c1988c0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec132d2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c112510 con 0x7fec0c10c6d0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec1106e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec1106e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58444/0 (socket says 172.21.15.5:58444) 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.454+0000 7fec1106e640 1 -- 172.21.15.5:0/920310265 learned_addr learned my addr 172.21.15.5:0/920310265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1086d640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c10c6d0 0x7fec0c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1186f640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c1988c0 0x7fec0c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1106e640 1 -- 172.21.15.5:0/920310265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c10c6d0 msgr2=0x7fec0c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1106e640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c10c6d0 0x7fec0c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1106e640 1 -- 172.21.15.5:0/920310265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c1988c0 msgr2=0x7fec0c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1106e640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c1988c0 0x7fec0c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1106e640 1 -- 172.21.15.5:0/920310265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c1993e0 con 0x7fec0c1031c0 2024-04-22T13:32:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1086d640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c10c6d0 0x7fec0c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1186f640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c1988c0 0x7fec0c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec1106e640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c19e6c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fec08005940 tx=0x7fec0802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec027fc640 1 -- 172.21.15.5:0/920310265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08043020 con 0x7fec0c1031c0 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec027fc640 1 -- 172.21.15.5:0/920310265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec0802ca70 con 0x7fec0c1031c0 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c199670 con 0x7fec0c1031c0 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec027fc640 1 -- 172.21.15.5:0/920310265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec080039f0 con 0x7fec0c1031c0 2024-04-22T13:32:10.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.455+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c1a5870 con 0x7fec0c1031c0 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.457+0000 7fec027fc640 1 -- 172.21.15.5:0/920310265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fec08046020 con 0x7fec0c1031c0 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 Processor -- start 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 -- start start 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d23cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 0x5572c1d2f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5572c1c87f00 con 0x5572c1d2ce10 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572c1c25590 con 0x5572c1d24290 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec15fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5572c1c83ec0 con 0x5572c1d238e0 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec027fc640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7febf80a3d20 0x7febf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 0x5572c1d2f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 0x5572c1d2f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58450/0 (socket says 172.21.15.5:58450) 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec1186f640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7febf80a3d20 0x7febf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec1086d640 1 -- 172.21.15.5:0/3510384235 learned_addr learned my addr 172.21.15.5:0/3510384235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec027fc640 1 -- 172.21.15.5:0/920310265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fec080f2870 con 0x7fec0c1031c0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec1186f640 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d23cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.458+0000 7fec1106e640 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1086d640 1 -- 172.21.15.5:0/3510384235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 msgr2=0x5572c1d23cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1086d640 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d23cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1086d640 1 -- 172.21.15.5:0/3510384235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 msgr2=0x5572c1d2c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1086d640 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d2c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1086d640 1 -- 172.21.15.5:0/3510384235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c1a627e0 con 0x5572c1d2ce10 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1186f640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7febf80a3d20 0x7febf80a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec04005fd0 tx=0x7fec04009db0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7fec1086d640 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 0x5572c1d2f260 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7febfc009c30 tx=0x7febfc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba14deba783cc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7febf77fe640 1 -- 172.21.15.5:0/3510384235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc005020 con 0x5572c1d2ce10 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7febf77fe640 1 -- 172.21.15.5:0/3510384235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc0047a0 con 0x5572c1d2ce10 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.459+0000 7febf77fe640 1 -- 172.21.15.5:0/3510384235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0126b0 con 0x5572c1d2ce10 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 -- 172.21.15.5:0/3510384235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 msgr2=0x5572c1d2f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 0x5572c1d2f260 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7febfc009c30 tx=0x7febfc0117b0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 -- 172.21.15.5:0/3510384235 shutdown_connections 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d2ce10 0x5572c1d2f260 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d2c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3510384235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d23cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 -- 172.21.15.5:0/3510384235 >> 172.21.15.5:0/3510384235 conn(0x5572c1d1dfb0 msgr2=0x5572c1d1e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 -- 172.21.15.5:0/3510384235 shutdown_connections 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 -- 172.21.15.5:0/3510384235 wait complete. 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 -- start start 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.460+0000 7fec15fa2740 1 Processor -- start 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d29d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 0x5572c1d2a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d2ce10 0x5572c1d2a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5572c1d2fd80 con 0x5572c1d24290 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572c1c41580 con 0x5572c1d2ce10 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5572c1d2ff00 con 0x5572c1d238e0 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec15fa2740 10 client.0 _ll_register_callbacks cb 0x5572c1c47650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 0x5572c1d2a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 0x5572c1d2a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58464/0 (socket says 172.21.15.5:58464) 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 -- 172.21.15.5:0/3396160366 learned_addr learned my addr 172.21.15.5:0/3396160366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1106e640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d29d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1186f640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d2ce10 0x5572c1d2a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 -- 172.21.15.5:0/3396160366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 msgr2=0x5572c1d29d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d29d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 -- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d2ce10 msgr2=0x5572c1d2a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d2ce10 0x5572c1d2a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 -- 172.21.15.5:0/3396160366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c1ac2f70 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7fec1086d640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 0x5572c1d2a290 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7febfc012970 tx=0x7febfc0129a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453cf772364237d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.461+0000 7febf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7febf4ff9640 1 -- 172.21.15.5:0/3396160366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc028030 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7febf4ff9640 1 -- 172.21.15.5:0/3396160366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7febfc01f050 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc0047a0 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7febf4ff9640 1 -- 172.21.15.5:0/3396160366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7febfc032050 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 10 client.17568 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572c195a170 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7febf4ff9640 1 -- 172.21.15.5:0/3396160366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc0047a0 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7febf4ff9640 1 -- 172.21.15.5:0/3396160366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7febfc024a30 con 0x5572c1d24290 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 10 client.17568 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 20 client.17568 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d3d1f0 0x5572c1d3f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 4 client.17568 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.462+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c1c95920 con 0x5572c1d3d1f0 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.463+0000 7fec1106e640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d3d1f0 0x5572c1d3f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.463+0000 7fec1106e640 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d3d1f0 0x5572c1d3f5e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536a5a47890e00 server_cookie=6a834cb9fa391144 in_seq=0 out_seq=0 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.463+0000 7febf4ff9640 10 client.17568 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7febf4ff9640 1 -- 172.21.15.5:0/3396160366 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572c1c95920 con 0x5572c1d3d1f0 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7febf4ff9640 10 client.17568 handle_command_reply: tid=0 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 client.17568 shutdown 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 msgr2=0x5572c1d2a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 0x5572c1d2a290 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7febfc012970 tx=0x7febfc0129a0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 shutdown_connections 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d3d1f0 0x5572c1d3f5e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d24290 0x5572c1d2a290 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d2ce10 0x5572c1d2a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 --2- 172.21.15.5:0/3396160366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d238e0 0x5572c1d29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 >> 172.21.15.5:0/3396160366 conn(0x5572c1d1dfb0 msgr2=0x5572c1d1fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7febf4ff9640 0 client.17568 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 shutdown_connections 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 1 -- 172.21.15.5:0/3396160366 wait complete. 2024-04-22T13:32:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.464+0000 7fec15fa2740 20 client.17568 trim_cache size 0 max 16384 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.482+0000 7fec15fa2740 1 Processor -- start 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.482+0000 7fec15fa2740 1 -- start start 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.482+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d2c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d38c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d44c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec15fa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1adb20 con 0x5572c1d238e0 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec15fa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1af660 con 0x5572c1d24290 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec15fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1ae1f0 con 0x5572c1d1dfb0 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d38c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d38c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58470/0 (socket says 172.21.15.5:58470) 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 -- 172.21.15.5:0/1308493282 learned_addr learned my addr 172.21.15.5:0/1308493282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1186f640 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d44c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1106e640 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d2c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 -- 172.21.15.5:0/1308493282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 msgr2=0x5572c1d2c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d2c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 -- 172.21.15.5:0/1308493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 msgr2=0x5572c1d44c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d44c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 -- 172.21.15.5:0/1308493282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c1d282a0 con 0x5572c1d238e0 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1106e640 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d2c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7fec1086d640 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d38c40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958cbb54be8d0a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7febf6ffd640 1 -- 172.21.15.5:0/1308493282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc03a020 con 0x5572c1d238e0 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7febf6ffd640 1 -- 172.21.15.5:0/1308493282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc009530 con 0x5572c1d238e0 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.483+0000 7febf6ffd640 1 -- 172.21.15.5:0/1308493282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc028070 con 0x5572c1d238e0 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 -- 172.21.15.5:0/1308493282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 msgr2=0x5572c1d38c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d38c40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc011cb0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 -- 172.21.15.5:0/1308493282 shutdown_connections 2024-04-22T13:32:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d38c40 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d44c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 --2- 172.21.15.5:0/1308493282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d2c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 -- 172.21.15.5:0/1308493282 >> 172.21.15.5:0/1308493282 conn(0x5572c1d249e0 msgr2=0x5572c1d1fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 -- 172.21.15.5:0/1308493282 shutdown_connections 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 -- 172.21.15.5:0/1308493282 wait complete. 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 -- start start 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.484+0000 7fec15fa2740 1 Processor -- start 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d31cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d9f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1aef90 con 0x5572c1d238e0 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1ae8c0 con 0x5572c1d24290 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1abb20 con 0x5572c1d1dfb0 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec15fa2740 10 client.0 _ll_register_callbacks cb 0x7fec080057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d31cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d31cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58480/0 (socket says 172.21.15.5:58480) 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 -- 172.21.15.5:0/338486078 learned_addr learned my addr 172.21.15.5:0/338486078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1106e640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1186f640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d9f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 -- 172.21.15.5:0/338486078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 msgr2=0x5572c1d31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 -- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 msgr2=0x5572c1d9f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d9f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1086d640 1 -- 172.21.15.5:0/338486078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c1da00b0 con 0x5572c1d238e0 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1106e640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.485+0000 7fec1186f640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d9f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7fec1086d640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d31cb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7febfc024ea0 tx=0x7febfc024710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8a5a2b4025550 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7febf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7febf4ff9640 1 -- 172.21.15.5:0/338486078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc043060 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7febf4ff9640 1 -- 172.21.15.5:0/338486078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7febfc012970 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc009530 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7febf4ff9640 1 -- 172.21.15.5:0/338486078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7febfc03a030 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7fec15fa2740 10 client.17574 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572c1da8450 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7febf4ff9640 1 -- 172.21.15.5:0/338486078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc009530 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7febf4ff9640 1 -- 172.21.15.5:0/338486078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7febfc048b60 con 0x5572c1d238e0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7fec15fa2740 10 client.17574 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.486+0000 7fec15fa2740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7fec15fa2740 20 client.17574 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7fec15fa2740 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d2ce10 0x5572c1daab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7fec15fa2740 4 client.17574 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5572c1c8f790 con 0x5572c1d2ce10 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7fec1106e640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d2ce10 0x5572c1daab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7febf4ff9640 10 client.17574 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.487+0000 7fec1106e640 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d2ce10 0x5572c1daab30 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eded6b68449b0a40 server_cookie=1c1582e220321b6f in_seq=0 out_seq=0 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7febf4ff9640 1 -- 172.21.15.5:0/338486078 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5572c1c8f790 con 0x5572c1d2ce10 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7febf4ff9640 10 client.17574 handle_command_reply: tid=0 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7fec15fa2740 1 client.17574 shutdown 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 msgr2=0x5572c1d31cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7fec15fa2740 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d31cb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7febfc024ea0 tx=0x7febfc024710 comp rx=0 tx=0).stop 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 shutdown_connections 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7fec15fa2740 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5572c1d2ce10 0x5572c1daab30 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.488+0000 7fec15fa2740 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c1d238e0 0x5572c1d31cb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7febf4ff9640 0 client.17574 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec15fa2740 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572c1d24290 0x5572c1d9f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec15fa2740 1 --2- 172.21.15.5:0/338486078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572c1d1dfb0 0x5572c1d31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 >> 172.21.15.5:0/338486078 conn(0x5572c1d249e0 msgr2=0x5572c1d29cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 shutdown_connections 2024-04-22T13:32:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec15fa2740 1 -- 172.21.15.5:0/338486078 wait complete. 2024-04-22T13:32:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec15fa2740 20 client.17574 trim_cache size 0 max 16384 2024-04-22T13:32:10.491 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:32:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "1af8b33b-74a5-48e8-984e-e20d79561794", 2024-04-22T13:32:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:32:10.491 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7febf80a3d20 msgr2=0x7febf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7febf80a3d20 0x7febf80a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec04005fd0 tx=0x7fec04009db0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 msgr2=0x7fec0c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c19e6c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fec08005940 tx=0x7fec0802c5f0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 shutdown_connections 2024-04-22T13:32:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7febf80a3d20 0x7febf80a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec0c1988c0 0x7fec0c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c10c6d0 0x7fec0c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 --2- 172.21.15.5:0/920310265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec0c1031c0 0x7fec0c19e6c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.489+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 >> 172.21.15.5:0/920310265 conn(0x7fec0c077f70 msgr2=0x7fec0c0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.490+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 shutdown_connections 2024-04-22T13:32:10.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.490+0000 7fec132d2640 1 -- 172.21.15.5:0/920310265 wait complete. 2024-04-22T13:32:10.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:10.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 -- 172.21.15.5:0/2325027211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6704103ae0 msgr2=0x7f670410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 --2- 172.21.15.5:0/2325027211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6704103ae0 0x7f670410bfc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009900 tx=0x7f66f002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 -- 172.21.15.5:0/2325027211 shutdown_connections 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 --2- 172.21.15.5:0/2325027211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6704103ae0 0x7f670410bfc0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 --2- 172.21.15.5:0/2325027211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 0x7f670410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 --2- 172.21.15.5:0/2325027211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67041031c0 0x7f67041035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 -- 172.21.15.5:0/2325027211 >> 172.21.15.5:0/2325027211 conn(0x7f6704077fa0 msgr2=0x7f67040783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 -- 172.21.15.5:0/2325027211 shutdown_connections 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.605+0000 7f670ae3f640 1 -- 172.21.15.5:0/2325027211 wait complete. 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 Processor -- start 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 -- start start 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 0x7f670419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6704103ae0 0x7f670419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 0x7f67041985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67041121d0 con 0x7f67041031c0 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6704112050 con 0x7f670410c590 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f670ae3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6704112350 con 0x7f6704103ae0 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 0x7f670419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 0x7f670419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58490/0 (socket says 172.21.15.5:58490) 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 -- 172.21.15.5:0/695400919 learned_addr learned my addr 172.21.15.5:0/695400919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6703fff640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6704103ae0 0x7f670419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f67093dc640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 0x7f67041985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 -- 172.21.15.5:0/695400919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6704103ae0 msgr2=0x7f670419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6704103ae0 0x7f670419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 -- 172.21.15.5:0/695400919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 msgr2=0x7f67041985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 0x7f67041985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6708bdb640 1 -- 172.21.15.5:0/695400919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6704198d50 con 0x7f67041031c0 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f67093dc640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 0x7f67041985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.606+0000 7f6703fff640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6704103ae0 0x7f670419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.607+0000 7f6708bdb640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 0x7f670419e420 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f66f800e9f0 tx=0x7f66f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.607+0000 7f6701ffb640 1 -- 172.21.15.5:0/695400919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f800cd80 con 0x7f67041031c0 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.607+0000 7f6701ffb640 1 -- 172.21.15.5:0/695400919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f80054d0 con 0x7f67041031c0 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.607+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6704199040 con 0x7f67041031c0 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.607+0000 7f6701ffb640 1 -- 172.21.15.5:0/695400919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8010690 con 0x7f67041031c0 2024-04-22T13:32:10.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.607+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a52e0 con 0x7f67041031c0 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.609+0000 7f6701ffb640 1 -- 172.21.15.5:0/695400919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f66f8004e90 con 0x7f67041031c0 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.609+0000 7f670da2e740 1 Processor -- start 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 -- start start 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5382c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f53839950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f5383c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f537911e0 con 0x557f5382cb50 2024-04-22T13:32:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557f5372e390 con 0x557f5382c230 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f670da2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557f5378d1a0 con 0x557f53839e90 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6701ffb640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f53839950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f53839950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58500/0 (socket says 172.21.15.5:58500) 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 -- 172.21.15.5:0/446743311 learned_addr learned my addr 172.21.15.5:0/446743311 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f67093dc640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6703fff640 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f5383c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f67093dc640 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5382c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6701ffb640 1 -- 172.21.15.5:0/695400919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f66f80d0ce0 con 0x7f67041031c0 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 -- 172.21.15.5:0/446743311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 msgr2=0x557f5383c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f5383c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 -- 172.21.15.5:0/446743311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 msgr2=0x557f5382c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5382c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 -- 172.21.15.5:0/446743311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5356d7d0 con 0x557f5382cb50 2024-04-22T13:32:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6703fff640 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f5383c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.610+0000 7f6708bdb640 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f53839950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c6a0 tx=0x7f66f8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483cdb2605bd1bae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f67093dc640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66f40a3d20 0x7f66f40a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005c40 tx=0x7f66fc009d50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f66eeffd640 1 -- 172.21.15.5:0/446743311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f809c0a0 con 0x557f5382cb50 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f66eeffd640 1 -- 172.21.15.5:0/446743311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f809d350 con 0x557f5382cb50 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f66eeffd640 1 -- 172.21.15.5:0/446743311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8005180 con 0x557f5382cb50 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 -- 172.21.15.5:0/446743311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 msgr2=0x557f53839950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f53839950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c6a0 tx=0x7f66f8004a90 comp rx=0 tx=0).stop 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 -- 172.21.15.5:0/446743311 shutdown_connections 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f53839950 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f5383c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 --2- 172.21.15.5:0/446743311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5382c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 -- 172.21.15.5:0/446743311 >> 172.21.15.5:0/446743311 conn(0x557f53826cb0 msgr2=0x557f53827090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 -- 172.21.15.5:0/446743311 shutdown_connections 2024-04-22T13:32:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.611+0000 7f670da2e740 1 -- 172.21.15.5:0/446743311 wait complete. 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 -- start start 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 Processor -- start 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f53836e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f538373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f53837900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f5383cc70 con 0x557f5382cb50 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557f5374a2f0 con 0x557f5382c230 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557f5383cdf0 con 0x557f53839e90 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f670da2e740 10 client.0 _ll_register_callbacks cb 0x557f53757580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f6703fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f538373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f6703fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f538373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58508/0 (socket says 172.21.15.5:58508) 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f6703fff640 1 -- 172.21.15.5:0/4059984719 learned_addr learned my addr 172.21.15.5:0/4059984719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.612+0000 7f6708bdb640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f53836e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f67093dc640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f53837900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f6703fff640 1 -- 172.21.15.5:0/4059984719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 msgr2=0x557f53837900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f6703fff640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f53837900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f6703fff640 1 -- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 msgr2=0x557f53836e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f6703fff640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f53836e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f6703fff640 1 -- 172.21.15.5:0/4059984719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f534a33f0 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f6703fff640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f538373c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009900 tx=0x7f66f002cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81c59f2ba3c2349 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f66e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f66e3fff640 1 -- 172.21.15.5:0/4059984719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0043070 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f66e3fff640 1 -- 172.21.15.5:0/4059984719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f66f00035f0 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f809d350 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f66e3fff640 1 -- 172.21.15.5:0/4059984719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66f002c690 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f670da2e740 10 client.17586 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.613+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f534630f0 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f66e3fff640 1 -- 172.21.15.5:0/4059984719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f809d350 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f66e3fff640 1 -- 172.21.15.5:0/4059984719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66f0050970 con 0x557f5382cb50 2024-04-22T13:32:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f670da2e740 10 client.17586 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f670da2e740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f670da2e740 20 client.17586 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f670da2e740 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53849db0 0x557f5384c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f670da2e740 4 client.17586 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f5379ec00 con 0x557f53849db0 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f6708bdb640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53849db0 0x557f5384c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f6708bdb640 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53849db0 0x557f5384c1e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f01f1d2354f6247 server_cookie=c7a9e53ca76d9648 in_seq=0 out_seq=0 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.614+0000 7f66e3fff640 10 client.17586 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f66e3fff640 1 -- 172.21.15.5:0/4059984719 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f5379ec00 con 0x557f53849db0 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f66e3fff640 10 client.17586 handle_command_reply: tid=0 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 client.17586 shutdown 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 msgr2=0x557f538373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f538373c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009900 tx=0x7f66f002cab0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 shutdown_connections 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53849db0 0x557f5384c1e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5382cb50 0x557f538373c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f53839e90 0x557f53837900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f66e3fff640 0 client.17586 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 --2- 172.21.15.5:0/4059984719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f53836e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 >> 172.21.15.5:0/4059984719 conn(0x557f53826cb0 msgr2=0x557f53828a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 shutdown_connections 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 1 -- 172.21.15.5:0/4059984719 wait complete. 2024-04-22T13:32:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.615+0000 7f670da2e740 20 client.17586 trim_cache size 0 max 16384 2024-04-22T13:32:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 Processor -- start 2024-04-22T13:32:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 -- start start 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f538395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382c230 0x557f53835dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382cb50 0x557f53845b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ab950 con 0x557f53826cb0 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ad490 con 0x557f5382cb50 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f670da2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ac020 con 0x557f5382c230 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f538395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f538395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58510/0 (socket says 172.21.15.5:58510) 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 -- 172.21.15.5:0/1469920671 learned_addr learned my addr 172.21.15.5:0/1469920671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6703fff640 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382c230 0x557f53835dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 -- 172.21.15.5:0/1469920671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382c230 msgr2=0x557f53835dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382c230 0x557f53835dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 -- 172.21.15.5:0/1469920671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382cb50 msgr2=0x557f53845b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382cb50 0x557f53845b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.633+0000 7f6708bdb640 1 -- 172.21.15.5:0/1469920671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f53831c60 con 0x557f53826cb0 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f6708bdb640 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f538395b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c6a0 tx=0x7f66f809c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df65ec323b5793bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f66ee7fc640 1 -- 172.21.15.5:0/1469920671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80b6070 con 0x557f53826cb0 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f66ee7fc640 1 -- 172.21.15.5:0/1469920671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f809d350 con 0x557f53826cb0 2024-04-22T13:32:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f66ee7fc640 1 -- 172.21.15.5:0/1469920671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80a58c0 con 0x557f53826cb0 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 -- 172.21.15.5:0/1469920671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 msgr2=0x557f538395b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f538395b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c6a0 tx=0x7f66f809c4c0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 -- 172.21.15.5:0/1469920671 shutdown_connections 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382cb50 0x557f53845b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382c230 0x557f53835dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 --2- 172.21.15.5:0/1469920671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f538395b0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 -- 172.21.15.5:0/1469920671 >> 172.21.15.5:0/1469920671 conn(0x557f53832850 msgr2=0x557f53828990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 -- 172.21.15.5:0/1469920671 shutdown_connections 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.634+0000 7f670da2e740 1 -- 172.21.15.5:0/1469920671 wait complete. 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 -- start start 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 Processor -- start 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f5383e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5383eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382cb50 0x557f538a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67041acdc0 con 0x557f53826cb0 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ac6f0 con 0x557f5382c230 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67041a9930 con 0x557f5382cb50 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f670da2e740 10 client.0 _ll_register_callbacks cb 0x7f66f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f5383e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f5383e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58514/0 (socket says 172.21.15.5:58514) 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 -- 172.21.15.5:0/3652620128 learned_addr learned my addr 172.21.15.5:0/3652620128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6703fff640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5383eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f67093dc640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382cb50 0x557f538a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 -- 172.21.15.5:0/3652620128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382cb50 msgr2=0x557f538a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382cb50 0x557f538a82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 -- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 msgr2=0x557f5383eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5383eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6708bdb640 1 -- 172.21.15.5:0/3652620128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f538a89a0 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.635+0000 7f6703fff640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5383eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f6708bdb640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f5383e660 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f66f800cfd0 tx=0x7f66f809c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba35ee616c9598e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f66e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f66e3fff640 1 -- 172.21.15.5:0/3652620128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80b6040 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f66e3fff640 1 -- 172.21.15.5:0/3652620128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f66f80992c0 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f809d350 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f66e3fff640 1 -- 172.21.15.5:0/3652620128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66f8005180 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 10 client.17592 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f538b0e50 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f66e3fff640 1 -- 172.21.15.5:0/3652620128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f809d350 con 0x557f53826cb0 2024-04-22T13:32:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f66e3fff640 1 -- 172.21.15.5:0/3652620128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66f80bd0b0 con 0x557f53826cb0 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 10 client.17592 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 20 client.17592 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53839e90 0x557f538b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 4 client.17592 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.636+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f53798a70 con 0x557f53839e90 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f6703fff640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53839e90 0x557f538b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f6703fff640 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53839e90 0x557f538b3030 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f912d300216e1145 server_cookie=2dbbc4face1413c4 in_seq=0 out_seq=0 2024-04-22T13:32:10.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f66e3fff640 10 client.17592 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f66e3fff640 1 -- 172.21.15.5:0/3652620128 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f53798a70 con 0x557f53839e90 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f66e3fff640 10 client.17592 handle_command_reply: tid=0 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 client.17592 shutdown 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 msgr2=0x557f5383e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f5383e660 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f66f800cfd0 tx=0x7f66f809c2e0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 shutdown_connections 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x557f53839e90 0x557f538b3030 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f5382cb50 0x557f538a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f5382c230 0x557f5383eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 --2- 172.21.15.5:0/3652620128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f53826cb0 0x557f5383e660 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f66e3fff640 0 client.17592 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 >> 172.21.15.5:0/3652620128 conn(0x557f53832850 msgr2=0x557f53836e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 shutdown_connections 2024-04-22T13:32:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.637+0000 7f670da2e740 1 -- 172.21.15.5:0/3652620128 wait complete. 2024-04-22T13:32:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670da2e740 20 client.17592 trim_cache size 0 max 16384 2024-04-22T13:32:10.644 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:32:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:32:10.644 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66f40a3d20 msgr2=0x7f66f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66f40a3d20 0x7f66f40a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005c40 tx=0x7f66fc009d50 comp rx=0 tx=0).stop 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 msgr2=0x7f670419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 0x7f670419e420 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f66f800e9f0 tx=0x7f66f800edf0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 shutdown_connections 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670410c590 0x7f67041985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6704103ae0 0x7f670419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 --2- 172.21.15.5:0/695400919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67041031c0 0x7f670419e420 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 >> 172.21.15.5:0/695400919 conn(0x7f6704077fa0 msgr2=0x7f67040ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 shutdown_connections 2024-04-22T13:32:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.638+0000 7f670ae3f640 1 -- 172.21.15.5:0/695400919 wait complete. 2024-04-22T13:32:10.647 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:32:10.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.757+0000 7efc9bcc1640 1 -- 172.21.15.5:0/2408234465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc94108060 msgr2=0x7efc94108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.757+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/2408234465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc94108060 0x7efc94108440 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7efc90009930 tx=0x7efc9002bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- 172.21.15.5:0/2408234465 shutdown_connections 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/2408234465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9406b000 0x7efc9410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/2408234465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 0x7efc9406aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/2408234465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc94108060 0x7efc94108440 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- 172.21.15.5:0/2408234465 >> 172.21.15.5:0/2408234465 conn(0x7efc94077c40 msgr2=0x7efc94078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- 172.21.15.5:0/2408234465 shutdown_connections 2024-04-22T13:32:10.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- 172.21.15.5:0/2408234465 wait complete. 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 Processor -- start 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- start start 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 0x7efc941986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 0x7efc94198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc94108060 0x7efc9419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc941121a0 con 0x7efc9406b000 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc94112020 con 0x7efc94108060 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.758+0000 7efc9bcc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc94112320 con 0x7efc9406a640 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 0x7efc94198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 0x7efc94198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58526/0 (socket says 172.21.15.5:58526) 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 -- 172.21.15.5:0/722194093 learned_addr learned my addr 172.21.15.5:0/722194093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc99a5d640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 0x7efc941986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9a25e640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc94108060 0x7efc9419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 -- 172.21.15.5:0/722194093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 msgr2=0x7efc941986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 0x7efc941986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 -- 172.21.15.5:0/722194093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc94108060 msgr2=0x7efc9419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc94108060 0x7efc9419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 -- 172.21.15.5:0/722194093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9419d690 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc99a5d640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 0x7efc941986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9925c640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 0x7efc94198be0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efc8400e9f0 tx=0x7efc8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400cd80 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc840054d0 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9419d980 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84010690 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.759+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc941a5210 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.760+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94103650 con 0x7efc9406b000 2024-04-22T13:32:10.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.761+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efc84004e90 con 0x7efc9406b000 2024-04-22T13:32:10.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.762+0000 7efc8affd640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efc800a3d20 0x7efc800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:10.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.762+0000 7efc99a5d640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efc800a3d20 0x7efc800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:10.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.762+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efc840d0bc0 con 0x7efc9406b000 2024-04-22T13:32:10.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.763+0000 7efc99a5d640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efc800a3d20 0x7efc800a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc90005ab0 tx=0x7efc90005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:10.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.767+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc8409d290 con 0x7efc9406b000 2024-04-22T13:32:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.995+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc941999f0 con 0x7efc9406b000 2024-04-22T13:32:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.996+0000 7efc8affd640 1 -- 172.21.15.5:0/722194093 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7efc8409cb90 con 0x7efc9406b000 2024-04-22T13:32:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:32:10.992 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:32:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efc800a3d20 msgr2=0x7efc800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efc800a3d20 0x7efc800a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc90005ab0 tx=0x7efc90005fd0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 msgr2=0x7efc94198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 0x7efc94198be0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efc8400e9f0 tx=0x7efc8400edf0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 shutdown_connections 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9406b000 0x7efc94198be0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efc800a3d20 0x7efc800a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc94108060 0x7efc9419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 --2- 172.21.15.5:0/722194093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9406a640 0x7efc941986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.998+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 >> 172.21.15.5:0/722194093 conn(0x7efc94077c40 msgr2=0x7efc940ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.999+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 shutdown_connections 2024-04-22T13:32:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:10.999+0000 7efc9bcc1640 1 -- 172.21.15.5:0/722194093 wait complete. 2024-04-22T13:32:11.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:32:11.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 -- 172.21.15.5:0/1734918726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103400 msgr2=0x7f2c2c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 --2- 172.21.15.5:0/1734918726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103400 0x7f2c2c10de00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009cd0 tx=0x7f2c2402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 -- 172.21.15.5:0/1734918726 shutdown_connections 2024-04-22T13:32:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 --2- 172.21.15.5:0/1734918726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103400 0x7f2c2c10de00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 --2- 172.21.15.5:0/1734918726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c2c102a60 0x7f2c2c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 --2- 172.21.15.5:0/1734918726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c101180 0x7f2c2c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 -- 172.21.15.5:0/1734918726 >> 172.21.15.5:0/1734918726 conn(0x7f2c2c077f70 msgr2=0x7f2c2c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 -- 172.21.15.5:0/1734918726 shutdown_connections 2024-04-22T13:32:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.114+0000 7f2c3446d640 1 -- 172.21.15.5:0/1734918726 wait complete. 2024-04-22T13:32:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 Processor -- start 2024-04-22T13:32:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 -- start start 2024-04-22T13:32:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 0x7f2c2c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c2c102a60 0x7f2c2c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c103400 0x7f2c2c1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111180 con 0x7f2c2c101180 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111000 con 0x7f2c2c103400 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c3446d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111300 con 0x7f2c2c102a60 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 0x7f2c2c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 0x7f2c2c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58544/0 (socket says 172.21.15.5:58544) 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 -- 172.21.15.5:0/3451750847 learned_addr learned my addr 172.21.15.5:0/3451750847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c31a08640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c2c102a60 0x7f2c2c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c103400 0x7f2c2c1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 -- 172.21.15.5:0/3451750847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c2c102a60 msgr2=0x7f2c2c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c2c102a60 0x7f2c2c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 -- 172.21.15.5:0/3451750847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c103400 msgr2=0x7f2c2c1a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c103400 0x7f2c2c1a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 -- 172.21.15.5:0/3451750847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1a4cc0 con 0x7f2c2c101180 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c103400 0x7f2c2c1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.115+0000 7f2c32209640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 0x7f2c2c199700 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800c890 tx=0x7f2c2800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.116+0000 7f2c237fe640 1 -- 172.21.15.5:0/3451750847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28004220 con 0x7f2c2c101180 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.116+0000 7f2c237fe640 1 -- 172.21.15.5:0/3451750847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c280043c0 con 0x7f2c2c101180 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.116+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1a4e90 con 0x7f2c2c101180 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.116+0000 7f2c237fe640 1 -- 172.21.15.5:0/3451750847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28004a00 con 0x7f2c2c101180 2024-04-22T13:32:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.116+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1a53a0 con 0x7f2c2c101180 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.118+0000 7f2c237fe640 1 -- 172.21.15.5:0/3451750847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2c280079b0 con 0x7f2c2c101180 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.118+0000 7f2c37066740 1 Processor -- start 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 -- start start 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 0x5641ed2fa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed303820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed303e40 0x5641ed3042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641ed25f1f0 con 0x5641ed2fa330 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5641ed1fc4b0 con 0x5641ed2face0 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c37066740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5641ed25b1b0 con 0x5641ed303e40 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c237fe640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2c180a3da0 0x7f2c180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 0x5641ed2fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 0x5641ed2fa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58546/0 (socket says 172.21.15.5:58546) 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 -- 172.21.15.5:0/310984182 learned_addr learned my addr 172.21.15.5:0/310984182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c31a08640 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed303e40 0x5641ed3042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2c180a3da0 0x7f2c180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c237fe640 1 -- 172.21.15.5:0/3451750847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2c280d0500 con 0x7f2c2c101180 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32209640 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed303820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 -- 172.21.15.5:0/310984182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed303e40 msgr2=0x5641ed3042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed303e40 0x5641ed3042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 -- 172.21.15.5:0/310984182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 msgr2=0x5641ed303820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed303820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32a0a640 1 -- 172.21.15.5:0/310984182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ed1ee1b0 con 0x5641ed2fa330 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.119+0000 7f2c32209640 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed303820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.120+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 0x5641ed2fa710 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f2c24032790 tx=0x7f2c240327c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac871556ff3462e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.120+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2c180a3da0 0x7f2c180a6260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c630 tx=0x7f2c24040000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.120+0000 7f2c20bfe640 1 -- 172.21.15.5:0/310984182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24050070 con 0x5641ed2fa330 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.120+0000 7f2c20bfe640 1 -- 172.21.15.5:0/310984182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2403a040 con 0x5641ed2fa330 2024-04-22T13:32:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.120+0000 7f2c20bfe640 1 -- 172.21.15.5:0/310984182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24039040 con 0x5641ed2fa330 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- 172.21.15.5:0/310984182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 msgr2=0x5641ed2fa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 0x5641ed2fa710 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f2c24032790 tx=0x7f2c240327c0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- 172.21.15.5:0/310984182 shutdown_connections 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed303e40 0x5641ed3042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed303820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- 172.21.15.5:0/310984182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2fa330 0x5641ed2fa710 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- 172.21.15.5:0/310984182 >> 172.21.15.5:0/310984182 conn(0x5641ed2f4ba0 msgr2=0x5641ed2f4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- 172.21.15.5:0/310984182 shutdown_connections 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- 172.21.15.5:0/310984182 wait complete. 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- start start 2024-04-22T13:32:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 Processor -- start 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed30a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed3075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed307b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641ed30ad60 con 0x5641ed303e40 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5641ed218500 con 0x5641ed2face0 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5641ed30aee0 con 0x5641ed2fa330 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c37066740 10 client.0 _ll_register_callbacks cb 0x5641ed1e6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.121+0000 7f2c32a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed307b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed307b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58548/0 (socket says 172.21.15.5:58548) 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 -- 172.21.15.5:0/2656911767 learned_addr learned my addr 172.21.15.5:0/2656911767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32209640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed30a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c31a08640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed3075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 -- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 msgr2=0x5641ed30a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed30a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 -- 172.21.15.5:0/2656911767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 msgr2=0x5641ed3075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed3075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 -- 172.21.15.5:0/2656911767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ecf713f0 con 0x5641ed303e40 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32209640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed30a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed307b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2c24032830 tx=0x7f2c240044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaa7890a4fb8992 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c15ffb640 1 -- 172.21.15.5:0/2656911767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24050070 con 0x5641ed303e40 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c15ffb640 1 -- 172.21.15.5:0/2656911767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2c24055460 con 0x5641ed303e40 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2403a040 con 0x5641ed303e40 2024-04-22T13:32:11.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c15ffb640 1 -- 172.21.15.5:0/2656911767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c24039040 con 0x5641ed303e40 2024-04-22T13:32:11.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c37066740 10 client.17610 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641ed03b7d0 con 0x5641ed303e40 2024-04-22T13:32:11.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.122+0000 7f2c15ffb640 1 -- 172.21.15.5:0/2656911767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2403a040 con 0x5641ed303e40 2024-04-22T13:32:11.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c15ffb640 1 -- 172.21.15.5:0/2656911767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2c24055b10 con 0x5641ed303e40 2024-04-22T13:32:11.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c37066740 10 client.17610 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c37066740 10 client.17610 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:32:11.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c37066740 20 client.17610 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c37066740 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed317eb0 0x5641ed31a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c37066740 4 client.17610 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641ed1fe930 con 0x5641ed317eb0 2024-04-22T13:32:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c32209640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed317eb0 0x5641ed31a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c15ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.123+0000 7f2c32209640 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed317eb0 0x5641ed31a2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc13a2ac69ee1979 server_cookie=e9697b44729884b in_seq=0 out_seq=0 2024-04-22T13:32:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c15ffb640 1 -- 172.21.15.5:0/2656911767 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641ed1fe930 con 0x5641ed317eb0 2024-04-22T13:32:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c15ffb640 10 client.17610 handle_command_reply: tid=0 2024-04-22T13:32:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 client.17610 shutdown 2024-04-22T13:32:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 msgr2=0x5641ed307b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed307b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2c24032830 tx=0x7f2c240044b0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 shutdown_connections 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed317eb0 0x5641ed31a2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed307b60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2face0 0x5641ed3075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 --2- 172.21.15.5:0/2656911767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed30a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c15ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 >> 172.21.15.5:0/2656911767 conn(0x5641ed2f4ba0 msgr2=0x5641ed2f68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 shutdown_connections 2024-04-22T13:32:11.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 1 -- 172.21.15.5:0/2656911767 wait complete. 2024-04-22T13:32:11.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.124+0000 7f2c37066740 20 client.17610 trim_cache size 0 max 16384 2024-04-22T13:32:11.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 Processor -- start 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 -- start start 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed305230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 0x5641ed313c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed303e40 0x5641ed2f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1ab960 con 0x5641ed2face0 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1ad4a0 con 0x5641ed303e40 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c37066740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1ac030 con 0x5641ed2fa330 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 0x5641ed313c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 0x5641ed313c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58560/0 (socket says 172.21.15.5:58560) 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 -- 172.21.15.5:0/1190078154 learned_addr learned my addr 172.21.15.5:0/1190078154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c32209640 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed305230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 -- 172.21.15.5:0/1190078154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 msgr2=0x5641ed305230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed305230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 -- 172.21.15.5:0/1190078154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed303e40 msgr2=0x5641ed2f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed303e40 0x5641ed2f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 -- 172.21.15.5:0/1190078154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ed30aed0 con 0x5641ed2face0 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c32209640 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed305230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.138+0000 7f2c31a08640 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 0x5641ed313c60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b690 tx=0x7f2c1c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ce1c1ed1f86457 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c17fff640 1 -- 172.21.15.5:0/1190078154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c003ad0 con 0x5641ed2face0 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c17fff640 1 -- 172.21.15.5:0/1190078154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1c0054d0 con 0x5641ed2face0 2024-04-22T13:32:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c17fff640 1 -- 172.21.15.5:0/1190078154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c004de0 con 0x5641ed2face0 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 -- 172.21.15.5:0/1190078154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 msgr2=0x5641ed313c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 0x5641ed313c60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b690 tx=0x7f2c1c00bb50 comp rx=0 tx=0).stop 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 -- 172.21.15.5:0/1190078154 shutdown_connections 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed2face0 0x5641ed313c60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed303e40 0x5641ed2f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 --2- 172.21.15.5:0/1190078154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2fa330 0x5641ed305230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 -- 172.21.15.5:0/1190078154 >> 172.21.15.5:0/1190078154 conn(0x5641ed2f4ba0 msgr2=0x5641ed2f3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 -- 172.21.15.5:0/1190078154 shutdown_connections 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 -- 172.21.15.5:0/1190078154 wait complete. 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.139+0000 7f2c37066740 1 -- start start 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 Processor -- start 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2fa330 0x5641ed310360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2face0 0x5641ed30d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed30d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1acdd0 con 0x5641ed303e40 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1ac700 con 0x5641ed2fa330 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1a9940 con 0x5641ed2face0 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c37066740 10 client.0 _ll_register_callbacks cb 0x7f2c24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed30d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed30d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58576/0 (socket says 172.21.15.5:58576) 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 -- 172.21.15.5:0/3032173603 learned_addr learned my addr 172.21.15.5:0/3032173603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c31a08640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2face0 0x5641ed30d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32209640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2fa330 0x5641ed310360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 -- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2face0 msgr2=0x5641ed30d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2face0 0x5641ed30d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 -- 172.21.15.5:0/3032173603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2fa330 msgr2=0x5641ed310360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2fa330 0x5641ed310360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 -- 172.21.15.5:0/3032173603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ed30dbc0 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c32a0a640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed30d680 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c6d0 tx=0x7f2c24032710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c726327c1ce3689c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.140+0000 7f2c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c15ffb640 1 -- 172.21.15.5:0/3032173603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24062070 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c1c0054d0 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c15ffb640 1 -- 172.21.15.5:0/3032173603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2c240043e0 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c15ffb640 1 -- 172.21.15.5:0/3032173603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c2406e050 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 10 client.17616 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641ed37eda0 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c15ffb640 1 -- 172.21.15.5:0/3032173603 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1c0054d0 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c15ffb640 1 -- 172.21.15.5:0/3032173603 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2c24055460 con 0x5641ed303e40 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 10 client.17616 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 10 client.17616 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 20 client.17616 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed37f460 0x5641ed381880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 4 client.17616 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641ed26cc10 con 0x5641ed37f460 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.141+0000 7f2c32209640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed37f460 0x5641ed381880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c32209640 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed37f460 0x5641ed381880 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dec3dab71d8c9489 server_cookie=c6f17fd7163f62f in_seq=0 out_seq=0 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c15ffb640 10 client.17616 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c15ffb640 1 -- 172.21.15.5:0/3032173603 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641ed26cc10 con 0x5641ed37f460 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c15ffb640 10 client.17616 handle_command_reply: tid=0 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 client.17616 shutdown 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 msgr2=0x5641ed30d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed30d680 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c6d0 tx=0x7f2c24032710 comp rx=0 tx=0).stop 2024-04-22T13:32:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 shutdown_connections 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5641ed37f460 0x5641ed381880 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ed303e40 0x5641ed30d680 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641ed2face0 0x5641ed30d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c15ffb640 0 client.17616 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 --2- 172.21.15.5:0/3032173603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5641ed2fa330 0x5641ed310360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 >> 172.21.15.5:0/3032173603 conn(0x5641ed2f4ba0 msgr2=0x5641ed2f3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 shutdown_connections 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 1 -- 172.21.15.5:0/3032173603 wait complete. 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.142+0000 7f2c37066740 20 client.17616 trim_cache size 0 max 16384 2024-04-22T13:32:11.156 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2c180a3da0 msgr2=0x7f2c180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2c180a3da0 0x7f2c180a6260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c630 tx=0x7f2c24040000 comp rx=0 tx=0).stop 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 msgr2=0x7f2c2c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 0x7f2c2c199700 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800c890 tx=0x7f2c2800cd50 comp rx=0 tx=0).stop 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 shutdown_connections 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2c180a3da0 0x7f2c180a6260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c2c103400 0x7f2c2c1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c2c102a60 0x7f2c2c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 --2- 172.21.15.5:0/3451750847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c101180 0x7f2c2c199700 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 >> 172.21.15.5:0/3451750847 conn(0x7f2c2c077f70 msgr2=0x7f2c2c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 shutdown_connections 2024-04-22T13:32:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.143+0000 7f2c3446d640 1 -- 172.21.15.5:0/3451750847 wait complete. 2024-04-22T13:32:11.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:32:11.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:10 smithi005 ceph-mon[28006]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:32:11.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:10 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:32:11.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:10 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:11.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:10 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:32:11.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:10 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:11.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:10 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 -- 172.21.15.5:0/3057802343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c103b70 msgr2=0x7f6b8c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 --2- 172.21.15.5:0/3057802343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c103b70 0x7f6b8c10bfc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009900 tx=0x7f6b7c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 -- 172.21.15.5:0/3057802343 shutdown_connections 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 --2- 172.21.15.5:0/3057802343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c103b70 0x7f6b8c10bfc0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 --2- 172.21.15.5:0/3057802343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 --2- 172.21.15.5:0/3057802343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 0x7f6b8c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 -- 172.21.15.5:0/3057802343 >> 172.21.15.5:0/3057802343 conn(0x7f6b8c077f70 msgr2=0x7f6b8c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 -- 172.21.15.5:0/3057802343 shutdown_connections 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.268+0000 7f6b92681640 1 -- 172.21.15.5:0/3057802343 wait complete. 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 Processor -- start 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 -- start start 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 0x7f6b8c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b8c198740 0x7f6b8c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112350 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1121d0 con 0x7f6b8c198740 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b92681640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1124d0 con 0x7f6b8c1031c0 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.269+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58594/0 (socket says 172.21.15.5:58594) 2024-04-22T13:32:11.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/552252638 learned_addr learned my addr 172.21.15.5:0/552252638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b8c198740 0x7f6b8c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 0x7f6b8c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/552252638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 msgr2=0x7f6b8c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 0x7f6b8c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/552252638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b8c198740 msgr2=0x7f6b8c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b8c198740 0x7f6b8c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/552252638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c199340 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 0x7f6b8c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c19ebc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c036040 tx=0x7f6b7c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b897fa640 1 -- 172.21.15.5:0/552252638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c043020 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1995d0 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b897fa640 1 -- 172.21.15.5:0/552252638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c02cb70 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b897fa640 1 -- 172.21.15.5:0/552252638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c003bc0 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.270+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1a57c0 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:10 smithi049 ceph-mon[31379]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:32:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:10 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:32:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:10 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:10 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:32:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:10 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:11.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:10 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.272+0000 7f6b897fa640 1 -- 172.21.15.5:0/552252638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6b7c046020 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 Processor -- start 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 -- start start 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b897fa640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6b780a3d20 0x7f6b780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b69b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b69de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b77b5f71f0 con 0x55b77b69ba00 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b77b5944b0 con 0x55b77b6924d0 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b95271740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b77b5f31b0 con 0x55b77b692e80 2024-04-22T13:32:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6b780a3d20 0x7f6b780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b897fa640 1 -- 172.21.15.5:0/552252638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f6b7c0f2820 con 0x7f6b8c10c6d0 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b90c1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b69de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b69b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36068/0 (socket says 172.21.15.5:36068) 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.273+0000 7f6b8bfff640 1 -- 172.21.15.5:0/1745616755 learned_addr learned my addr 172.21.15.5:0/1745616755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1745616755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 msgr2=0x55b77b6928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1745616755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 msgr2=0x55b77b69de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b69de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1745616755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77b5861b0 con 0x55b77b692e80 2024-04-22T13:32:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6b780a3d20 0x7f6b780a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6b84005fd0 tx=0x7f6b84009e90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b69b430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0f00b0 tx=0x7f6b7c0be790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d54b926153cfad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b827fc640 1 -- 172.21.15.5:0/1745616755 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0a25e0 con 0x55b77b692e80 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b827fc640 1 -- 172.21.15.5:0/1745616755 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c0bf000 con 0x55b77b692e80 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.274+0000 7f6b827fc640 1 -- 172.21.15.5:0/1745616755 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0a4d50 con 0x55b77b692e80 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 -- 172.21.15.5:0/1745616755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 msgr2=0x55b77b69b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b69b430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0f00b0 tx=0x7f6b7c0be790 comp rx=0 tx=0).stop 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 -- 172.21.15.5:0/1745616755 shutdown_connections 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b69de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b69b430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- 172.21.15.5:0/1745616755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 -- 172.21.15.5:0/1745616755 >> 172.21.15.5:0/1745616755 conn(0x55b77b68cba0 msgr2=0x55b77b68cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 -- 172.21.15.5:0/1745616755 shutdown_connections 2024-04-22T13:32:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 -- 172.21.15.5:0/1745616755 wait complete. 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 -- start start 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 Processor -- start 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b69b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b698650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.275+0000 7f6b95271740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b698bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b95271740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b77b69e730 con 0x55b77b69ba00 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b95271740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b77b5b0500 con 0x55b77b6924d0 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b95271740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b77b69e8b0 con 0x55b77b692e80 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b95271740 10 client.0 _ll_register_callbacks cb 0x55b77b57eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b90c1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b698bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b698650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b698650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:36576/0 (socket says 172.21.15.5:36576) 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b90c1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b698bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58614/0 (socket says 172.21.15.5:58614) 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/2318450696 learned_addr learned my addr 172.21.15.5:0/2318450696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b69b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 msgr2=0x55b77b69b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b69b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/2318450696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 msgr2=0x55b77b698bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b698bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/2318450696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77b3093f0 con 0x55b77b692e80 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b698bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b698650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c000c00 tx=0x7f6b7c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238a4373867c41a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/2318450696 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0be900 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/2318450696 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f6b7c0a90b0 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.276+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c0bf000 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/2318450696 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b7c0c2cb0 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 10 client.27004 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b77b3d37d0 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/2318450696 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c0bf000 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/2318450696 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6b7c0c71e0 con 0x55b77b692e80 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 10 client.27004 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 10 client.27004 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 20 client.27004 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b6ab790 0x55b77b6adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 4 client.27004 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77b596930 con 0x55b77b6ab790 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.277+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b6ab790 0x55b77b6adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.278+0000 7f6b6b7fe640 10 client.27004 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.278+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b6ab790 0x55b77b6adb80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1211ea7364273413 server_cookie=4337cf35fd2bd328 in_seq=0 out_seq=0 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.278+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/2318450696 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b77b596930 con 0x55b77b6ab790 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b6b7fe640 10 client.27004 handle_command_reply: tid=0 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 client.27004 shutdown 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 msgr2=0x55b77b698650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b698650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c000c00 tx=0x7f6b7c009260 comp rx=0 tx=0).stop 2024-04-22T13:32:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 shutdown_connections 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b6ab790 0x55b77b6adb80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b69ba00 0x55b77b698bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b6b7fe640 0 client.27004 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b692e80 0x55b77b698650 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 --2- 172.21.15.5:0/2318450696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b69b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 >> 172.21.15.5:0/2318450696 conn(0x55b77b68cba0 msgr2=0x55b77b68e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 shutdown_connections 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 1 -- 172.21.15.5:0/2318450696 wait complete. 2024-04-22T13:32:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.279+0000 7f6b95271740 20 client.27004 trim_cache size 0 max 16384 2024-04-22T13:32:11.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:10 smithi107 ceph-mon[31859]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:32:11.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:10 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:32:11.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:10 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:11.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:10 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:32:11.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:10 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:11.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:10 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:32:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 Processor -- start 2024-04-22T13:32:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 -- start start 2024-04-22T13:32:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b68b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b69ba00 0x55b77b69ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1adb70 con 0x55b77b692e80 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1af6b0 con 0x55b77b6924d0 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b95271740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1ae240 con 0x55b77b69ba00 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.292+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b68b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b68b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58616/0 (socket says 172.21.15.5:58616) 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1884552342 learned_addr learned my addr 172.21.15.5:0/1884552342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b69ba00 0x55b77b69ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1884552342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b69ba00 msgr2=0x55b77b69ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b69ba00 0x55b77b69ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1884552342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 msgr2=0x55b77b6a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/1884552342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77b68c220 con 0x55b77b692e80 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b69ba00 0x55b77b69ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b68b370 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c005ec0 tx=0x7f6b7c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fe31124c9a15eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b81ffb640 1 -- 172.21.15.5:0/1884552342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0be260 con 0x55b77b692e80 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b81ffb640 1 -- 172.21.15.5:0/1884552342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c0bf000 con 0x55b77b692e80 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.293+0000 7f6b81ffb640 1 -- 172.21.15.5:0/1884552342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0a90b0 con 0x55b77b692e80 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 -- 172.21.15.5:0/1884552342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 msgr2=0x55b77b68b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b68b370 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c005ec0 tx=0x7f6b7c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 -- 172.21.15.5:0/1884552342 shutdown_connections 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b68b370 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b69ba00 0x55b77b69ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 --2- 172.21.15.5:0/1884552342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b6924d0 0x55b77b6a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 -- 172.21.15.5:0/1884552342 >> 172.21.15.5:0/1884552342 conn(0x55b77b68cba0 msgr2=0x55b77b697680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 -- 172.21.15.5:0/1884552342 shutdown_connections 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 -- 172.21.15.5:0/1884552342 wait complete. 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 -- start start 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.294+0000 7f6b95271740 1 Processor -- start 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b6924d0 0x55b77b6a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b6a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b69ba00 0x55b77b694e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1aefe0 con 0x55b77b692e80 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1ae910 con 0x55b77b69ba00 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1abb70 con 0x55b77b6924d0 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 10 client.0 _ll_register_callbacks cb 0x7f6b7c0baba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b6a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b6a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58630/0 (socket says 172.21.15.5:58630) 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/603608678 learned_addr learned my addr 172.21.15.5:0/603608678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b6924d0 0x55b77b6a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b90c1e640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b69ba00 0x55b77b694e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/603608678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b6924d0 msgr2=0x55b77b6a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b6924d0 0x55b77b6a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b69ba00 msgr2=0x55b77b694e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b69ba00 0x55b77b694e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/603608678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77b6956b0 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b8b7fe640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b6a3f80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0bad60 tx=0x7f6b7c0ba6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e43197d157304a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/603608678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0be260 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/603608678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f6b7c0c70e0 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c0bf000 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/603608678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b7c0be900 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 10 client.17634 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.295+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b77b6956b0 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/603608678 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c0bf000 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/603608678 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6b7c0bd7c0 con 0x55b77b692e80 2024-04-22T13:32:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b95271740 10 client.17634 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b95271740 10 client.17634 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b95271740 20 client.17634 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b95271740 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b711a80 0x55b77b713ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b95271740 4 client.17634 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b77b604c10 con 0x55b77b711a80 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.296+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b711a80 0x55b77b713ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b711a80 0x55b77b713ea0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19124e371d515dea server_cookie=cd5a966b372b9f0f in_seq=0 out_seq=0 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b6b7fe640 10 client.17634 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b6b7fe640 1 -- 172.21.15.5:0/603608678 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b77b604c10 con 0x55b77b711a80 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b6b7fe640 10 client.17634 handle_command_reply: tid=0 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 client.17634 shutdown 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 msgr2=0x55b77b6a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b6a3f80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0bad60 tx=0x7f6b7c0ba6d0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 shutdown_connections 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b77b711a80 0x55b77b713ea0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77b692e80 0x55b77b6a3f80 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b77b69ba00 0x55b77b694e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 --2- 172.21.15.5:0/603608678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77b6924d0 0x55b77b6a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b6b7fe640 0 client.17634 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 >> 172.21.15.5:0/603608678 conn(0x55b77b68cba0 msgr2=0x55b77b68e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 shutdown_connections 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 1 -- 172.21.15.5:0/603608678 wait complete. 2024-04-22T13:32:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.297+0000 7f6b95271740 20 client.17634 trim_cache size 0 max 16384 2024-04-22T13:32:11.301 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6b780a3d20 msgr2=0x7f6b780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6b780a3d20 0x7f6b780a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6b84005fd0 tx=0x7f6b84009e90 comp rx=0 tx=0).stop 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 msgr2=0x7f6b8c19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c19ebc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c036040 tx=0x7f6b7c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 shutdown_connections 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c10c6d0 0x7f6b8c19ebc0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6b780a3d20 0x7f6b780a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b8c198740 0x7f6b8c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 --2- 172.21.15.5:0/552252638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8c1031c0 0x7f6b8c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 >> 172.21.15.5:0/552252638 conn(0x7f6b8c077f70 msgr2=0x7f6b8c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 shutdown_connections 2024-04-22T13:32:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.298+0000 7f6b92681640 1 -- 172.21.15.5:0/552252638 wait complete. 2024-04-22T13:32:11.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:32:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.452+0000 7efdf08eb640 1 -- 172.21.15.5:0/326186452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec1044f0 msgr2=0x7efdec10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 --2- 172.21.15.5:0/326186452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec1044f0 0x7efdec10eef0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7efde4009cd0 tx=0x7efde402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 -- 172.21.15.5:0/326186452 shutdown_connections 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 --2- 172.21.15.5:0/326186452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec1044f0 0x7efdec10eef0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 --2- 172.21.15.5:0/326186452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec103b50 0x7efdec103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 --2- 172.21.15.5:0/326186452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec102270 0x7efdec102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 -- 172.21.15.5:0/326186452 >> 172.21.15.5:0/326186452 conn(0x7efdec0fda10 msgr2=0x7efdec0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 -- 172.21.15.5:0/326186452 shutdown_connections 2024-04-22T13:32:11.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 -- 172.21.15.5:0/326186452 wait complete. 2024-04-22T13:32:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 Processor -- start 2024-04-22T13:32:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.453+0000 7efdf08eb640 1 -- start start 2024-04-22T13:32:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 0x7efdec19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec103b50 0x7efdec19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1044f0 0x7efdec1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdec112290 con 0x7efdec102270 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdec112110 con 0x7efdec103b50 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdec112410 con 0x7efdec1044f0 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 0x7efdec19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 0x7efdec19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58646/0 (socket says 172.21.15.5:58646) 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 -- 172.21.15.5:0/2401771494 learned_addr learned my addr 172.21.15.5:0/2401771494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efde9d9b640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec103b50 0x7efdec19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 -- 172.21.15.5:0/2401771494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1044f0 msgr2=0x7efdec1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdead9d640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1044f0 0x7efdec1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1044f0 0x7efdec1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 -- 172.21.15.5:0/2401771494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec103b50 msgr2=0x7efdec19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec103b50 0x7efdec19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 -- 172.21.15.5:0/2401771494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec198d70 con 0x7efdec102270 2024-04-22T13:32:11.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efde9d9b640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec103b50 0x7efdec19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdea59c640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 0x7efdec19e440 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7efddc00d950 tx=0x7efddc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efddb7fe640 1 -- 172.21.15.5:0/2401771494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc017070 con 0x7efdec102270 2024-04-22T13:32:11.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec199060 con 0x7efdec102270 2024-04-22T13:32:11.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efddb7fe640 1 -- 172.21.15.5:0/2401771494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc009c30 con 0x7efdec102270 2024-04-22T13:32:11.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.454+0000 7efddb7fe640 1 -- 172.21.15.5:0/2401771494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc00f620 con 0x7efdec102270 2024-04-22T13:32:11.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.455+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec1a5300 con 0x7efdec102270 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efddb7fe640 1 -- 172.21.15.5:0/2401771494 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efddc010460 con 0x7efdec102270 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 Processor -- start 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 -- start start 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb86b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557cb86051f0 con 0x557cb86a5290 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557cb85a24b0 con 0x557cb86a48e0 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.457+0000 7efdf35ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cb86011b0 con 0x557cb86ade10 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58656/0 (socket says 172.21.15.5:58656) 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 -- 172.21.15.5:0/4188925732 learned_addr learned my addr 172.21.15.5:0/4188925732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdead9d640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:36132/0 (socket says 172.21.15.5:36132) 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efddb7fe640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efdd40a3d20 0x7efdd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efde9d9b640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb86b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 -- 172.21.15.5:0/4188925732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 msgr2=0x557cb86b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb86b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 -- 172.21.15.5:0/4188925732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 msgr2=0x557cb86a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 -- 172.21.15.5:0/4188925732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb85941b0 con 0x557cb86a5290 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdead9d640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efde9d9b640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb86b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdead9d640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efdd40a3d20 0x7efdd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efddb7fe640 1 -- 172.21.15.5:0/2401771494 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efddc0d04e0 con 0x7efdec102270 2024-04-22T13:32:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdea59c640 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86ad840 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efddc0cdbd0 tx=0x7efddc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d97bdf8141427e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdd8ff9640 1 -- 172.21.15.5:0/4188925732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc09bff0 con 0x557cb86a5290 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdead9d640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efdd40a3d20 0x7efdd40a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efde402c700 tx=0x7efde40095c0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdd8ff9640 1 -- 172.21.15.5:0/4188925732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc09cbb0 con 0x557cb86a5290 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.458+0000 7efdd8ff9640 1 -- 172.21.15.5:0/4188925732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc09be10 con 0x557cb86a5290 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 -- 172.21.15.5:0/4188925732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 msgr2=0x557cb86ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86ad840 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efddc0cdbd0 tx=0x7efddc022000 comp rx=0 tx=0).stop 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 -- 172.21.15.5:0/4188925732 shutdown_connections 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86ad840 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb86b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 --2- 172.21.15.5:0/4188925732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 -- 172.21.15.5:0/4188925732 >> 172.21.15.5:0/4188925732 conn(0x557cb869efb0 msgr2=0x557cb869f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 -- 172.21.15.5:0/4188925732 shutdown_connections 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.459+0000 7efdf35ba740 1 -- 172.21.15.5:0/4188925732 wait complete. 2024-04-22T13:32:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 -- start start 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 Processor -- start 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 0x557cb86aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557cb86b0d20 con 0x557cb86ade10 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557cb85be500 con 0x557cb86a48e0 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cb86b0ea0 con 0x557cb86a5290 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdf35ba740 10 client.0 _ll_register_callbacks cb 0x557cb858cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 0x557cb86aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 0x557cb86aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58664/0 (socket says 172.21.15.5:58664) 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 -- 172.21.15.5:0/669288074 learned_addr learned my addr 172.21.15.5:0/669288074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efde9d9b640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdea59c640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 -- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 msgr2=0x557cb86aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 -- 172.21.15.5:0/669288074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 msgr2=0x557cb86ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efdead9d640 1 -- 172.21.15.5:0/669288074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb83173f0 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.460+0000 7efde9d9b640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdead9d640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 0x557cb86aaed0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efde4036000 tx=0x7efde4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972ee1a73108578a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdd1ffb640 1 -- 172.21.15.5:0/669288074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4045070 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdd1ffb640 1 -- 172.21.15.5:0/669288074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efde4004450 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efddc09cbb0 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdd1ffb640 1 -- 172.21.15.5:0/669288074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efde405a050 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 10 client.17646 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cb83e17d0 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdd1ffb640 1 -- 172.21.15.5:0/669288074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc09cbb0 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdd1ffb640 1 -- 172.21.15.5:0/669288074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efde403f040 con 0x557cb86ade10 2024-04-22T13:32:11.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 10 client.17646 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 10 client.17646 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 20 client.17646 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb86bdf50 0x557cb86c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 4 client.17646 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.461+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cb85a4930 con 0x557cb86bdf50 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.462+0000 7efdea59c640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb86bdf50 0x557cb86c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.462+0000 7efdd1ffb640 10 client.17646 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.462+0000 7efdea59c640 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb86bdf50 0x557cb86c0390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=732ca89825304252 server_cookie=fbaecbdfb2de630d in_seq=0 out_seq=0 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdd1ffb640 1 -- 172.21.15.5:0/669288074 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557cb85a4930 con 0x557cb86bdf50 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdd1ffb640 10 client.17646 handle_command_reply: tid=0 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 client.17646 shutdown 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 msgr2=0x557cb86aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 0x557cb86aaed0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efde4036000 tx=0x7efde4005d80 comp rx=0 tx=0).stop 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 shutdown_connections 2024-04-22T13:32:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb86bdf50 0x557cb86c0390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86ade10 0x557cb86aaed0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdd1ffb640 0 client.17646 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 --2- 172.21.15.5:0/669288074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 >> 172.21.15.5:0/669288074 conn(0x557cb869efb0 msgr2=0x557cb86a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 shutdown_connections 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 1 -- 172.21.15.5:0/669288074 wait complete. 2024-04-22T13:32:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.463+0000 7efdf35ba740 20 client.17646 trim_cache size 0 max 16384 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 Processor -- start 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 -- start start 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 0x557cb86b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86ade10 0x557cb8699a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1ab860 con 0x557cb86a48e0 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1ad3a0 con 0x557cb86ade10 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdf35ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1abf30 con 0x557cb86a5290 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 0x557cb86b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 0x557cb86b9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58670/0 (socket says 172.21.15.5:58670) 2024-04-22T13:32:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 -- 172.21.15.5:0/2662901254 learned_addr learned my addr 172.21.15.5:0/2662901254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efde9d9b640 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 -- 172.21.15.5:0/2662901254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 msgr2=0x557cb86a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 -- 172.21.15.5:0/2662901254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86ade10 msgr2=0x557cb8699a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86ade10 0x557cb8699a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 -- 172.21.15.5:0/2662901254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb86ab4d0 con 0x557cb86a48e0 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efde9d9b640 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.477+0000 7efdea59c640 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 0x557cb86b9c10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ce260 tx=0x7efddc010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa14ff1b08c613d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdd3fff640 1 -- 172.21.15.5:0/2662901254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc009de0 con 0x557cb86a48e0 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdd3fff640 1 -- 172.21.15.5:0/2662901254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc09c4b0 con 0x557cb86a48e0 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdd3fff640 1 -- 172.21.15.5:0/2662901254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc09a840 con 0x557cb86a48e0 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 -- 172.21.15.5:0/2662901254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 msgr2=0x557cb86b9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 0x557cb86b9c10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ce260 tx=0x7efddc010cb0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 -- 172.21.15.5:0/2662901254 shutdown_connections 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86ade10 0x557cb8699a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86a5290 0x557cb86a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 --2- 172.21.15.5:0/2662901254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a48e0 0x557cb86b9c10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 -- 172.21.15.5:0/2662901254 >> 172.21.15.5:0/2662901254 conn(0x557cb869efb0 msgr2=0x557cb8699370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 -- 172.21.15.5:0/2662901254 shutdown_connections 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 -- 172.21.15.5:0/2662901254 wait complete. 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.478+0000 7efdf35ba740 1 -- start start 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 Processor -- start 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb87224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1accd0 con 0x557cb86a5290 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1ac600 con 0x557cb86a48e0 2024-04-22T13:32:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1a9840 con 0x557cb86ade10 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdf35ba740 10 client.0 _ll_register_callbacks cb 0x7efde4032930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58686/0 (socket says 172.21.15.5:58686) 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 -- 172.21.15.5:0/151838557 learned_addr learned my addr 172.21.15.5:0/151838557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdead9d640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb87224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdea59c640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 -- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 msgr2=0x557cb87224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb87224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 -- 172.21.15.5:0/151838557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 msgr2=0x557cb86b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 -- 172.21.15.5:0/151838557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb8722ab0 con 0x557cb86a5290 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efde9d9b640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86b1810 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7efdec199730 tx=0x7efde0012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b80285887ed52b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.479+0000 7efdd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdd1ffb640 1 -- 172.21.15.5:0/151838557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0010c90 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efddc09c4b0 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdd1ffb640 1 -- 172.21.15.5:0/151838557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7efde0004020 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdd1ffb640 1 -- 172.21.15.5:0/151838557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efde0010e30 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 10 client.17652 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cb8723dc0 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdd1ffb640 1 -- 172.21.15.5:0/151838557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc09c4b0 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdd1ffb640 1 -- 172.21.15.5:0/151838557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efde0024430 con 0x557cb86a5290 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 10 client.17652 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 10 client.17652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 20 client.17652 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb87245f0 0x557cb8726a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 4 client.17652 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cb8612c10 con 0x557cb87245f0 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.480+0000 7efdea59c640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb87245f0 0x557cb8726a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdd1ffb640 10 client.17652 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdea59c640 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb87245f0 0x557cb8726a10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6292453f57b8ee90 server_cookie=4588b7190f380626 in_seq=0 out_seq=0 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdd1ffb640 1 -- 172.21.15.5:0/151838557 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cb8612c10 con 0x557cb87245f0 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdd1ffb640 10 client.17652 handle_command_reply: tid=0 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 client.17652 shutdown 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 msgr2=0x557cb86b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86b1810 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7efdec199730 tx=0x7efde0012ed0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 shutdown_connections 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x557cb87245f0 0x557cb8726a10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cb86a5290 0x557cb86b1810 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb86ade10 0x557cb87224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdd1ffb640 0 client.17652 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 --2- 172.21.15.5:0/151838557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cb86a48e0 0x557cb86b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 >> 172.21.15.5:0/151838557 conn(0x557cb869efb0 msgr2=0x557cb86a99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 shutdown_connections 2024-04-22T13:32:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 1 -- 172.21.15.5:0/151838557 wait complete. 2024-04-22T13:32:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.481+0000 7efdf35ba740 20 client.17652 trim_cache size 0 max 16384 2024-04-22T13:32:11.484 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efdd40a3d20 msgr2=0x7efdd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efdd40a3d20 0x7efdd40a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efde402c700 tx=0x7efde40095c0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 msgr2=0x7efdec19e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 0x7efdec19e440 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7efddc00d950 tx=0x7efddc00de10 comp rx=0 tx=0).stop 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 shutdown_connections 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efdd40a3d20 0x7efdd40a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1044f0 0x7efdec1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdec103b50 0x7efdec19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 --2- 172.21.15.5:0/2401771494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdec102270 0x7efdec19e440 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 >> 172.21.15.5:0/2401771494 conn(0x7efdec0fda10 msgr2=0x7efdec0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 shutdown_connections 2024-04-22T13:32:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.482+0000 7efdf08eb640 1 -- 172.21.15.5:0/2401771494 wait complete. 2024-04-22T13:32:11.488 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 -- 172.21.15.5:0/3815711543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 msgr2=0x7fcfb410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/3815711543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb410eb60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009ca0 tx=0x7fcfac02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 -- 172.21.15.5:0/3815711543 shutdown_connections 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/3815711543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb410eb60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/3815711543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103b70 0x7fcfb410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/3815711543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfb41031c0 0x7fcfb41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 -- 172.21.15.5:0/3815711543 >> 172.21.15.5:0/3815711543 conn(0x7fcfb4077f70 msgr2=0x7fcfb4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 -- 172.21.15.5:0/3815711543 shutdown_connections 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 -- 172.21.15.5:0/3815711543 wait complete. 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.597+0000 7fcfba1e6640 1 Processor -- start 2024-04-22T13:32:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 -- start start 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfb41031c0 0x7fcfb4100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4101890 0x7fcfb41a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41123a0 con 0x7fcfb410c6d0 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4112220 con 0x7fcfb41031c0 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfba1e6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4112520 con 0x7fcfb4101890 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb4101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58696/0 (socket says 172.21.15.5:58696) 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfb41031c0 0x7fcfb4100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/683308787 learned_addr learned my addr 172.21.15.5:0/683308787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4101890 0x7fcfb41a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/683308787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4101890 msgr2=0x7fcfb41a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4101890 0x7fcfb41a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/683308787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfb41031c0 msgr2=0x7fcfb4100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfb41031c0 0x7fcfb4100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.598+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/683308787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb41a71d0 con 0x7fcfb410c6d0 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.599+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb4101350 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800d750 tx=0x7fcfa800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.599+0000 7fcfb0ff9640 1 -- 172.21.15.5:0/683308787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8017070 con 0x7fcfb410c6d0 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.599+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41a74c0 con 0x7fcfb410c6d0 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.599+0000 7fcfb0ff9640 1 -- 172.21.15.5:0/683308787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa80054d0 con 0x7fcfb410c6d0 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.599+0000 7fcfb0ff9640 1 -- 172.21.15.5:0/683308787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8004c30 con 0x7fcfb410c6d0 2024-04-22T13:32:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.599+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a7a10 con 0x7fcfb410c6d0 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.601+0000 7fcfb0ff9640 1 -- 172.21.15.5:0/683308787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fcfa8004dd0 con 0x7fcfb410c6d0 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.601+0000 7fcfbcdd1740 1 Processor -- start 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.601+0000 7fcfbcdd1740 1 -- start start 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a93430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a95e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dab19ef1f0 con 0x55dab1a8ae80 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dab198c4b0 con 0x55dab1a93a00 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dab19eb1b0 con 0x55dab1a8a4d0 2024-04-22T13:32:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a93430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb0ff9640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcf9c0a3d50 0x7fcf9c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a93430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58712/0 (socket says 172.21.15.5:58712) 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 -- 172.21.15.5:0/1794150847 learned_addr learned my addr 172.21.15.5:0/1794150847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcf9c0a3d50 0x7fcf9c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a95e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb0ff9640 1 -- 172.21.15.5:0/683308787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fcfa80d0090 con 0x7fcfb410c6d0 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 -- 172.21.15.5:0/1794150847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 msgr2=0x55dab1a8a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a8a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 -- 172.21.15.5:0/1794150847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 msgr2=0x55dab1a95e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a95e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb37fe640 1 -- 172.21.15.5:0/1794150847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab197e1b0 con 0x55dab1a8ae80 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.602+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a95e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a93430 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009cf0 tx=0x7fcfa4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937e81d75f0531a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcf9c0a3d50 0x7fcf9c0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcfac004af0 tx=0x7fcfac005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcf99ffb640 1 -- 172.21.15.5:0/1794150847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004230 con 0x55dab1a8ae80 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcf99ffb640 1 -- 172.21.15.5:0/1794150847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa40043d0 con 0x55dab1a8ae80 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcf99ffb640 1 -- 172.21.15.5:0/1794150847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40197c0 con 0x55dab1a8ae80 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/1794150847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 msgr2=0x55dab1a93430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a93430 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009cf0 tx=0x7fcfa4011810 comp rx=0 tx=0).stop 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/1794150847 shutdown_connections 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a93430 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a95e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/1794150847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a8a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/1794150847 >> 172.21.15.5:0/1794150847 conn(0x55dab1a84ba0 msgr2=0x55dab1a84f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/1794150847 shutdown_connections 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.603+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/1794150847 wait complete. 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 -- start start 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 Processor -- start 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8a4d0 0x55dab1a93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8ae80 0x55dab1a90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a90b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dab1a96970 con 0x55dab1a93a00 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dab19a8500 con 0x55dab1a8a4d0 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dab1a96af0 con 0x55dab1a8ae80 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfbcdd1740 10 client.0 _ll_register_callbacks cb 0x55dab1976b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a90b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a90b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58722/0 (socket says 172.21.15.5:58722) 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3322395140 learned_addr learned my addr 172.21.15.5:0/3322395140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8a4d0 0x55dab1a93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.604+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8ae80 0x55dab1a90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3322395140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8ae80 msgr2=0x55dab1a90620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8ae80 0x55dab1a90620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8a4d0 msgr2=0x55dab1a93290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8a4d0 0x55dab1a93290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3322395140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab17013f0 con 0x55dab1a93a00 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a90b60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcfac036000 tx=0x7fcfac0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ecd32c2e884064 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcf92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3322395140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac054060 con 0x55dab1a93a00 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3322395140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fcfac032bc0 con 0x55dab1a93a00 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa40043d0 con 0x55dab1a93a00 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3322395140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfac039040 con 0x55dab1a93a00 2024-04-22T13:32:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfbcdd1740 10 client.17664 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dab17cb7d0 con 0x55dab1a93a00 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.605+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3322395140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa40043d0 con 0x55dab1a93a00 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3322395140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcfac04f710 con 0x55dab1a93a00 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfbcdd1740 10 client.17664 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfbcdd1740 10 client.17664 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfbcdd1740 20 client.17664 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1aa3b40 0x55dab1aa5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfbcdd1740 4 client.17664 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab198e930 con 0x55dab1aa3b40 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1aa3b40 0x55dab1aa5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcf92ffd640 10 client.17664 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.606+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1aa3b40 0x55dab1aa5f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4d783814e9ab2bf server_cookie=965bc2b46b2b0b73 in_seq=0 out_seq=0 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3322395140 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dab198e930 con 0x55dab1aa3b40 2024-04-22T13:32:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcf92ffd640 10 client.17664 handle_command_reply: tid=0 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 client.17664 shutdown 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 msgr2=0x55dab1a90b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a90b60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcfac036000 tx=0x7fcfac0095c0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 shutdown_connections 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1aa3b40 0x55dab1aa5f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a90b60 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8ae80 0x55dab1a90620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcf92ffd640 0 client.17664 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3322395140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8a4d0 0x55dab1a93290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 >> 172.21.15.5:0/3322395140 conn(0x55dab1a84ba0 msgr2=0x55dab1a868f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 shutdown_connections 2024-04-22T13:32:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.607+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3322395140 wait complete. 2024-04-22T13:32:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.608+0000 7fcfbcdd1740 20 client.17664 trim_cache size 0 max 16384 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 Processor -- start 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 -- start start 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a92d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41afd60 con 0x55dab1a8ae80 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b18a0 con 0x55dab1a93a00 2024-04-22T13:32:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b0430 con 0x55dab1a8a4d0 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a83a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58730/0 (socket says 172.21.15.5:58730) 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/4192095443 learned_addr learned my addr 172.21.15.5:0/4192095443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/4192095443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 msgr2=0x55dab1a9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a92d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/4192095443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 msgr2=0x55dab1a92d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a92d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.621+0000 7fcfb2ffd640 1 -- 172.21.15.5:0/4192095443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab1a8eb70 con 0x55dab1a8ae80 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a83a80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80cd870 tx=0x7fcfa8016500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6417fed5366d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcf997fa640 1 -- 172.21.15.5:0/4192095443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8020040 con 0x55dab1a8ae80 2024-04-22T13:32:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcf997fa640 1 -- 172.21.15.5:0/4192095443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa80a0410 con 0x55dab1a8ae80 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcf997fa640 1 -- 172.21.15.5:0/4192095443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80a01f0 con 0x55dab1a8ae80 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/4192095443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 msgr2=0x55dab1a83a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a83a80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80cd870 tx=0x7fcfa8016500 comp rx=0 tx=0).stop 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/4192095443 shutdown_connections 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a8ae80 0x55dab1a83a80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a93a00 0x55dab1a92d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/4192095443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/4192095443 >> 172.21.15.5:0/4192095443 conn(0x55dab1a84ba0 msgr2=0x55dab1a83370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/4192095443 shutdown_connections 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.622+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/4192095443 wait complete. 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 -- start start 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 Processor -- start 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8ae80 0x55dab1a97510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a97a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b11d0 con 0x55dab1a93a00 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b0b00 con 0x55dab1a8ae80 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41add60 con 0x55dab1a8a4d0 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfbcdd1740 10 client.0 _ll_register_callbacks cb 0x7fcfac0328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a97a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a97a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58732/0 (socket says 172.21.15.5:58732) 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3989861973 learned_addr learned my addr 172.21.15.5:0/3989861973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3989861973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 msgr2=0x55dab1a9c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8ae80 msgr2=0x55dab1a97510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb2ffd640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8ae80 0x55dab1a97510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8ae80 0x55dab1a97510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.623+0000 7fcfb3fff640 1 -- 172.21.15.5:0/3989861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab1a982c0 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfb3fff640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a97a90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcfac02c630 tx=0x7fcfac04f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9ed9352f129ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcf92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3989861973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac054070 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3989861973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fcfac04f890 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa80a0410 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3989861973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfac06a050 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 10 client.17670 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dab1b0a150 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3989861973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa80a0410 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3989861973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcfac061070 con 0x55dab1a93a00 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 10 client.17670 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 10 client.17670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:32:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 20 client.17670 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1b0a870 0x55dab1b0cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 4 client.17670 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.624+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dab1a90ef0 con 0x55dab1b0a870 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1b0a870 0x55dab1b0cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfb37fe640 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1b0a870 0x55dab1b0cc90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a842607c1a9211a0 server_cookie=fdc04235bb789c08 in_seq=0 out_seq=0 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcf92ffd640 10 client.17670 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcf92ffd640 1 -- 172.21.15.5:0/3989861973 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dab1a90ef0 con 0x55dab1b0a870 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcf92ffd640 10 client.17670 handle_command_reply: tid=0 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 client.17670 shutdown 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 msgr2=0x55dab1a97a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a97a90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcfac02c630 tx=0x7fcfac04f6a0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 shutdown_connections 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55dab1b0a870 0x55dab1b0cc90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dab1a93a00 0x55dab1a97a90 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcf92ffd640 0 client.17670 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dab1a8ae80 0x55dab1a97510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 --2- 172.21.15.5:0/3989861973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab1a8a4d0 0x55dab1a9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 >> 172.21.15.5:0/3989861973 conn(0x55dab1a84ba0 msgr2=0x55dab1a86880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 shutdown_connections 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.625+0000 7fcfbcdd1740 1 -- 172.21.15.5:0/3989861973 wait complete. 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfbcdd1740 20 client.17670 trim_cache size 0 max 16384 2024-04-22T13:32:11.628 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcf9c0a3d50 msgr2=0x7fcf9c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcf9c0a3d50 0x7fcf9c0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcfac004af0 tx=0x7fcfac005c50 comp rx=0 tx=0).stop 2024-04-22T13:32:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 msgr2=0x7fcfb4101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb4101350 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800d750 tx=0x7fcfa800dc10 comp rx=0 tx=0).stop 2024-04-22T13:32:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 shutdown_connections 2024-04-22T13:32:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb410c6d0 0x7fcfb4101350 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fcf9c0a3d50 0x7fcf9c0a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4101890 0x7fcfb41a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 --2- 172.21.15.5:0/683308787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfb41031c0 0x7fcfb4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 >> 172.21.15.5:0/683308787 conn(0x7fcfb4077f70 msgr2=0x7fcfb40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 shutdown_connections 2024-04-22T13:32:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.626+0000 7fcfba1e6640 1 -- 172.21.15.5:0/683308787 wait complete. 2024-04-22T13:32:11.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:32:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.741+0000 7f829410e640 1 -- 172.21.15.5:0/433569738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 msgr2=0x7f828c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.741+0000 7f829410e640 1 --2- 172.21.15.5:0/433569738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c101470 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8284009900 tx=0x7f828402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- 172.21.15.5:0/433569738 shutdown_connections 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 --2- 172.21.15.5:0/433569738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c106160 0x7f828c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 --2- 172.21.15.5:0/433569738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f828c101a40 0x7f828c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 --2- 172.21.15.5:0/433569738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c101470 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- 172.21.15.5:0/433569738 >> 172.21.15.5:0/433569738 conn(0x7f828c078000 msgr2=0x7f828c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- 172.21.15.5:0/433569738 shutdown_connections 2024-04-22T13:32:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- 172.21.15.5:0/433569738 wait complete. 2024-04-22T13:32:11.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 Processor -- start 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- start start 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f828c106160 0x7f828c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c19a050 0x7f828c1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1110a0 con 0x7f828c101090 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f828c110f20 con 0x7f828c106160 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.742+0000 7f829410e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f828c111220 con 0x7f828c19a050 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58750/0 (socket says 172.21.15.5:58750) 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 -- 172.21.15.5:0/2237407827 learned_addr learned my addr 172.21.15.5:0/2237407827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f82916a9640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f828c106160 0x7f828c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f82926ab640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c19a050 0x7f828c1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 -- 172.21.15.5:0/2237407827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c19a050 msgr2=0x7f828c1a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c19a050 0x7f828c1a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 -- 172.21.15.5:0/2237407827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f828c106160 msgr2=0x7f828c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f828c106160 0x7f828c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 -- 172.21.15.5:0/2237407827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c1a4e00 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f82926ab640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c19a050 0x7f828c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8291eaa640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c199580 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8284005940 tx=0x7f828402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8282ffd640 1 -- 172.21.15.5:0/2237407827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284043020 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c1a5090 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8282ffd640 1 -- 172.21.15.5:0/2237407827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828402cbe0 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f8282ffd640 1 -- 172.21.15.5:0/2237407827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284003bf0 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.743+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c1a5630 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8282ffd640 1 -- 172.21.15.5:0/2237407827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8284046020 con 0x7f828c101090 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 Processor -- start 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 -- start start 2024-04-22T13:32:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da01253b0 0x562da0125790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da012e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da012ee60 0x562da012f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562da008a1f0 con 0x562da0125d60 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562da00274b0 con 0x562da012ee60 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8296ddc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562da00861b0 con 0x562da01253b0 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8282ffd640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82780a3d20 0x7f82780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.746+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da012e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da012e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58764/0 (socket says 172.21.15.5:58764) 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 -- 172.21.15.5:0/2246949965 learned_addr learned my addr 172.21.15.5:0/2246949965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f82916a9640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da012ee60 0x562da012f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f82926ab640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da01253b0 0x562da0125790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f82926ab640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82780a3d20 0x7f82780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8282ffd640 1 -- 172.21.15.5:0/2237407827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f82840f2930 con 0x7f828c101090 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 -- 172.21.15.5:0/2246949965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da01253b0 msgr2=0x562da0125790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da01253b0 0x562da0125790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 -- 172.21.15.5:0/2246949965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da012ee60 msgr2=0x562da012f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da012ee60 0x562da012f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 -- 172.21.15.5:0/2246949965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da00191b0 con 0x562da0125d60 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f82916a9640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da012ee60 0x562da012f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f82926ab640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da01253b0 0x562da0125790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8291eaa640 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da012e890 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f8284006010 tx=0x7f82840be910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec263816d1c3c004 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8277fff640 1 -- 172.21.15.5:0/2246949965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840c2dc0 con 0x562da0125d60 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f82926ab640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82780a3d20 0x7f82780a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8288005fd0 tx=0x7f8288009240 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8277fff640 1 -- 172.21.15.5:0/2246949965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82840bf110 con 0x562da0125d60 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.747+0000 7f8277fff640 1 -- 172.21.15.5:0/2246949965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840054d0 con 0x562da0125d60 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 -- 172.21.15.5:0/2246949965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 msgr2=0x562da012e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da012e890 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f8284006010 tx=0x7f82840be910 comp rx=0 tx=0).stop 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 -- 172.21.15.5:0/2246949965 shutdown_connections 2024-04-22T13:32:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da012e890 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da012ee60 0x562da012f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 --2- 172.21.15.5:0/2246949965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da01253b0 0x562da0125790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 -- 172.21.15.5:0/2246949965 >> 172.21.15.5:0/2246949965 conn(0x562da011fba0 msgr2=0x562da011ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 -- 172.21.15.5:0/2246949965 shutdown_connections 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 -- 172.21.15.5:0/2246949965 wait complete. 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.748+0000 7f8296ddc740 1 -- start start 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 Processor -- start 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da01253b0 0x562da01352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da0132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0132bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562da0135b60 con 0x562da0125d60 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562da0043500 con 0x562da01253b0 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562da0135ce0 con 0x562da012ee60 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f8296ddc740 10 client.0 _ll_register_callbacks cb 0x562da0011b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da0132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da0132650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58778/0 (socket says 172.21.15.5:58778) 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 -- 172.21.15.5:0/3708810907 learned_addr learned my addr 172.21.15.5:0/3708810907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82926ab640 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0132bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 -- 172.21.15.5:0/3708810907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 msgr2=0x562da0132bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0132bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 -- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da01253b0 msgr2=0x562da01352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da01253b0 0x562da01352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 -- 172.21.15.5:0/3708810907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9fd9c3f0 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82916a9640 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da0132650 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f827c00bad0 tx=0x7f827c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94349079bd71d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.749+0000 7f82757fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f82757fa640 1 -- 172.21.15.5:0/3708810907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c011440 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f82757fa640 1 -- 172.21.15.5:0/3708810907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f827c0115e0 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82840bf110 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f82757fa640 1 -- 172.21.15.5:0/3708810907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f827c0117e0 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 10 client.17682 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d9fe667d0 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f82757fa640 1 -- 172.21.15.5:0/3708810907 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82840bf110 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f82757fa640 1 -- 172.21.15.5:0/3708810907 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f827c01b970 con 0x562da0125d60 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 10 client.17682 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 20 client.17682 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da0142e10 0x562da0145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 4 client.17682 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.750+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da0029930 con 0x562da0142e10 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.751+0000 7f8291eaa640 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da0142e10 0x562da0145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.751+0000 7f8291eaa640 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da0142e10 0x562da0145270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e7bdc64919ca14 server_cookie=1f35bd8d8588d6b7 in_seq=0 out_seq=0 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.751+0000 7f82757fa640 10 client.17682 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f82757fa640 1 -- 172.21.15.5:0/3708810907 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562da0029930 con 0x562da0142e10 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f82757fa640 10 client.17682 handle_command_reply: tid=0 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 client.17682 shutdown 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 msgr2=0x562da0132650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da0132650 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f827c00bad0 tx=0x7f827c00bf90 comp rx=0 tx=0).stop 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 shutdown_connections 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da0142e10 0x562da0145270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da0125d60 0x562da0132650 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f82757fa640 0 client.17682 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 --2- 172.21.15.5:0/3708810907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da01253b0 0x562da01352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 >> 172.21.15.5:0/3708810907 conn(0x562da011fba0 msgr2=0x562da01218f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 shutdown_connections 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 1 -- 172.21.15.5:0/3708810907 wait complete. 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.752+0000 7f8296ddc740 20 client.17682 trim_cache size 0 max 16384 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.765+0000 7f8296ddc740 1 Processor -- start 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.765+0000 7f8296ddc740 1 -- start start 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da013e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da012e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0130250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8296ddc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1aba10 con 0x562da01253b0 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8296ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1ad550 con 0x562da0125d60 2024-04-22T13:32:11.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8296ddc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1ac0e0 con 0x562da012ee60 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da013e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da013e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58784/0 (socket says 172.21.15.5:58784) 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 -- 172.21.15.5:0/118006717 learned_addr learned my addr 172.21.15.5:0/118006717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f82916a9640 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da012e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f82926ab640 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0130250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 -- 172.21.15.5:0/118006717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 msgr2=0x562da0130250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0130250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 -- 172.21.15.5:0/118006717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 msgr2=0x562da012e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da012e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 -- 172.21.15.5:0/118006717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da0124fe0 con 0x562da01253b0 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f82926ab640 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0130250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f8291eaa640 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da013e900 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f82840be710 tx=0x7f82840a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5c069ba9fe5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f82777fe640 1 -- 172.21.15.5:0/118006717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840bf110 con 0x562da01253b0 2024-04-22T13:32:11.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f82777fe640 1 -- 172.21.15.5:0/118006717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82840c2dc0 con 0x562da01253b0 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.766+0000 7f82777fe640 1 -- 172.21.15.5:0/118006717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840bb080 con 0x562da01253b0 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 -- 172.21.15.5:0/118006717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 msgr2=0x562da013e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da013e900 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f82840be710 tx=0x7f82840a9040 comp rx=0 tx=0).stop 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 -- 172.21.15.5:0/118006717 shutdown_connections 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0130250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da012e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 --2- 172.21.15.5:0/118006717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da013e900 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 -- 172.21.15.5:0/118006717 >> 172.21.15.5:0/118006717 conn(0x562da011fba0 msgr2=0x562da011e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 -- 172.21.15.5:0/118006717 shutdown_connections 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 -- 172.21.15.5:0/118006717 wait complete. 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 -- start start 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.767+0000 7f8296ddc740 1 Processor -- start 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da0134340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da0134880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1ace80 con 0x562da01253b0 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1ac7b0 con 0x562da0125d60 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1a99f0 con 0x562da012ee60 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 10 client.0 _ll_register_callbacks cb 0x7f827c004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da0134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da0134340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 -- 172.21.15.5:0/1951531062 learned_addr learned my addr 172.21.15.5:0/1951531062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 -- 172.21.15.5:0/1951531062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 msgr2=0x562da0134dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82926ab640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82916a9640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da0134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0134dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 -- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 msgr2=0x562da0134880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da0134880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 -- 172.21.15.5:0/1951531062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da01a8480 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82926ab640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82916a9640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da0134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8291eaa640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da0134340 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f82840be9a0 tx=0x7f828402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc05127b920534d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82757fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82757fa640 1 -- 172.21.15.5:0/1951531062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840bf110 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f82757fa640 1 -- 172.21.15.5:0/1951531062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f82840054d0 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.768+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82840c2dc0 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f82757fa640 1 -- 172.21.15.5:0/1951531062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82840bea10 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 10 client.17688 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562da01a96f0 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f82757fa640 1 -- 172.21.15.5:0/1951531062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82840c2dc0 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f82757fa640 1 -- 172.21.15.5:0/1951531062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f82840f0300 con 0x562da01253b0 2024-04-22T13:32:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 10 client.17688 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 20 client.17688 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da01a9a30 0x562da01abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 4 client.17688 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562da0097c10 con 0x562da01a9a30 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.769+0000 7f82916a9640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da01a9a30 0x562da01abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f82916a9640 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da01a9a30 0x562da01abe20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b3a3337474006a server_cookie=d85c02e8a01980c6 in_seq=0 out_seq=0 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f82757fa640 10 client.17688 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f82757fa640 1 -- 172.21.15.5:0/1951531062 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562da0097c10 con 0x562da01a9a30 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f82757fa640 10 client.17688 handle_command_reply: tid=0 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 client.17688 shutdown 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 msgr2=0x562da0134340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da0134340 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f82840be9a0 tx=0x7f828402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 shutdown_connections 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x562da01a9a30 0x562da01abe20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562da012ee60 0x562da0134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f82757fa640 0 client.17688 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562da0125d60 0x562da0134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 --2- 172.21.15.5:0/1951531062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da01253b0 0x562da0134340 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 >> 172.21.15.5:0/1951531062 conn(0x562da011fba0 msgr2=0x562da012e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 shutdown_connections 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 1 -- 172.21.15.5:0/1951531062 wait complete. 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.770+0000 7f8296ddc740 20 client.17688 trim_cache size 0 max 16384 2024-04-22T13:32:11.845 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82780a3d20 msgr2=0x7f82780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82780a3d20 0x7f82780a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8288005fd0 tx=0x7f8288009240 comp rx=0 tx=0).stop 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 msgr2=0x7f828c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c199580 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8284005940 tx=0x7f828402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 shutdown_connections 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f82780a3d20 0x7f82780a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f828c19a050 0x7f828c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f828c106160 0x7f828c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 --2- 172.21.15.5:0/2237407827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c101090 0x7f828c199580 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 >> 172.21.15.5:0/2237407827 conn(0x7f828c078000 msgr2=0x7f828c0ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 shutdown_connections 2024-04-22T13:32:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:11.771+0000 7f829410e640 1 -- 172.21.15.5:0/2237407827 wait complete. 2024-04-22T13:32:12.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:11 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/722194093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:12.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:11 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/722194093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:12.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:11 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/722194093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:12.849 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:32:12.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:32:12.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 -- 172.21.15.5:0/3823059276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b41044f0 msgr2=0x7f21b410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3823059276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b41044f0 0x7f21b410eef0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009cd0 tx=0x7f21ac02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 -- 172.21.15.5:0/3823059276 shutdown_connections 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3823059276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b41044f0 0x7f21b410eef0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3823059276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b4103b50 0x7f21b4103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3823059276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21b4102270 0x7f21b4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.974+0000 7f21bc03f640 1 -- 172.21.15.5:0/3823059276 >> 172.21.15.5:0/3823059276 conn(0x7f21b40fda10 msgr2=0x7f21b40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 -- 172.21.15.5:0/3823059276 shutdown_connections 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 -- 172.21.15.5:0/3823059276 wait complete. 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 Processor -- start 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 -- start start 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21b4102270 0x7f21b419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 0x7f21b419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b41044f0 0x7f21b41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4112290 con 0x7f21b4103b50 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4112110 con 0x7f21b4102270 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21bc03f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4112410 con 0x7f21b41044f0 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.975+0000 7f21b95da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 0x7f21b419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 0x7f21b419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58800/0 (socket says 172.21.15.5:58800) 2024-04-22T13:32:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 -- 172.21.15.5:0/3879254901 learned_addr learned my addr 172.21.15.5:0/3879254901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21b4102270 0x7f21b419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b41044f0 0x7f21b41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 -- 172.21.15.5:0/3879254901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b41044f0 msgr2=0x7f21b41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b41044f0 0x7f21b41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 -- 172.21.15.5:0/3879254901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21b4102270 msgr2=0x7f21b419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21b4102270 0x7f21b419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 -- 172.21.15.5:0/3879254901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b4198d70 con 0x7f21b4103b50 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21b95da640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 0x7f21b419e980 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f21a400d770 tx=0x7f21a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21aaffd640 1 -- 172.21.15.5:0/3879254901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4017070 con 0x7f21b4103b50 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b4199060 con 0x7f21b4103b50 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21aaffd640 1 -- 172.21.15.5:0/3879254901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a40054d0 con 0x7f21b4103b50 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21aaffd640 1 -- 172.21.15.5:0/3879254901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4004c30 con 0x7f21b4103b50 2024-04-22T13:32:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.976+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b41a5300 con 0x7f21b4103b50 2024-04-22T13:32:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21aaffd640 1 -- 172.21.15.5:0/3879254901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f21a4004df0 con 0x7f21b4103b50 2024-04-22T13:32:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 Processor -- start 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 -- start start 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33c7c0 0x55784a33cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33d170 0x55784a345ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a3466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55784a2a0f00 con 0x55784a346270 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55784a23e590 con 0x55784a33c7c0 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21bec2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55784a29cec0 con 0x55784a33d170 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21b95da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a3466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21aaffd640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f21a00a3d20 0x7f21a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21b95da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a3466d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58812/0 (socket says 172.21.15.5:58812) 2024-04-22T13:32:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.979+0000 7f21b95da640 1 -- 172.21.15.5:0/807622032 learned_addr learned my addr 172.21.15.5:0/807622032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33c7c0 0x55784a33cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f21a00a3d20 0x7f21a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33d170 0x55784a345ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21aaffd640 1 -- 172.21.15.5:0/3879254901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f21a40d0210 con 0x7f21b4103b50 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b95da640 1 -- 172.21.15.5:0/807622032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33d170 msgr2=0x55784a345ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b95da640 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33d170 0x55784a345ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b95da640 1 -- 172.21.15.5:0/807622032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33c7c0 msgr2=0x55784a33cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b95da640 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33c7c0 0x55784a33cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b95da640 1 -- 172.21.15.5:0/807622032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784a07b7e0 con 0x55784a346270 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33c7c0 0x55784a33cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21b95da640 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a3466d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f21a40cda40 tx=0x7f21a409c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bd9d27063bbfcb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f219ffff640 1 -- 172.21.15.5:0/807622032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4020030 con 0x55784a346270 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f21a00a3d20 0x7f21a00a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f21ac006010 tx=0x7f21ac0095c0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:12.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f219ffff640 1 -- 172.21.15.5:0/807622032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a40a0590 con 0x55784a346270 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.980+0000 7f219ffff640 1 -- 172.21.15.5:0/807622032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40cdda0 con 0x55784a346270 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 -- 172.21.15.5:0/807622032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 msgr2=0x55784a3466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a3466d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f21a40cda40 tx=0x7f21a409c0e0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 -- 172.21.15.5:0/807622032 shutdown_connections 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a3466d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33d170 0x55784a345ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 --2- 172.21.15.5:0/807622032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33c7c0 0x55784a33cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 -- 172.21.15.5:0/807622032 >> 172.21.15.5:0/807622032 conn(0x55784a336fb0 msgr2=0x55784a337390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 -- 172.21.15.5:0/807622032 shutdown_connections 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 -- 172.21.15.5:0/807622032 wait complete. 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.981+0000 7f21bec2b740 1 -- start start 2024-04-22T13:32:12.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 Processor -- start 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a349d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a34a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55784a34d150 con 0x55784a346270 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55784a25a580 con 0x55784a33d170 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55784a34d2d0 con 0x55784a33c7c0 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21bec2b740 10 client.0 _ll_register_callbacks cb 0x55784a260650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21ba5dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a34a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21ba5dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a34a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58814/0 (socket says 172.21.15.5:58814) 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21ba5dc640 1 -- 172.21.15.5:0/1886131136 learned_addr learned my addr 172.21.15.5:0/1886131136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a349d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.982+0000 7f21b95da640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21ba5dc640 1 -- 172.21.15.5:0/1886131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 msgr2=0x55784a349d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a349d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21ba5dc640 1 -- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 msgr2=0x55784a34a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21ba5dc640 1 -- 172.21.15.5:0/1886131136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784a0dbf70 con 0x55784a346270 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a34a7b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0329e0 tx=0x7f21ac02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c640a5a3e375e79b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f219d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f219d7fa640 1 -- 172.21.15.5:0/1886131136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac052060 con 0x55784a346270 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f219d7fa640 1 -- 172.21.15.5:0/1886131136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f21ac004040 con 0x55784a346270 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a40a0590 con 0x55784a346270 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f219d7fa640 1 -- 172.21.15.5:0/1886131136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21ac039070 con 0x55784a346270 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21bec2b740 10 client.17700 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557849f73170 con 0x55784a346270 2024-04-22T13:32:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f219d7fa640 1 -- 172.21.15.5:0/1886131136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a40a0590 con 0x55784a346270 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.983+0000 7f219d7fa640 1 -- 172.21.15.5:0/1886131136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f21ac034890 con 0x55784a346270 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21bec2b740 10 client.17700 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21bec2b740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21bec2b740 20 client.17700 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21bec2b740 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a35a720 0x55784a35cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21bec2b740 4 client.17700 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55784a2ae920 con 0x55784a35a720 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a35a720 0x55784a35cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f219d7fa640 10 client.17700 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:12.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.984+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a35a720 0x55784a35cb30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db4adbacb22b23e server_cookie=dfa489ccd8173226 in_seq=0 out_seq=0 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f219d7fa640 1 -- 172.21.15.5:0/1886131136 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55784a2ae920 con 0x55784a35a720 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f219d7fa640 10 client.17700 handle_command_reply: tid=0 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 client.17700 shutdown 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 msgr2=0x55784a34a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a34a7b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0329e0 tx=0x7f21ac02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 shutdown_connections 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a35a720 0x55784a35cb30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a346270 0x55784a34a7b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f219d7fa640 0 client.17700 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 --2- 172.21.15.5:0/1886131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a349d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.985+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 >> 172.21.15.5:0/1886131136 conn(0x55784a336fb0 msgr2=0x55784a338d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.986+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 shutdown_connections 2024-04-22T13:32:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.986+0000 7f21bec2b740 1 -- 172.21.15.5:0/1886131136 wait complete. 2024-04-22T13:32:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:12.986+0000 7f21bec2b740 20 client.17700 trim_cache size 0 max 16384 2024-04-22T13:32:13.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.003+0000 7f21bec2b740 1 Processor -- start 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.003+0000 7f21bec2b740 1 -- start start 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a356050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a347680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a338ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21bec2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41ab8a0 con 0x55784a336fb0 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21bec2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41ad3e0 con 0x55784a33d170 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21bec2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41abf70 con 0x55784a33c7c0 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a356050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a356050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58826/0 (socket says 172.21.15.5:58826) 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 -- 172.21.15.5:0/2450041601 learned_addr learned my addr 172.21.15.5:0/2450041601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b95da640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a347680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 -- 172.21.15.5:0/2450041601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 msgr2=0x55784a347680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a347680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a338ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 -- 172.21.15.5:0/2450041601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 msgr2=0x55784a338ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a338ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 -- 172.21.15.5:0/2450041601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784a348280 con 0x55784a336fb0 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b95da640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a347680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a338ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a356050 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x55784a35d3a0 tx=0x7f21b000afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79125c5e6494d340 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f219f7fe640 1 -- 172.21.15.5:0/2450041601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0005220 con 0x55784a336fb0 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f219f7fe640 1 -- 172.21.15.5:0/2450041601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b0023070 con 0x55784a336fb0 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.004+0000 7f219f7fe640 1 -- 172.21.15.5:0/2450041601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b001f440 con 0x55784a336fb0 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 -- 172.21.15.5:0/2450041601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 msgr2=0x55784a356050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a356050 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x55784a35d3a0 tx=0x7f21b000afd0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 -- 172.21.15.5:0/2450041601 shutdown_connections 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a338ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a347680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 --2- 172.21.15.5:0/2450041601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a356050 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 -- 172.21.15.5:0/2450041601 >> 172.21.15.5:0/2450041601 conn(0x55784a348970 msgr2=0x55784a3383b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 -- 172.21.15.5:0/2450041601 shutdown_connections 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 -- 172.21.15.5:0/2450041601 wait complete. 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 -- start start 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.005+0000 7f21bec2b740 1 Processor -- start 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a3524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a352a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41acd10 con 0x55784a336fb0 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41ac640 con 0x55784a33d170 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41a98a0 con 0x55784a33c7c0 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21bec2b740 10 client.0 _ll_register_callbacks cb 0x7f21ac009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a3524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a3524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58842/0 (socket says 172.21.15.5:58842) 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 -- 172.21.15.5:0/3825115302 learned_addr learned my addr 172.21.15.5:0/3825115302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b95da640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a352a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21ba5dc640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 -- 172.21.15.5:0/3825115302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 msgr2=0x55784a352a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a352a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 -- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 msgr2=0x55784a34fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.006+0000 7f21b9ddb640 1 -- 172.21.15.5:0/3825115302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784a350400 con 0x55784a336fb0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21b9ddb640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a3524d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f21b0005e90 tx=0x7f21b0005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4582a0c9f3a655 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f219d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f219d7fa640 1 -- 172.21.15.5:0/3825115302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0004220 con 0x55784a336fb0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f219d7fa640 1 -- 172.21.15.5:0/3825115302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f21b00043c0 con 0x55784a336fb0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b0023070 con 0x55784a336fb0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f219d7fa640 1 -- 172.21.15.5:0/3825115302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21b001f3d0 con 0x55784a336fb0 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 10 client.17706 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:13.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55784a3c2860 con 0x55784a336fb0 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f219d7fa640 1 -- 172.21.15.5:0/3825115302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b0023070 con 0x55784a336fb0 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f219d7fa640 1 -- 172.21.15.5:0/3825115302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f21b00035f0 con 0x55784a336fb0 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 10 client.17706 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 20 client.17706 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a346270 0x55784a3c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 4 client.17706 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.007+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55784a2a8790 con 0x55784a346270 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.008+0000 7f21b95da640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a346270 0x55784a3c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.008+0000 7f21b95da640 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a346270 0x55784a3c4fb0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff5c73441ff4b5d server_cookie=d2c3d9d7251b731 in_seq=0 out_seq=0 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.008+0000 7f219d7fa640 10 client.17706 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f219d7fa640 1 -- 172.21.15.5:0/3825115302 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55784a2a8790 con 0x55784a346270 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f219d7fa640 10 client.17706 handle_command_reply: tid=0 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 client.17706 shutdown 2024-04-22T13:32:13.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 msgr2=0x55784a3524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a3524d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f21b0005e90 tx=0x7f21b0005480 comp rx=0 tx=0).stop 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 shutdown_connections 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55784a346270 0x55784a3c4fb0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55784a33d170 0x55784a34fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55784a33c7c0 0x55784a352a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f219d7fa640 0 client.17706 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 --2- 172.21.15.5:0/3825115302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55784a336fb0 0x55784a3524d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 >> 172.21.15.5:0/3825115302 conn(0x55784a348970 msgr2=0x55784a34a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 shutdown_connections 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.009+0000 7f21bec2b740 1 -- 172.21.15.5:0/3825115302 wait complete. 2024-04-22T13:32:13.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bec2b740 20 client.17706 trim_cache size 0 max 16384 2024-04-22T13:32:13.013 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:32:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d1604b23-6b09-46b5-a18c-695e21b1e9a1", 2024-04-22T13:32:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:32:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:13.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f21a00a3d20 msgr2=0x7f21a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f21a00a3d20 0x7f21a00a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f21ac006010 tx=0x7f21ac0095c0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 msgr2=0x7f21b419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 0x7f21b419e980 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f21a400d770 tx=0x7f21a400dc30 comp rx=0 tx=0).stop 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 shutdown_connections 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21b4103b50 0x7f21b419e980 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f21a00a3d20 0x7f21a00a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b41044f0 0x7f21b41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 --2- 172.21.15.5:0/3879254901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21b4102270 0x7f21b419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 >> 172.21.15.5:0/3879254901 conn(0x7f21b40fda10 msgr2=0x7f21b40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 shutdown_connections 2024-04-22T13:32:13.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.010+0000 7f21bc03f640 1 -- 172.21.15.5:0/3879254901 wait complete. 2024-04-22T13:32:13.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:13.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 -- 172.21.15.5:0/3085037969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 msgr2=0x7f754810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 --2- 172.21.15.5:0/3085037969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f754810bfc0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7530009900 tx=0x7f753002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 -- 172.21.15.5:0/3085037969 shutdown_connections 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 --2- 172.21.15.5:0/3085037969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f754810bfc0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 --2- 172.21.15.5:0/3085037969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f754810c590 0x7f754810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 --2- 172.21.15.5:0/3085037969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75481031c0 0x7f75481035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 -- 172.21.15.5:0/3085037969 >> 172.21.15.5:0/3085037969 conn(0x7f7548077f40 msgr2=0x7f7548078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 -- 172.21.15.5:0/3085037969 shutdown_connections 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.128+0000 7f754d58f640 1 -- 172.21.15.5:0/3085037969 wait complete. 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 Processor -- start 2024-04-22T13:32:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 -- start start 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75481031c0 0x7f75481a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f75481a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f754810c590 0x7f754819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75481121b0 con 0x7f7548103ae0 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7548112030 con 0x7f754810c590 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f754d58f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7548112330 con 0x7f75481031c0 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f753ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f75481a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f753ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f75481a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58856/0 (socket says 172.21.15.5:58856) 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.129+0000 7f753ffff640 1 -- 172.21.15.5:0/2182968665 learned_addr learned my addr 172.21.15.5:0/2182968665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f7546ffd640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75481031c0 0x7f75481a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f75477fe640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f754810c590 0x7f754819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f753ffff640 1 -- 172.21.15.5:0/2182968665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75481031c0 msgr2=0x7f75481a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f753ffff640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75481031c0 0x7f75481a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f753ffff640 1 -- 172.21.15.5:0/2182968665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f754810c590 msgr2=0x7f754819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f753ffff640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f754810c590 0x7f754819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f753ffff640 1 -- 172.21.15.5:0/2182968665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754819b040 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f753ffff640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f75481a0bb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7530005940 tx=0x7f75300045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f7544ff9640 1 -- 172.21.15.5:0/2182968665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530043020 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f7544ff9640 1 -- 172.21.15.5:0/2182968665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753002dba0 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f7544ff9640 1 -- 172.21.15.5:0/2182968665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530003930 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754819b2d0 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.130+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75481a75f0 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.132+0000 7f7544ff9640 1 -- 172.21.15.5:0/2182968665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7530046020 con 0x7f7548103ae0 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 Processor -- start 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 -- start start 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b20960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b29540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b29a80 0x558446b2be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558446a851e0 con 0x558446b20580 2024-04-22T13:32:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558446a22390 con 0x558446b29a80 2024-04-22T13:32:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7550180740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558446a811a0 con 0x558446b20ea0 2024-04-22T13:32:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7544ff9640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f75340a3da0 0x7f75340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f75477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b20960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f75477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b20960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58868/0 (socket says 172.21.15.5:58868) 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f75477fe640 1 -- 172.21.15.5:0/3079230594 learned_addr learned my addr 172.21.15.5:0/3079230594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7544ff9640 1 -- 172.21.15.5:0/2182968665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f75300f2730 con 0x7f7548103ae0 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f7546ffd640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b29540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f75477fe640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f75340a3da0 0x7f75340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.133+0000 7f753ffff640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b29a80 0x558446b2be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 -- 172.21.15.5:0/3079230594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 msgr2=0x558446b29540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b29540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 -- 172.21.15.5:0/3079230594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b29a80 msgr2=0x558446b2be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b29a80 0x558446b2be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 -- 172.21.15.5:0/3079230594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584468617d0 con 0x558446b20580 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f753ffff640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b29a80 0x558446b2be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f7546ffd640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b29540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b20960 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7540012cb0 tx=0x7f754000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a38f27df8c4324 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f75477fe640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f75340a3da0 0x7f75340a6260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f754000afd0 tx=0x7f754001a000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f753d7fa640 1 -- 172.21.15.5:0/3079230594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540020070 con 0x558446b20580 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f753d7fa640 1 -- 172.21.15.5:0/3079230594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754000aa70 con 0x558446b20580 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.134+0000 7f753d7fa640 1 -- 172.21.15.5:0/3079230594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754001cb80 con 0x558446b20580 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 -- 172.21.15.5:0/3079230594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 msgr2=0x558446b20960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b20960 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7540012cb0 tx=0x7f754000f730 comp rx=0 tx=0).stop 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 -- 172.21.15.5:0/3079230594 shutdown_connections 2024-04-22T13:32:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b29a80 0x558446b2be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b29540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 --2- 172.21.15.5:0/3079230594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b20960 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 -- 172.21.15.5:0/3079230594 >> 172.21.15.5:0/3079230594 conn(0x558446b1acb0 msgr2=0x558446b1b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 -- 172.21.15.5:0/3079230594 shutdown_connections 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 -- 172.21.15.5:0/3079230594 wait complete. 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 -- start start 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.135+0000 7f7550180740 1 Processor -- start 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b26b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b29a80 0x558446b275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558446b2c810 con 0x558446b20580 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558446a3e2f0 con 0x558446b20ea0 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558446b2c990 con 0x558446b29a80 2024-04-22T13:32:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b26b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 10 client.0 _ll_register_callbacks cb 0x558446a4b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b26b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58874/0 (socket says 172.21.15.5:58874) 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 -- 172.21.15.5:0/1724292568 learned_addr learned my addr 172.21.15.5:0/1724292568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f75477fe640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b29a80 0x558446b275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 -- 172.21.15.5:0/1724292568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b29a80 msgr2=0x558446b275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b29a80 0x558446b275e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 -- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 msgr2=0x558446b270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f753ffff640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 -- 172.21.15.5:0/1724292568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584467973f0 con 0x558446b20580 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b26b60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7538012740 tx=0x7f7538012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e6971a1ffd54f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f752e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f752e7fc640 1 -- 172.21.15.5:0/1724292568 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538004030 con 0x558446b20580 2024-04-22T13:32:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f752e7fc640 1 -- 172.21.15.5:0/1724292568 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f75380041d0 con 0x558446b20580 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.136+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f754000aa70 con 0x558446b20580 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f752e7fc640 1 -- 172.21.15.5:0/1724292568 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75380043d0 con 0x558446b20580 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 10 client.17718 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584467570f0 con 0x558446b20580 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f752e7fc640 1 -- 172.21.15.5:0/1724292568 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754000aa70 con 0x558446b20580 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f752e7fc640 1 -- 172.21.15.5:0/1724292568 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7538003b40 con 0x558446b20580 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 10 client.17718 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 20 client.17718 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446b39b30 0x558446b3bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 4 client.17718 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558446a92c00 con 0x558446b39b30 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.137+0000 7f753ffff640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446b39b30 0x558446b3bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.138+0000 7f752e7fc640 10 client.17718 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.138+0000 7f753ffff640 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446b39b30 0x558446b3bfc0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0dbc02af2df0491 server_cookie=79cc782ad69a829c in_seq=0 out_seq=0 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.138+0000 7f752e7fc640 1 -- 172.21.15.5:0/1724292568 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558446a92c00 con 0x558446b39b30 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.138+0000 7f752e7fc640 10 client.17718 handle_command_reply: tid=0 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.138+0000 7f7550180740 1 client.17718 shutdown 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 msgr2=0x558446b26b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b26b60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7538012740 tx=0x7f7538012c00 comp rx=0 tx=0).stop 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 shutdown_connections 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446b39b30 0x558446b3bfc0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b29a80 0x558446b275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f752e7fc640 0 client.17718 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 --2- 172.21.15.5:0/1724292568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b20580 0x558446b26b60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 >> 172.21.15.5:0/1724292568 conn(0x558446b1acb0 msgr2=0x558446b1ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 shutdown_connections 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 1 -- 172.21.15.5:0/1724292568 wait complete. 2024-04-22T13:32:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.139+0000 7f7550180740 20 client.17718 trim_cache size 0 max 16384 2024-04-22T13:32:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 Processor -- start 2024-04-22T13:32:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 -- start start 2024-04-22T13:32:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20580 0x558446b29290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b25aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75481afbd0 con 0x558446b29a80 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75481b1710 con 0x558446b20ea0 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7550180740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75481b02a0 con 0x558446b20580 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f75477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f75477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58884/0 (socket says 172.21.15.5:58884) 2024-04-22T13:32:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f75477fe640 1 -- 172.21.15.5:0/1816960190 learned_addr learned my addr 172.21.15.5:0/1816960190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20580 0x558446b29290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f753ffff640 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b25aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f75477fe640 1 -- 172.21.15.5:0/1816960190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20580 msgr2=0x558446b29290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f75477fe640 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20580 0x558446b29290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.157+0000 7f75477fe640 1 -- 172.21.15.5:0/1816960190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 msgr2=0x558446b25aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f75477fe640 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b25aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f75477fe640 1 -- 172.21.15.5:0/1816960190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558446b1bcb0 con 0x558446b29a80 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f75477fe640 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b356d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f754000ae70 tx=0x7f754000ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51949db8a1e1706 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f753cff9640 1 -- 172.21.15.5:0/1816960190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540020050 con 0x558446b29a80 2024-04-22T13:32:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f753cff9640 1 -- 172.21.15.5:0/1816960190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7540029070 con 0x558446b29a80 2024-04-22T13:32:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f753cff9640 1 -- 172.21.15.5:0/1816960190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540005330 con 0x558446b29a80 2024-04-22T13:32:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f7550180740 1 -- 172.21.15.5:0/1816960190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 msgr2=0x558446b356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.158+0000 7f7550180740 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b356d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f754000ae70 tx=0x7f754000ae00 comp rx=0 tx=0).stop 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- 172.21.15.5:0/1816960190 shutdown_connections 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b356d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20ea0 0x558446b25aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 --2- 172.21.15.5:0/1816960190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20580 0x558446b29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- 172.21.15.5:0/1816960190 >> 172.21.15.5:0/1816960190 conn(0x558446b1acb0 msgr2=0x558446b19d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- 172.21.15.5:0/1816960190 shutdown_connections 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- 172.21.15.5:0/1816960190 wait complete. 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- start start 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 Processor -- start 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20580 0x558446b31dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b2e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75481b1040 con 0x558446b29a80 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75481b0970 con 0x558446b20580 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75481adbb0 con 0x558446b20ea0 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7550180740 10 client.0 _ll_register_callbacks cb 0x7f7538009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f75477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f75477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58894/0 (socket says 172.21.15.5:58894) 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f75477fe640 1 -- 172.21.15.5:0/1054976126 learned_addr learned my addr 172.21.15.5:0/1054976126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20580 0x558446b31dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.159+0000 7f753ffff640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b2e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f75477fe640 1 -- 172.21.15.5:0/1054976126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 msgr2=0x558446b2e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f75477fe640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b2e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f75477fe640 1 -- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20580 msgr2=0x558446b31dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f75477fe640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20580 0x558446b31dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f75477fe640 1 -- 172.21.15.5:0/1054976126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558446b2ecc0 con 0x558446b29a80 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f753ffff640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f75477fe640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b2e680 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7540019e10 tx=0x7f7540019e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642cfec07b5fd425 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f752e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f752e7fc640 1 -- 172.21.15.5:0/1054976126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754003a060 con 0x558446b29a80 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f752e7fc640 1 -- 172.21.15.5:0/1054976126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f7540004dd0 con 0x558446b29a80 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7540029070 con 0x558446b29a80 2024-04-22T13:32:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f752e7fc640 1 -- 172.21.15.5:0/1054976126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7540020030 con 0x558446b29a80 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f7550180740 10 client.17724 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558446ba0f30 con 0x558446b29a80 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.160+0000 7f752e7fc640 1 -- 172.21.15.5:0/1054976126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7540029070 con 0x558446b29a80 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f752e7fc640 1 -- 172.21.15.5:0/1054976126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f754001b070 con 0x558446b29a80 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 10 client.17724 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 20 client.17724 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446ba1450 0x558446ba3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 4 client.17724 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558446a8ca70 con 0x558446ba1450 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446ba1450 0x558446ba3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f752e7fc640 10 client.17724 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7546ffd640 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446ba1450 0x558446ba3840 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c985bbb7afd3ebf server_cookie=b39f117bd1f04748 in_seq=0 out_seq=0 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f752e7fc640 1 -- 172.21.15.5:0/1054976126 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558446a8ca70 con 0x558446ba1450 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f752e7fc640 10 client.17724 handle_command_reply: tid=0 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.161+0000 7f7550180740 1 client.17724 shutdown 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 msgr2=0x558446b2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b2e680 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7540019e10 tx=0x7f7540019e40 comp rx=0 tx=0).stop 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 shutdown_connections 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558446ba1450 0x558446ba3840 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558446b29a80 0x558446b2e680 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558446b20ea0 0x558446b2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f752e7fc640 0 client.17724 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 --2- 172.21.15.5:0/1054976126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558446b20580 0x558446b31dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 >> 172.21.15.5:0/1054976126 conn(0x558446b1acb0 msgr2=0x558446b194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 shutdown_connections 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 1 -- 172.21.15.5:0/1054976126 wait complete. 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f7550180740 20 client.17724 trim_cache size 0 max 16384 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "d1604b23-6b09-46b5-a18c-695e21b1e9a1": { 2024-04-22T13:32:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:32:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1604b23-6b09-46b5-a18c-695e21b1e9a1", 2024-04-22T13:32:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:32:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:32:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:32:13.173 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f75340a3da0 msgr2=0x7f75340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f75340a3da0 0x7f75340a6260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f754000afd0 tx=0x7f754001a000 comp rx=0 tx=0).stop 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 msgr2=0x7f75481a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f75481a0bb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7530005940 tx=0x7f75300045e0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 shutdown_connections 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7548103ae0 0x7f75481a0bb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f75340a3da0 0x7f75340a6260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f754810c590 0x7f754819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 --2- 172.21.15.5:0/2182968665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75481031c0 0x7f75481a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 >> 172.21.15.5:0/2182968665 conn(0x7f7548077f40 msgr2=0x7f75481084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 shutdown_connections 2024-04-22T13:32:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:13.162+0000 7f754d58f640 1 -- 172.21.15.5:0/2182968665 wait complete. 2024-04-22T13:32:13.175 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:12 smithi005 ceph-mon[28006]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:32:13.176 INFO:tasks.cephfs.filesystem:scrub status for tag:d1604b23-6b09-46b5-a18c-695e21b1e9a1 - {'path': '/', 'tag': 'd1604b23-6b09-46b5-a18c-695e21b1e9a1', 'options': 'recursive,force'} 2024-04-22T13:32:13.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:12 smithi049 ceph-mon[31379]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:32:13.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:12 smithi107 ceph-mon[31859]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:32:14.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:13 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:32:14.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:13 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:14.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:13 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:32:14.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:13 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:14.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:13 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:32:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:13 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:13 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:32:14.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:13 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:13 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:32:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:13 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:13 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:32:14.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:13 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:15.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:14 smithi049 ceph-mon[31379]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:32:15.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:14 smithi107 ceph-mon[31859]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:32:15.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:14 smithi005 ceph-mon[28006]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:32:16.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:15 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:32:16.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:15 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:32:16.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:15 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:32:17.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:16 smithi049 ceph-mon[31379]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-04-22T13:32:17.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:16 smithi107 ceph-mon[31859]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-04-22T13:32:17.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:16 smithi005 ceph-mon[28006]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-04-22T13:32:19.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:18 smithi049 ceph-mon[31379]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:32:19.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:18 smithi107 ceph-mon[31859]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:32:19.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:18 smithi005 ceph-mon[28006]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:32:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:32:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:32:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:32:20.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:19 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:32:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:32:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:32:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:32:20.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:19 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:32:20.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:32:20.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:32:20.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:32:20.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:19 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:32:21.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:20 smithi049 ceph-mon[31379]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:32:21.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:20 smithi107 ceph-mon[31859]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:32:21.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:20 smithi005 ceph-mon[28006]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:32:23.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:22 smithi049 ceph-mon[31379]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-22T13:32:23.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:22 smithi107 ceph-mon[31859]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-22T13:32:23.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:22 smithi005 ceph-mon[28006]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-22T13:32:25.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:24 smithi049 ceph-mon[31379]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:32:25.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:24 smithi107 ceph-mon[31859]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:32:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:24 smithi005 ceph-mon[28006]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:32:27.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:26 smithi049 ceph-mon[31379]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:32:27.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:26 smithi107 ceph-mon[31859]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:32:27.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:26 smithi005 ceph-mon[28006]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:32:29.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:28 smithi049 ceph-mon[31379]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:29.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:28 smithi107 ceph-mon[31859]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:29.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:28 smithi005 ceph-mon[28006]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:31.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:30 smithi049 ceph-mon[31379]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:31.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:30 smithi107 ceph-mon[31859]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:30 smithi005 ceph-mon[28006]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:33.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:32 smithi049 ceph-mon[31379]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:33.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:32 smithi107 ceph-mon[31859]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:33.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:32 smithi005 ceph-mon[28006]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:35.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:34 smithi049 ceph-mon[31379]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:35.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:34 smithi107 ceph-mon[31859]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:35.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:34 smithi005 ceph-mon[28006]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:37.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:36 smithi049 ceph-mon[31379]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:37.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:36 smithi107 ceph-mon[31859]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:36 smithi005 ceph-mon[28006]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:39.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:38 smithi049 ceph-mon[31379]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:39.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:38 smithi107 ceph-mon[31859]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:39.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:38 smithi005 ceph-mon[28006]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:41.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:40 smithi049 ceph-mon[31379]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:41.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:40 smithi107 ceph-mon[31859]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:40 smithi005 ceph-mon[28006]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:43.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:43.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:43.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:43 smithi049 ceph-mon[31379]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:43.286 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:43 smithi107 ceph-mon[31859]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:43.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 -- 172.21.15.5:0/4003602731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9118100d40 msgr2=0x7f9118101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 --2- 172.21.15.5:0/4003602731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9118100d40 0x7f9118101120 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9114009900 tx=0x7f911402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 -- 172.21.15.5:0/4003602731 shutdown_connections 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 --2- 172.21.15.5:0/4003602731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f911810e8f0 0x7f9118110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 --2- 172.21.15.5:0/4003602731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118101660 0x7f911810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 --2- 172.21.15.5:0/4003602731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9118100d40 0x7f9118101120 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.299+0000 7f911f27b640 1 -- 172.21.15.5:0/4003602731 >> 172.21.15.5:0/4003602731 conn(0x7f9118077fa0 msgr2=0x7f91180783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 -- 172.21.15.5:0/4003602731 shutdown_connections 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 -- 172.21.15.5:0/4003602731 wait complete. 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 Processor -- start 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 -- start start 2024-04-22T13:32:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9118100d40 0x7f91181a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118101660 0x7f91181a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 0x7f911819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9118114410 con 0x7f911810e8f0 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9118114290 con 0x7f9118100d40 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.300+0000 7f911f27b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9118114590 con 0x7f9118101660 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 0x7f911819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 0x7f911819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35958/0 (socket says 172.21.15.5:35958) 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 -- 172.21.15.5:0/2499864481 learned_addr learned my addr 172.21.15.5:0/2499864481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d017640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9118100d40 0x7f91181a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911c816640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118101660 0x7f91181a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 -- 172.21.15.5:0/2499864481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118101660 msgr2=0x7f91181a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118101660 0x7f91181a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 -- 172.21.15.5:0/2499864481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9118100d40 msgr2=0x7f91181a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9118100d40 0x7f91181a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 -- 172.21.15.5:0/2499864481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911819af80 con 0x7f911810e8f0 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911d818640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 0x7f911819a7b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9110009fd0 tx=0x7f911000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f910e7fc640 1 -- 172.21.15.5:0/2499864481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110004d20 con 0x7f911810e8f0 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f910e7fc640 1 -- 172.21.15.5:0/2499864481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9110005020 con 0x7f911810e8f0 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911819b270 con 0x7f911810e8f0 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f910e7fc640 1 -- 172.21.15.5:0/2499864481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91100116b0 con 0x7f911810e8f0 2024-04-22T13:32:43.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.301+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91181a7490 con 0x7f911810e8f0 2024-04-22T13:32:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.304+0000 7f910e7fc640 1 -- 172.21.15.5:0/2499864481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f91100051d0 con 0x7f911810e8f0 2024-04-22T13:32:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.304+0000 7f9121e72740 1 Processor -- start 2024-04-22T13:32:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.304+0000 7f9121e72740 1 -- start start 2024-04-22T13:32:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f9121e72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb03580 0x55b5eeb03960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f9121e72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb0c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f9121e72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f9121e72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b5eea681e0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f9121e72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b5eea05390 con 0x55b5eeb03ea0 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f9121e72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b5eea641a0 con 0x55b5eeb03580 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f910e7fc640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f91040a3d20 0x7f91040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35970/0 (socket says 172.21.15.5:35970) 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 -- 172.21.15.5:0/214205437 learned_addr learned my addr 172.21.15.5:0/214205437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb03580 0x55b5eeb03960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911d017640 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb0c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911d818640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f91040a3d20 0x7f91040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 -- 172.21.15.5:0/214205437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb03580 msgr2=0x55b5eeb03960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb03580 0x55b5eeb03960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 -- 172.21.15.5:0/214205437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 msgr2=0x55b5eeb0c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb0c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f910e7fc640 1 -- 172.21.15.5:0/2499864481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f91100d0940 con 0x7f911810e8f0 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 -- 172.21.15.5:0/214205437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ee8447d0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.305+0000 7f911c816640 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0ee70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9108012770 tx=0x7f9108004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf476e99787a130 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f911d818640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f91040a3d20 0x7f91040a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f911009c8f0 tx=0x7f911006f450 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f91037fe640 1 -- 172.21.15.5:0/214205437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108012cf0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f91037fe640 1 -- 172.21.15.5:0/214205437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9108010770 con 0x55b5eeb0ca80 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f91037fe640 1 -- 172.21.15.5:0/214205437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91080044e0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 -- 172.21.15.5:0/214205437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 msgr2=0x55b5eeb0ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0ee70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9108012770 tx=0x7f9108004730 comp rx=0 tx=0).stop 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 -- 172.21.15.5:0/214205437 shutdown_connections 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0ee70 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb0c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 --2- 172.21.15.5:0/214205437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb03580 0x55b5eeb03960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 -- 172.21.15.5:0/214205437 >> 172.21.15.5:0/214205437 conn(0x55b5eeafdcb0 msgr2=0x55b5eeafe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 -- 172.21.15.5:0/214205437 shutdown_connections 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.306+0000 7f9121e72740 1 -- 172.21.15.5:0/214205437 wait complete. 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 -- start start 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 Processor -- start 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb09aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b5eeb0f870 con 0x55b5eeb03580 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b5eea212f0 con 0x55b5eeb03ea0 2024-04-22T13:32:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b5eeb0f9f0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f9121e72740 10 client.0 _ll_register_callbacks cb 0x55b5eea2e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.307+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb09aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb09aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35982/0 (socket says 172.21.15.5:35982) 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 -- 172.21.15.5:0/3616920105 learned_addr learned my addr 172.21.15.5:0/3616920105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911d017640 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 -- 172.21.15.5:0/3616920105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 msgr2=0x55b5eeb0a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 -- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 msgr2=0x55b5eeb09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 -- 172.21.15.5:0/3616920105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ee77a3f0 con 0x55b5eeb03580 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f911c816640 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb09aa0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f9108004700 tx=0x7f9108010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc09b314918e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9100ff9640 1 -- 172.21.15.5:0/3616920105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910802c060 con 0x55b5eeb03580 2024-04-22T13:32:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9100ff9640 1 -- 172.21.15.5:0/3616920105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f910801f660 con 0x55b5eeb03580 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9108010770 con 0x55b5eeb03580 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9100ff9640 1 -- 172.21.15.5:0/3616920105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f910801f860 con 0x55b5eeb03580 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9121e72740 10 client.17736 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.308+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5ee73a0f0 con 0x55b5eeb03580 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9100ff9640 1 -- 172.21.15.5:0/3616920105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9108010770 con 0x55b5eeb03580 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9100ff9640 1 -- 172.21.15.5:0/3616920105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f910802f8b0 con 0x55b5eeb03580 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9121e72740 10 client.17736 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9121e72740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9121e72740 20 client.17736 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9121e72740 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb1c910 0x55b5eeb1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9121e72740 4 client.17736 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5eea75c00 con 0x55b5eeb1c910 2024-04-22T13:32:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.309+0000 7f911d017640 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb1c910 0x55b5eeb1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f911d017640 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb1c910 0x55b5eeb1ed00 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e97f3045948504e server_cookie=abd0e2950e567b80 in_seq=0 out_seq=0 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9100ff9640 10 client.17736 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9100ff9640 1 -- 172.21.15.5:0/3616920105 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5eea75c00 con 0x55b5eeb1c910 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9100ff9640 10 client.17736 handle_command_reply: tid=0 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 client.17736 shutdown 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 msgr2=0x55b5eeb09aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb09aa0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f9108004700 tx=0x7f9108010f00 comp rx=0 tx=0).stop 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 shutdown_connections 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb1c910 0x55b5eeb1ed00 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb0a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9100ff9640 0 client.17736 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 --2- 172.21.15.5:0/3616920105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb09aa0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.310+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 >> 172.21.15.5:0/3616920105 conn(0x55b5eeafdcb0 msgr2=0x55b5eeaffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.311+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 shutdown_connections 2024-04-22T13:32:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.311+0000 7f9121e72740 1 -- 172.21.15.5:0/3616920105 wait complete. 2024-04-22T13:32:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.311+0000 7f9121e72740 20 client.17736 trim_cache size 0 max 16384 2024-04-22T13:32:43.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:42 smithi005 ceph-mon[28006]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:43.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.328+0000 7f9121e72740 1 Processor -- start 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.328+0000 7f9121e72740 1 -- start start 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.328+0000 7f9121e72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb18760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.328+0000 7f9121e72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9121e72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9121e72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91181afd00 con 0x55b5eeb03580 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9121e72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91181b1840 con 0x55b5eeb03ea0 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9121e72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91181b03d0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb18760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb18760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35992/0 (socket says 172.21.15.5:35992) 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 -- 172.21.15.5:0/1908837116 learned_addr learned my addr 172.21.15.5:0/1908837116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 -- 172.21.15.5:0/1908837116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 msgr2=0x55b5eeb24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 -- 172.21.15.5:0/1908837116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 msgr2=0x55b5eeb09380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911d017640 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 -- 172.21.15.5:0/1908837116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5eeb0ee70 con 0x55b5eeb03580 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f911c816640 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb18760 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9108010fb0 tx=0x7f9108005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede38b2b34010194 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9102ffd640 1 -- 172.21.15.5:0/1908837116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910802c070 con 0x55b5eeb03580 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9102ffd640 1 -- 172.21.15.5:0/1908837116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9108016070 con 0x55b5eeb03580 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.329+0000 7f9102ffd640 1 -- 172.21.15.5:0/1908837116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910801f460 con 0x55b5eeb03580 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 -- 172.21.15.5:0/1908837116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 msgr2=0x55b5eeb18760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb18760 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9108010fb0 tx=0x7f9108005c50 comp rx=0 tx=0).stop 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 -- 172.21.15.5:0/1908837116 shutdown_connections 2024-04-22T13:32:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb09380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 --2- 172.21.15.5:0/1908837116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb18760 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 -- 172.21.15.5:0/1908837116 >> 172.21.15.5:0/1908837116 conn(0x55b5eeafdcb0 msgr2=0x55b5eeaff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 -- 172.21.15.5:0/1908837116 shutdown_connections 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 -- 172.21.15.5:0/1908837116 wait complete. 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 -- start start 2024-04-22T13:32:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.330+0000 7f9121e72740 1 Processor -- start 2024-04-22T13:32:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb06b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb07060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91181b1170 con 0x55b5eeb03580 2024-04-22T13:32:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91181b0aa0 con 0x55b5eeb03ea0 2024-04-22T13:32:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91181adce0 con 0x55b5eeb0ca80 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9121e72740 10 client.0 _ll_register_callbacks cb 0x7f910800be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35996/0 (socket says 172.21.15.5:35996) 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 -- 172.21.15.5:0/2926656618 learned_addr learned my addr 172.21.15.5:0/2926656618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911d017640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb06b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb07060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 -- 172.21.15.5:0/2926656618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 msgr2=0x55b5eeb07060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb07060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 -- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 msgr2=0x55b5eeb06b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb06b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 -- 172.21.15.5:0/2926656618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5eeafd320 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f911c816640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb065e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9108015040 tx=0x7f910800b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7689dbca4f53e9df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.331+0000 7f9100ff9640 1 -- 172.21.15.5:0/2926656618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910802c040 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9108016070 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9100ff9640 1 -- 172.21.15.5:0/2926656618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f910801fc00 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9100ff9640 1 -- 172.21.15.5:0/2926656618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91080304a0 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 10 client.17742 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5eeb82da0 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9100ff9640 1 -- 172.21.15.5:0/2926656618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9108016070 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9100ff9640 1 -- 172.21.15.5:0/2926656618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f910802f830 con 0x55b5eeb03580 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 10 client.17742 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 20 client.17742 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb83410 0x55b5eeb85830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 4 client.17742 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5eea6fa70 con 0x55b5eeb83410 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.332+0000 7f911d017640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb83410 0x55b5eeb85830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9100ff9640 10 client.17742 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f911d017640 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb83410 0x55b5eeb85830 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257e229062265761 server_cookie=99811647edb9b1d0 in_seq=0 out_seq=0 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9100ff9640 1 -- 172.21.15.5:0/2926656618 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b5eea6fa70 con 0x55b5eeb83410 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9100ff9640 10 client.17742 handle_command_reply: tid=0 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 client.17742 shutdown 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 msgr2=0x55b5eeb065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb065e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9108015040 tx=0x7f910800b520 comp rx=0 tx=0).stop 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 shutdown_connections 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b5eeb83410 0x55b5eeb85830 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5eeb0ca80 0x55b5eeb07060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b5eeb03ea0 0x55b5eeb06b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9100ff9640 0 client.17742 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 --2- 172.21.15.5:0/2926656618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5eeb03580 0x55b5eeb065e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 >> 172.21.15.5:0/2926656618 conn(0x55b5eeafdcb0 msgr2=0x55b5eeb09a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 shutdown_connections 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 1 -- 172.21.15.5:0/2926656618 wait complete. 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.333+0000 7f9121e72740 20 client.17742 trim_cache size 0 max 16384 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:32:43.338 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f91040a3d20 msgr2=0x7f91040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f91040a3d20 0x7f91040a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f911009c8f0 tx=0x7f911006f450 comp rx=0 tx=0).stop 2024-04-22T13:32:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 msgr2=0x7f911819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 0x7f911819a7b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9110009fd0 tx=0x7f911000ee60 comp rx=0 tx=0).stop 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 shutdown_connections 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f911810e8f0 0x7f911819a7b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f91040a3d20 0x7f91040a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118101660 0x7f91181a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 --2- 172.21.15.5:0/2499864481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9118100d40 0x7f91181a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 >> 172.21.15.5:0/2499864481 conn(0x7f9118077fa0 msgr2=0x7f911810fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 shutdown_connections 2024-04-22T13:32:43.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.334+0000 7f911f27b640 1 -- 172.21.15.5:0/2499864481 wait complete. 2024-04-22T13:32:43.341 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:32:43.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:32:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 -- 172.21.15.5:0/2301817450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 msgr2=0x7f7ea410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 --2- 172.21.15.5:0/2301817450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea410ee40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009900 tx=0x7f7e9402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 -- 172.21.15.5:0/2301817450 shutdown_connections 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 --2- 172.21.15.5:0/2301817450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea410ee40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 --2- 172.21.15.5:0/2301817450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea4102b50 0x7f7ea4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 --2- 172.21.15.5:0/2301817450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea41021a0 0x7f7ea4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 -- 172.21.15.5:0/2301817450 >> 172.21.15.5:0/2301817450 conn(0x7f7ea40793e0 msgr2=0x7f7ea40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 -- 172.21.15.5:0/2301817450 shutdown_connections 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.449+0000 7f7eaac25640 1 -- 172.21.15.5:0/2301817450 wait complete. 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 Processor -- start 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 -- start start 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea41021a0 0x7f7ea4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea4074cd0 0x7f7ea41a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4112210 con 0x7f7ea4107270 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4112090 con 0x7f7ea41021a0 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7eaac25640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4112390 con 0x7f7ea4074cd0 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36008/0 (socket says 172.21.15.5:36008) 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 -- 172.21.15.5:0/3596141378 learned_addr learned my addr 172.21.15.5:0/3596141378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea91c2640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea4074cd0 0x7f7ea41a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea89c1640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea41021a0 0x7f7ea4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 -- 172.21.15.5:0/3596141378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea4074cd0 msgr2=0x7f7ea41a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea4074cd0 0x7f7ea41a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 -- 172.21.15.5:0/3596141378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea41021a0 msgr2=0x7f7ea4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea41021a0 0x7f7ea4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea3fff640 1 -- 172.21.15.5:0/3596141378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea41a6f60 con 0x7f7ea4107270 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.450+0000 7f7ea89c1640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea41021a0 0x7f7ea4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.451+0000 7f7ea3fff640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea4074790 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c970 tx=0x7f7e8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.451+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c007bb0 con 0x7f7ea4107270 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.451+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c007d50 con 0x7f7ea4107270 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.451+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41a7250 con 0x7f7ea4107270 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.451+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00f450 con 0x7f7ea4107270 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.451+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41a77a0 con 0x7f7ea4107270 2024-04-22T13:32:43.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.452+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4069250 con 0x7f7ea4107270 2024-04-22T13:32:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.453+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7e8c004f20 con 0x7f7ea4107270 2024-04-22T13:32:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.453+0000 7f7ea1ffb640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e900a3d50 0x7f7e900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.454+0000 7f7ea89c1640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e900a3d50 0x7f7e900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.454+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f7e8c0d0400 con 0x7f7ea4107270 2024-04-22T13:32:43.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.454+0000 7f7ea89c1640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e900a3d50 0x7f7e900a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005fd0 tx=0x7f7e9c009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.458+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e8c014030 con 0x7f7ea4107270 2024-04-22T13:32:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.688+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ea410a4a0 con 0x7f7ea4107270 2024-04-22T13:32:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.689+0000 7f7ea1ffb640 1 -- 172.21.15.5:0/3596141378 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f7e8c09cad0 con 0x7f7ea4107270 2024-04-22T13:32:43.685 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:32:43.685 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:32:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:32:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e900a3d50 msgr2=0x7f7e900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e900a3d50 0x7f7e900a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005fd0 tx=0x7f7e9c009500 comp rx=0 tx=0).stop 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 msgr2=0x7f7ea4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea4074790 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c970 tx=0x7f7e8c00ce30 comp rx=0 tx=0).stop 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 shutdown_connections 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea4107270 0x7f7ea4074790 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e900a3d50 0x7f7e900a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea4074cd0 0x7f7ea41a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 --2- 172.21.15.5:0/3596141378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea41021a0 0x7f7ea4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 >> 172.21.15.5:0/3596141378 conn(0x7f7ea40793e0 msgr2=0x7f7ea40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 shutdown_connections 2024-04-22T13:32:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.691+0000 7f7eaac25640 1 -- 172.21.15.5:0/3596141378 wait complete. 2024-04-22T13:32:43.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:32:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/1975334689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f4103b70 msgr2=0x7ff0f410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/1975334689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f4103b70 0x7ff0f410bfc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009900 tx=0x7ff0e802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/1975334689 shutdown_connections 2024-04-22T13:32:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/1975334689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f4103b70 0x7ff0f410bfc0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/1975334689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f410c6d0 0x7ff0f410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/1975334689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f41031c0 0x7ff0f41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/1975334689 >> 172.21.15.5:0/1975334689 conn(0x7ff0f4077f70 msgr2=0x7ff0f4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/1975334689 shutdown_connections 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.807+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/1975334689 wait complete. 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 Processor -- start 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 -- start start 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 0x7ff0f419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f410c6d0 0x7ff0f419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f41987c0 0x7ff0f4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4112330 con 0x7ff0f41031c0 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41121b0 con 0x7ff0f41987c0 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fc4ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41124b0 con 0x7ff0f410c6d0 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fa256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 0x7ff0f419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fa256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 0x7ff0f419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36026/0 (socket says 172.21.15.5:36026) 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.808+0000 7ff0fa256640 1 -- 172.21.15.5:0/3369010018 learned_addr learned my addr 172.21.15.5:0/3369010018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0faa57640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f41987c0 0x7ff0f4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0f9a55640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f410c6d0 0x7ff0f419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fa256640 1 -- 172.21.15.5:0/3369010018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f410c6d0 msgr2=0x7ff0f419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fa256640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f410c6d0 0x7ff0f419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fa256640 1 -- 172.21.15.5:0/3369010018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f41987c0 msgr2=0x7ff0f4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fa256640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f41987c0 0x7ff0f4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fa256640 1 -- 172.21.15.5:0/3369010018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4199350 con 0x7ff0f41031c0 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0faa57640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f41987c0 0x7ff0f4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0f9a55640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f410c6d0 0x7ff0f419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fa256640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 0x7ff0f419e650 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00d950 tx=0x7ff0ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0f37fe640 1 -- 172.21.15.5:0/3369010018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec017030 con 0x7ff0f41031c0 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0f37fe640 1 -- 172.21.15.5:0/3369010018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ec004020 con 0x7ff0f41031c0 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f4199640 con 0x7ff0f41031c0 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0f37fe640 1 -- 172.21.15.5:0/3369010018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0047a0 con 0x7ff0f41031c0 2024-04-22T13:32:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.809+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a5870 con 0x7ff0f41031c0 2024-04-22T13:32:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.811+0000 7ff0f37fe640 1 -- 172.21.15.5:0/3369010018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff0ec009a30 con 0x7ff0f41031c0 2024-04-22T13:32:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 Processor -- start 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 -- start start 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 0x561a882858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a8828e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a8828ea00 0x561a88290e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a881ea1f0 con 0x561a882854d0 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a881874b0 con 0x561a8828ea00 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0ff187740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561a881e61b0 con 0x561a88285e80 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0f37fe640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff0dc0a3da0 0x7ff0dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 0x561a882858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 0x561a882858b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36036/0 (socket says 172.21.15.5:36036) 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0faa57640 1 -- 172.21.15.5:0/2322171156 learned_addr learned my addr 172.21.15.5:0/2322171156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0fa256640 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a8828e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0faa57640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff0dc0a3da0 0x7ff0dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.812+0000 7ff0f9a55640 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a8828ea00 0x561a88290e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0f37fe640 1 -- 172.21.15.5:0/3369010018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff0ec0d0270 con 0x7ff0f41031c0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 -- 172.21.15.5:0/2322171156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 msgr2=0x561a8828e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a8828e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 -- 172.21.15.5:0/2322171156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a8828ea00 msgr2=0x561a88290e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a8828ea00 0x561a88290e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 -- 172.21.15.5:0/2322171156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a881791b0 con 0x561a882854d0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 0x561a882858b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4012d00 tx=0x7ff0e400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308db62c8d302377 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0faa57640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff0dc0a3da0 0x7ff0dc0a6260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400fc30 tx=0x7ff0e401a000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0f0bfe640 1 -- 172.21.15.5:0/2322171156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4020070 con 0x561a882854d0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0f0bfe640 1 -- 172.21.15.5:0/2322171156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e400ab20 con 0x561a882854d0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.813+0000 7ff0f0bfe640 1 -- 172.21.15.5:0/2322171156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400d3d0 con 0x561a882854d0 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- 172.21.15.5:0/2322171156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 msgr2=0x561a882858b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 0x561a882858b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4012d00 tx=0x7ff0e400f730 comp rx=0 tx=0).stop 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- 172.21.15.5:0/2322171156 shutdown_connections 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a8828ea00 0x561a88290e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a8828e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- 172.21.15.5:0/2322171156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a882854d0 0x561a882858b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- 172.21.15.5:0/2322171156 >> 172.21.15.5:0/2322171156 conn(0x561a8827fba0 msgr2=0x561a8827ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- 172.21.15.5:0/2322171156 shutdown_connections 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- 172.21.15.5:0/2322171156 wait complete. 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- start start 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 Processor -- start 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a8828e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a8828b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.814+0000 7ff0ff187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a88291910 con 0x561a8828ea00 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0ff187740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561a881a3500 con 0x561a88285e80 2024-04-22T13:32:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0ff187740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561a88291a90 con 0x561a882854d0 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0ff187740 10 client.0 _ll_register_callbacks cb 0x561a88171b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36038/0 (socket says 172.21.15.5:36038) 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 -- 172.21.15.5:0/4268683891 learned_addr learned my addr 172.21.15.5:0/4268683891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0f9a55640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a8828b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a8828e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 -- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 msgr2=0x561a8828e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a8828e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 -- 172.21.15.5:0/4268683891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 msgr2=0x561a8828b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a8828b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 -- 172.21.15.5:0/4268683891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a87efc3f0 con 0x561a8828ea00 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a8828e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0faa57640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828be80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff0e401cc90 tx=0x7ff0e401cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a80783e66ac20ff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4268683891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4020040 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4268683891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff0e401c430 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e400ab20 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4268683891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0e400d3d0 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0ff187740 10 client.17760 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.815+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a87fc67d0 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4268683891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e400ab20 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4268683891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e401b070 con 0x561a8828ea00 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0ff187740 10 client.17760 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0ff187740 10 client.17760 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0ff187740 20 client.17760 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a8829ec30 0x561a882a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0ff187740 4 client.17760 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a88189930 con 0x561a8829ec30 2024-04-22T13:32:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.816+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a8829ec30 0x561a882a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0d9ffb640 10 client.17760 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a8829ec30 0x561a882a1020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d6f6169810b5bcec server_cookie=5e4127e81642e9fd in_seq=0 out_seq=0 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4268683891 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a88189930 con 0x561a8829ec30 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0d9ffb640 10 client.17760 handle_command_reply: tid=0 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 client.17760 shutdown 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 msgr2=0x561a8828be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828be80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff0e401cc90 tx=0x7ff0e401cf60 comp rx=0 tx=0).stop 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 shutdown_connections 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a8829ec30 0x561a882a1020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828be80 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a8828b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0d9ffb640 0 client.17760 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4268683891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a8828e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 >> 172.21.15.5:0/4268683891 conn(0x561a8827fba0 msgr2=0x561a882818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.817+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 shutdown_connections 2024-04-22T13:32:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.818+0000 7ff0ff187740 1 -- 172.21.15.5:0/4268683891 wait complete. 2024-04-22T13:32:43.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.818+0000 7ff0ff187740 20 client.17760 trim_cache size 0 max 16384 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 Processor -- start 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 -- start start 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a882854d0 0x561a8829a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a88281680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41adcb0 con 0x561a8828ea00 2024-04-22T13:32:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41af7f0 con 0x561a882854d0 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0ff187740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41ae380 con 0x561a88285e80 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36052/0 (socket says 172.21.15.5:36052) 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 -- 172.21.15.5:0/1113611667 learned_addr learned my addr 172.21.15.5:0/1113611667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0fa256640 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a882854d0 0x561a8829a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0f9a55640 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a88281680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 -- 172.21.15.5:0/1113611667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 msgr2=0x561a88281680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a88281680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 -- 172.21.15.5:0/1113611667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a882854d0 msgr2=0x561a8829a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a882854d0 0x561a8829a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.831+0000 7ff0faa57640 1 -- 172.21.15.5:0/1113611667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a882912b0 con 0x561a8828ea00 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0faa57640 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828dec0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400a930 tx=0x7ff0e4004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e18d263c6d8188f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0dbfff640 1 -- 172.21.15.5:0/1113611667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e403a030 con 0x561a8828ea00 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0dbfff640 1 -- 172.21.15.5:0/1113611667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e400a490 con 0x561a8828ea00 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0dbfff640 1 -- 172.21.15.5:0/1113611667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4020070 con 0x561a8828ea00 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 -- 172.21.15.5:0/1113611667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 msgr2=0x561a8828dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828dec0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400a930 tx=0x7ff0e4004a00 comp rx=0 tx=0).stop 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 -- 172.21.15.5:0/1113611667 shutdown_connections 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a8828dec0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a88285e80 0x561a88281680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 --2- 172.21.15.5:0/1113611667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a882854d0 0x561a8829a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 -- 172.21.15.5:0/1113611667 >> 172.21.15.5:0/1113611667 conn(0x561a8827fba0 msgr2=0x561a88280f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 -- 172.21.15.5:0/1113611667 shutdown_connections 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.832+0000 7ff0ff187740 1 -- 172.21.15.5:0/1113611667 wait complete. 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 -- start start 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 Processor -- start 2024-04-22T13:32:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a882884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a88288a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a88288f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41af120 con 0x561a8828ea00 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41aea50 con 0x561a88285e80 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41abcb0 con 0x561a882854d0 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0ff187740 10 client.0 _ll_register_callbacks cb 0x7ff0e4012b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a88288f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a88288f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36056/0 (socket says 172.21.15.5:36056) 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 -- 172.21.15.5:0/4149952903 learned_addr learned my addr 172.21.15.5:0/4149952903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a882884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0f9a55640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a88288a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 -- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 msgr2=0x561a882884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a882884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 -- 172.21.15.5:0/4149952903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 msgr2=0x561a88288a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a88288a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.833+0000 7ff0faa57640 1 -- 172.21.15.5:0/4149952903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a88304150 con 0x561a8828ea00 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0faa57640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a88288f70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff0e401cc90 tx=0x7ff0e40129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781720858c557a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4149952903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4043020 con 0x561a8828ea00 2024-04-22T13:32:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4149952903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff0e400ab20 con 0x561a8828ea00 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e400a490 con 0x561a8828ea00 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4149952903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0e404e050 con 0x561a8828ea00 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 10 client.17766 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a88305430 con 0x561a8828ea00 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4149952903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e400a490 con 0x561a8828ea00 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4149952903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e401b040 con 0x561a8828ea00 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 10 client.17766 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 10 client.17766 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 20 client.17766 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a88305a30 0x561a88307e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 4 client.17766 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.834+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a881f7c10 con 0x561a88305a30 2024-04-22T13:32:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a88305a30 0x561a88307e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0fa256640 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a88305a30 0x561a88307e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6242b48b89ad3fdc server_cookie=6be60221e395bac0 in_seq=0 out_seq=0 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0d9ffb640 10 client.17766 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0d9ffb640 1 -- 172.21.15.5:0/4149952903 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a881f7c10 con 0x561a88305a30 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0d9ffb640 10 client.17766 handle_command_reply: tid=0 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 client.17766 shutdown 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 msgr2=0x561a88288f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a88288f70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff0e401cc90 tx=0x7ff0e40129b0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 shutdown_connections 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x561a88305a30 0x561a88307e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a8828ea00 0x561a88288f70 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a88285e80 0x561a88288a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0d9ffb640 0 client.17766 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:32:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 --2- 172.21.15.5:0/4149952903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a882854d0 0x561a882884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 >> 172.21.15.5:0/4149952903 conn(0x561a8827fba0 msgr2=0x561a88281790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 shutdown_connections 2024-04-22T13:32:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.835+0000 7ff0ff187740 1 -- 172.21.15.5:0/4149952903 wait complete. 2024-04-22T13:32:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0ff187740 20 client.17766 trim_cache size 0 max 16384 2024-04-22T13:32:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff0dc0a3da0 msgr2=0x7ff0dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff0dc0a3da0 0x7ff0dc0a6260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400fc30 tx=0x7ff0e401a000 comp rx=0 tx=0).stop 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 msgr2=0x7ff0f419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 0x7ff0f419e650 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00d950 tx=0x7ff0ec00de10 comp rx=0 tx=0).stop 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 shutdown_connections 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff0dc0a3da0 0x7ff0dc0a6260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0f41987c0 0x7ff0f4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f410c6d0 0x7ff0f419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 --2- 172.21.15.5:0/3369010018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0f41031c0 0x7ff0f419e650 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 >> 172.21.15.5:0/3369010018 conn(0x7ff0f4077f70 msgr2=0x7ff0f40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 shutdown_connections 2024-04-22T13:32:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.836+0000 7ff0fc4ba640 1 -- 172.21.15.5:0/3369010018 wait complete. 2024-04-22T13:32:43.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:32:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 -- 172.21.15.5:0/4277743821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 msgr2=0x7ff6e0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 --2- 172.21.15.5:0/4277743821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e0102ec0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009930 tx=0x7ff6d402bc20 comp rx=0 tx=0).stop 2024-04-22T13:32:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 -- 172.21.15.5:0/4277743821 shutdown_connections 2024-04-22T13:32:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 --2- 172.21.15.5:0/4277743821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e0102ec0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 --2- 172.21.15.5:0/4277743821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0103400 0x7ff6e010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 --2- 172.21.15.5:0/4277743821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6e0101180 0x7ff6e0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 -- 172.21.15.5:0/4277743821 >> 172.21.15.5:0/4277743821 conn(0x7ff6e0077f70 msgr2=0x7ff6e0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 -- 172.21.15.5:0/4277743821 shutdown_connections 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 -- 172.21.15.5:0/4277743821 wait complete. 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 Processor -- start 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.951+0000 7ff6e6870640 1 -- start start 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e6870640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6e0101180 0x7ff6e019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e6870640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e6870640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0103400 0x7ff6e0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e6870640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e0112300 con 0x7ff6e0102a60 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e6870640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e0112180 con 0x7ff6e0101180 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e6870640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e0112480 con 0x7ff6e0103400 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e019e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36074/0 (socket says 172.21.15.5:36074) 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 -- 172.21.15.5:0/2243760473 learned_addr learned my addr 172.21.15.5:0/2243760473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0103400 0x7ff6e0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6dffff640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6e0101180 0x7ff6e019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 -- 172.21.15.5:0/2243760473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0103400 msgr2=0x7ff6e0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0103400 0x7ff6e0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 -- 172.21.15.5:0/2243760473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6e0101180 msgr2=0x7ff6e019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6e0101180 0x7ff6e019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 -- 172.21.15.5:0/2243760473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e0198dc0 con 0x7ff6e0102a60 2024-04-22T13:32:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.952+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e019e980 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009230 tx=0x7ff6d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.953+0000 7ff6dd7fa640 1 -- 172.21.15.5:0/2243760473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4043020 con 0x7ff6e0102a60 2024-04-22T13:32:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.953+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e0199050 con 0x7ff6e0102a60 2024-04-22T13:32:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.953+0000 7ff6dd7fa640 1 -- 172.21.15.5:0/2243760473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d402cb60 con 0x7ff6e0102a60 2024-04-22T13:32:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.953+0000 7ff6dd7fa640 1 -- 172.21.15.5:0/2243760473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4003b50 con 0x7ff6e0102a60 2024-04-22T13:32:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.953+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e01a5300 con 0x7ff6e0102a60 2024-04-22T13:32:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.955+0000 7ff6dd7fa640 1 -- 172.21.15.5:0/2243760473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff6d4046020 con 0x7ff6e0102a60 2024-04-22T13:32:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.955+0000 7ff6e945b740 1 Processor -- start 2024-04-22T13:32:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.955+0000 7ff6e945b740 1 -- start start 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.955+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0232c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.955+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02335820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc02335e40 0x55dc023362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6e945b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc022911f0 con 0x55dc0232cce0 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6e945b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc0222e4b0 con 0x55dc02335e40 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc0228d1b0 con 0x55dc0232c330 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02335820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dd7fa640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff6c80a3d20 0x7ff6c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02335820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36080/0 (socket says 172.21.15.5:36080) 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 -- 172.21.15.5:0/2967022767 learned_addr learned my addr 172.21.15.5:0/2967022767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0232c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff6c80a3d20 0x7ff6c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 -- 172.21.15.5:0/2967022767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 msgr2=0x55dc0232c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0232c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 -- 172.21.15.5:0/2967022767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc02335e40 msgr2=0x55dc023362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dd7fa640 1 -- 172.21.15.5:0/2243760473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff6d40f27b0 con 0x7ff6e0102a60 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc02335e40 0x55dc023362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc02335e40 0x55dc023362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 -- 172.21.15.5:0/2967022767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc022201b0 con 0x55dc0232cce0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc02335e40 0x55dc023362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6dffff640 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02335820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000cc60 tx=0x7ff6d000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1bbd61cff1abbb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6c67fc640 1 -- 172.21.15.5:0/2967022767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0014070 con 0x55dc0232cce0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6c67fc640 1 -- 172.21.15.5:0/2967022767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d00047a0 con 0x55dc0232cce0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.956+0000 7ff6c67fc640 1 -- 172.21.15.5:0/2967022767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0010690 con 0x55dc0232cce0 2024-04-22T13:32:43.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 -- 172.21.15.5:0/2967022767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 msgr2=0x55dc02335820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff6c80a3d20 0x7ff6c80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8005fd0 tx=0x7ff6d8009600 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02335820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000cc60 tx=0x7ff6d000e720 comp rx=0 tx=0).stop 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 -- 172.21.15.5:0/2967022767 shutdown_connections 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02335820 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc02335e40 0x55dc023362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 --2- 172.21.15.5:0/2967022767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0232c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 -- 172.21.15.5:0/2967022767 >> 172.21.15.5:0/2967022767 conn(0x55dc02326ba0 msgr2=0x55dc02326f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 -- 172.21.15.5:0/2967022767 shutdown_connections 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 -- 172.21.15.5:0/2967022767 wait complete. 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.957+0000 7ff6e945b740 1 -- start start 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 Processor -- start 2024-04-22T13:32:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232c330 0x55dc0233c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02339580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc02335e40 0x55dc02339ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc0233cdc0 con 0x55dc0232cce0 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55dc0224a500 con 0x55dc0232c330 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc0233cf40 con 0x55dc02335e40 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e945b740 10 client.0 _ll_register_callbacks cb 0x55dc02218b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02339580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02339580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36092/0 (socket says 172.21.15.5:36092) 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 -- 172.21.15.5:0/4127660456 learned_addr learned my addr 172.21.15.5:0/4127660456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc02335e40 0x55dc02339ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6dffff640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232c330 0x55dc0233c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 -- 172.21.15.5:0/4127660456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc02335e40 msgr2=0x55dc02339ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc02335e40 0x55dc02339ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 -- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232c330 msgr2=0x55dc0233c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232c330 0x55dc0233c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 -- 172.21.15.5:0/4127660456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc01fa33f0 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02339580 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40f00a0 tx=0x7ff6d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d672fcc4d6c336 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.958+0000 7ff6bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/4127660456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40be890 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/4127660456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff6d40a90b0 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d00047a0 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/4127660456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6d40c2c40 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 10 client.17778 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc0206d7d0 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/4127660456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d00047a0 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/4127660456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff6d40c7070 con 0x55dc0232cce0 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 10 client.17778 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 10 client.17778 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 20 client.17778 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02349f50 0x55dc0234c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 4 client.17778 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc02230930 con 0x55dc02349f50 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.959+0000 7ff6dffff640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02349f50 0x55dc0234c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.960+0000 7ff6dffff640 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02349f50 0x55dc0234c3c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b6d815236d66033 server_cookie=26257e183660fc8 in_seq=0 out_seq=0 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.960+0000 7ff6bf7fe640 10 client.17778 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/4127660456 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc02230930 con 0x55dc02349f50 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6bf7fe640 10 client.17778 handle_command_reply: tid=0 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 client.17778 shutdown 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 msgr2=0x55dc02339580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02339580 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40f00a0 tx=0x7ff6d4009260 comp rx=0 tx=0).stop 2024-04-22T13:32:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 shutdown_connections 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02349f50 0x55dc0234c3c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc0232cce0 0x55dc02339580 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc02335e40 0x55dc02339ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6bf7fe640 0 client.17778 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 --2- 172.21.15.5:0/4127660456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232c330 0x55dc0233c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 >> 172.21.15.5:0/4127660456 conn(0x55dc02326ba0 msgr2=0x55dc023288f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 shutdown_connections 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 1 -- 172.21.15.5:0/4127660456 wait complete. 2024-04-22T13:32:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.961+0000 7ff6e945b740 20 client.17778 trim_cache size 0 max 16384 2024-04-22T13:32:43.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.974+0000 7ff6e945b740 1 Processor -- start 2024-04-22T13:32:43.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.974+0000 7ff6e945b740 1 -- start start 2024-04-22T13:32:43.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.974+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc02345cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.974+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc02334800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc02325a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6e945b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e01ab810 con 0x55dc02326ba0 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6e945b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e01ad350 con 0x55dc0232cce0 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e01abee0 con 0x55dc0232c330 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc02345cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc02345cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36104/0 (socket says 172.21.15.5:36104) 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 -- 172.21.15.5:0/1027228441 learned_addr learned my addr 172.21.15.5:0/1027228441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc02334800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc02325a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 -- 172.21.15.5:0/1027228441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 msgr2=0x55dc02334800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc02334800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 -- 172.21.15.5:0/1027228441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 msgr2=0x55dc02325a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc02325a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 -- 172.21.15.5:0/1027228441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc02337ad0 con 0x55dc02326ba0 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc02325a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6dffff640 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc02345cf0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000cc60 tx=0x7ff6d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e305bdc08fc61626 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6c5ffb640 1 -- 172.21.15.5:0/1027228441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002c070 con 0x55dc02326ba0 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6c5ffb640 1 -- 172.21.15.5:0/1027228441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d0026820 con 0x55dc02326ba0 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.975+0000 7ff6c5ffb640 1 -- 172.21.15.5:0/1027228441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0014040 con 0x55dc02326ba0 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 -- 172.21.15.5:0/1027228441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 msgr2=0x55dc02345cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc02345cf0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000cc60 tx=0x7ff6d000cf60 comp rx=0 tx=0).stop 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 -- 172.21.15.5:0/1027228441 shutdown_connections 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc02325a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc02334800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 --2- 172.21.15.5:0/1027228441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc02345cf0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 -- 172.21.15.5:0/1027228441 >> 172.21.15.5:0/1027228441 conn(0x55dc02339c20 msgr2=0x55dc02325370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 -- 172.21.15.5:0/1027228441 shutdown_connections 2024-04-22T13:32:43.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 -- 172.21.15.5:0/1027228441 wait complete. 2024-04-22T13:32:43.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 -- start start 2024-04-22T13:32:43.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.976+0000 7ff6e945b740 1 Processor -- start 2024-04-22T13:32:43.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc0233b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0233b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc0233bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e01acc80 con 0x55dc02326ba0 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e01ac5b0 con 0x55dc0232cce0 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e01a9810 con 0x55dc0232c330 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 10 client.0 _ll_register_callbacks cb 0x7ff6d40be750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc0233b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc0233b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36110/0 (socket says 172.21.15.5:36110) 2024-04-22T13:32:43.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 -- 172.21.15.5:0/755227199 learned_addr learned my addr 172.21.15.5:0/755227199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:43.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc0233bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0233b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 -- 172.21.15.5:0/755227199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 msgr2=0x55dc0233b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0233b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 -- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 msgr2=0x55dc0233bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc0233bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 -- 172.21.15.5:0/755227199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc023af6c0 con 0x55dc02326ba0 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0233b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e4e0d640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc0233bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6dffff640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc0233b2f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff6d002a040 tx=0x7ff6d000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7048808d8f050e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/755227199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0014070 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/755227199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7ff6d0026580 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.977+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d0026820 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/755227199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6d002c070 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 10 client.17784 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc023b08f0 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/755227199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d0026820 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/755227199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff6d0003a00 con 0x55dc02326ba0 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 10 client.17784 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 10 client.17784 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 20 client.17784 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02335e40 0x55dc023b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 4 client.17784 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc0229ec10 con 0x55dc02335e40 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.978+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02335e40 0x55dc023b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6df7fe640 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02335e40 0x55dc023b2d20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b26a12bdda4dab1 server_cookie=acc38447fd37d337 in_seq=0 out_seq=0 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6bf7fe640 10 client.17784 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6bf7fe640 1 -- 172.21.15.5:0/755227199 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc0229ec10 con 0x55dc02335e40 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6bf7fe640 10 client.17784 handle_command_reply: tid=0 2024-04-22T13:32:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 client.17784 shutdown 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 msgr2=0x55dc0233b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc0233b2f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff6d002a040 tx=0x7ff6d000ed70 comp rx=0 tx=0).stop 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 shutdown_connections 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55dc02335e40 0x55dc023b2d20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55dc0232cce0 0x55dc0233bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc0232c330 0x55dc0233b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6bf7fe640 0 client.17784 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:32:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 --2- 172.21.15.5:0/755227199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc02326ba0 0x55dc0233b2f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 >> 172.21.15.5:0/755227199 conn(0x55dc02339c20 msgr2=0x55dc02325c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 shutdown_connections 2024-04-22T13:32:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 1 -- 172.21.15.5:0/755227199 wait complete. 2024-04-22T13:32:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.979+0000 7ff6e945b740 20 client.17784 trim_cache size 0 max 16384 2024-04-22T13:32:43.986 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff6c80a3d20 msgr2=0x7ff6c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff6c80a3d20 0x7ff6c80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8005fd0 tx=0x7ff6d8009600 comp rx=0 tx=0).stop 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 msgr2=0x7ff6e019e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e019e980 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009230 tx=0x7ff6d40045e0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 shutdown_connections 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6e0102a60 0x7ff6e019e980 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff6c80a3d20 0x7ff6c80a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0103400 0x7ff6e0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 --2- 172.21.15.5:0/2243760473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6e0101180 0x7ff6e019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 >> 172.21.15.5:0/2243760473 conn(0x7ff6e0077f70 msgr2=0x7ff6e0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 shutdown_connections 2024-04-22T13:32:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:43.980+0000 7ff6e6870640 1 -- 172.21.15.5:0/2243760473 wait complete. 2024-04-22T13:32:43.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/1737003169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8102a60 msgr2=0x7f0dc8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/1737003169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8102a60 0x7f0dc8102ec0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009930 tx=0x7f0db802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/1737003169 shutdown_connections 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/1737003169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8102a60 0x7f0dc8102ec0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/1737003169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103400 0x7f0dc810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/1737003169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 0x7f0dc8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/1737003169 >> 172.21.15.5:0/1737003169 conn(0x7f0dc8077f70 msgr2=0x7f0dc8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/1737003169 shutdown_connections 2024-04-22T13:32:44.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.100+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/1737003169 wait complete. 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 Processor -- start 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 -- start start 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 0x7f0dc819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8102a60 0x7f0dc819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 0x7f0dc8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81111a0 con 0x7f0dc8103400 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8111020 con 0x7f0dc8101180 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcfdd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8111320 con 0x7f0dc8102a60 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 0x7f0dc8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 0x7f0dc8198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36124/0 (socket says 172.21.15.5:36124) 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 0x7f0dc819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 -- 172.21.15.5:0/2418091464 learned_addr learned my addr 172.21.15.5:0/2418091464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcd370640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8102a60 0x7f0dc819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 -- 172.21.15.5:0/2418091464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8102a60 msgr2=0x7f0dc819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8102a60 0x7f0dc819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 -- 172.21.15.5:0/2418091464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 msgr2=0x7f0dc819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 0x7f0dc819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dce372640 1 -- 172.21.15.5:0/2418091464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc8198f30 con 0x7f0dc8103400 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.101+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 0x7f0dc819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.102+0000 7f0dce372640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 0x7f0dc8198670 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000eaa0 tx=0x7f0dc000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.102+0000 7f0dbeffd640 1 -- 172.21.15.5:0/2418091464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc000cdb0 con 0x7f0dc8103400 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.102+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc8199220 con 0x7f0dc8103400 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.102+0000 7f0dbeffd640 1 -- 172.21.15.5:0/2418091464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc0005020 con 0x7f0dc8103400 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.102+0000 7f0dbeffd640 1 -- 172.21.15.5:0/2418091464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00116d0 con 0x7f0dc8103400 2024-04-22T13:32:44.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.102+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a5530 con 0x7f0dc8103400 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.104+0000 7f0dbeffd640 1 -- 172.21.15.5:0/2418091464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0dc00054a0 con 0x7f0dc8103400 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.104+0000 7f0dd29c7740 1 Processor -- start 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.104+0000 7f0dd29c7740 1 -- start start 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292006790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x56129200f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x5612920102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561291f6b1f0 con 0x561292006d60 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561291f084b0 con 0x5612920063b0 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561291f671b0 con 0x56129200fe60 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x56129200f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x56129200f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36138/0 (socket says 172.21.15.5:36138) 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 -- 172.21.15.5:0/2802247308 learned_addr learned my addr 172.21.15.5:0/2802247308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcd370640 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292006790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dbeffd640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0db40a3d20 0x7f0db40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcd370640 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x5612920102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 -- 172.21.15.5:0/2802247308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 msgr2=0x5612920102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dce372640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0db40a3d20 0x7f0db40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x5612920102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 -- 172.21.15.5:0/2802247308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 msgr2=0x561292006790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292006790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 -- 172.21.15.5:0/2802247308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561291efa1b0 con 0x561292006d60 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dbeffd640 1 -- 172.21.15.5:0/2418091464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0dc00d08b0 con 0x7f0dc8103400 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.105+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x56129200f890 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40104d0 tx=0x7f0dc4010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6150bc9e72d1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dce372640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0db40a3d20 0x7f0db40a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0dc009c5d0 tx=0x7f0dc0091d60 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0db3fff640 1 -- 172.21.15.5:0/2802247308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004220 con 0x561292006d60 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0db3fff640 1 -- 172.21.15.5:0/2802247308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc40043c0 con 0x561292006d60 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0db3fff640 1 -- 172.21.15.5:0/2802247308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4011a00 con 0x561292006d60 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 -- 172.21.15.5:0/2802247308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 msgr2=0x56129200f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x56129200f890 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40104d0 tx=0x7f0dc4010990 comp rx=0 tx=0).stop 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 -- 172.21.15.5:0/2802247308 shutdown_connections 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x56129200f890 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x5612920102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/2802247308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292006790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.106+0000 7f0dd29c7740 1 -- 172.21.15.5:0/2802247308 >> 172.21.15.5:0/2802247308 conn(0x561292000ba0 msgr2=0x561292000f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 -- 172.21.15.5:0/2802247308 shutdown_connections 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 -- 172.21.15.5:0/2802247308 wait complete. 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 -- start start 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 Processor -- start 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612920063b0 0x561292016280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292013610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56129200fe60 0x561292013b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561292016da0 con 0x561292006d60 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x561291f24500 con 0x56129200fe60 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561292016f20 con 0x5612920063b0 2024-04-22T13:32:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dd29c7740 10 client.0 _ll_register_callbacks cb 0x561291ef2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292013610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292013610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36144/0 (socket says 172.21.15.5:36144) 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.107+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1594006806 learned_addr learned my addr 172.21.15.5:0/1594006806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612920063b0 0x561292016280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56129200fe60 0x561292013b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612920063b0 msgr2=0x561292016280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612920063b0 0x561292016280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1594006806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56129200fe60 msgr2=0x561292013b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56129200fe60 0x561292013b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1594006806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561291c7d3f0 con 0x561292006d60 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292013610 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c00 tx=0x7f0dc4004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1d8efb8ab76d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0db17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0db17fa640 1 -- 172.21.15.5:0/1594006806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4029060 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0db17fa640 1 -- 172.21.15.5:0/1594006806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0dc4005250 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc40043c0 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0db17fa640 1 -- 172.21.15.5:0/1594006806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0dc4005450 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dd29c7740 10 client.17796 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561291d477d0 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.108+0000 7f0db17fa640 1 -- 172.21.15.5:0/1594006806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc40043c0 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0db17fa640 1 -- 172.21.15.5:0/1594006806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0dc4003850 con 0x561292006d60 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dd29c7740 10 client.17796 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dd29c7740 10 client.17796 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dd29c7740 20 client.17796 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x561292023f90 0x561292026380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dd29c7740 4 client.17796 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561291f0a930 con 0x561292023f90 2024-04-22T13:32:44.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x561292023f90 0x561292026380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0db17fa640 10 client.17796 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.109+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x561292023f90 0x561292026380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccd77416f6b2b641 server_cookie=2787ec017b0fa8c8 in_seq=0 out_seq=0 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0db17fa640 1 -- 172.21.15.5:0/1594006806 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561291f0a930 con 0x561292023f90 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0db17fa640 10 client.17796 handle_command_reply: tid=0 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 client.17796 shutdown 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 msgr2=0x561292013610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292013610 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c00 tx=0x7f0dc4004230 comp rx=0 tx=0).stop 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 shutdown_connections 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x561292023f90 0x561292026380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292013610 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56129200fe60 0x561292013b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1594006806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612920063b0 0x561292016280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0db17fa640 0 client.17796 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 >> 172.21.15.5:0/1594006806 conn(0x561292000ba0 msgr2=0x5612920028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 shutdown_connections 2024-04-22T13:32:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.110+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1594006806 wait complete. 2024-04-22T13:32:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.111+0000 7f0dd29c7740 20 client.17796 trim_cache size 0 max 16384 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 Processor -- start 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 -- start start 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292011250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561292006d60 0x56129201f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 0x561292015d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81aba30 con 0x56129200fe60 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81ad570 con 0x5612920063b0 2024-04-22T13:32:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81ac100 con 0x561292006d60 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dcd370640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 0x561292015d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dcd370640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 0x561292015d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36160/0 (socket says 172.21.15.5:36160) 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dcd370640 1 -- 172.21.15.5:0/4116827686 learned_addr learned my addr 172.21.15.5:0/4116827686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dcd370640 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292011250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.124+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561292006d60 0x56129201f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dcd370640 1 -- 172.21.15.5:0/4116827686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561292006d60 msgr2=0x56129201f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dcd370640 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561292006d60 0x56129201f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dcd370640 1 -- 172.21.15.5:0/4116827686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 msgr2=0x561292011250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dcd370640 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292011250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dcd370640 1 -- 172.21.15.5:0/4116827686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561292011b60 con 0x56129200fe60 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dcd370640 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 0x561292015d40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005e00 tx=0x7f0db8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946e75fbee5072a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0db37fe640 1 -- 172.21.15.5:0/4116827686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db802c8a0 con 0x56129200fe60 2024-04-22T13:32:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0db37fe640 1 -- 172.21.15.5:0/4116827686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db80054f0 con 0x56129200fe60 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0db37fe640 1 -- 172.21.15.5:0/4116827686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db804d490 con 0x56129200fe60 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dd29c7740 1 -- 172.21.15.5:0/4116827686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 msgr2=0x561292015d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.125+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 0x561292015d40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005e00 tx=0x7f0db8005d90 comp rx=0 tx=0).stop 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- 172.21.15.5:0/4116827686 shutdown_connections 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56129200fe60 0x561292015d40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561292006d60 0x56129201f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/4116827686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292011250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- 172.21.15.5:0/4116827686 >> 172.21.15.5:0/4116827686 conn(0x561292000ba0 msgr2=0x56129201f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- 172.21.15.5:0/4116827686 shutdown_connections 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- 172.21.15.5:0/4116827686 wait complete. 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- start start 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 Processor -- start 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292029870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292026c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x561292027210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81acea0 con 0x561292006d60 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81ac7d0 con 0x5612920063b0 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81a9a10 con 0x56129200fe60 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dd29c7740 10 client.0 _ll_register_callbacks cb 0x7f0dc400bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292026c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dcdb71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292026c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36172/0 (socket says 172.21.15.5:36172) 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.126+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1792391041 learned_addr learned my addr 172.21.15.5:0/1792391041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292029870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x561292027210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 msgr2=0x561292027210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x561292027210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1792391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 msgr2=0x561292029870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292029870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcdb71640 1 -- 172.21.15.5:0/1792391041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561292027a10 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292029870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dcdb71640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292026c00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x561292014be0 tx=0x7f0dc4004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722b57272e577d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0db17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0db17fa640 1 -- 172.21.15.5:0/1792391041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4029040 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0db17fa640 1 -- 172.21.15.5:0/1792391041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f0dc4005250 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db80054f0 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0db17fa640 1 -- 172.21.15.5:0/1792391041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0dc403e050 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dd29c7740 10 client.17802 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56129208a270 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0db17fa640 1 -- 172.21.15.5:0/1792391041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db80054f0 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.127+0000 7f0db17fa640 1 -- 172.21.15.5:0/1792391041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0dc40115d0 con 0x561292006d60 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 10 client.17802 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 10 client.17802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 20 client.17802 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x56129208a930 0x56129208cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 4 client.17802 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561292013ee0 con 0x56129208a930 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x56129208a930 0x56129208cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dcd370640 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x56129208a930 0x56129208cd50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2b60669ade991ae server_cookie=a845b2d64eabeda7 in_seq=0 out_seq=0 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0db17fa640 10 client.17802 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0db17fa640 1 -- 172.21.15.5:0/1792391041 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561292013ee0 con 0x56129208a930 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0db17fa640 10 client.17802 handle_command_reply: tid=0 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.128+0000 7f0dd29c7740 1 client.17802 shutdown 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 msgr2=0x561292026c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292026c00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x561292014be0 tx=0x7f0dc4004660 comp rx=0 tx=0).stop 2024-04-22T13:32:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 shutdown_connections 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x56129208a930 0x56129208cd50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561292006d60 0x561292026c00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56129200fe60 0x561292027210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0db17fa640 0 client.17802 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 --2- 172.21.15.5:0/1792391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5612920063b0 0x561292029870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 >> 172.21.15.5:0/1792391041 conn(0x561292000ba0 msgr2=0x56129201fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 shutdown_connections 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 1 -- 172.21.15.5:0/1792391041 wait complete. 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dd29c7740 20 client.17802 trim_cache size 0 max 16384 2024-04-22T13:32:44.131 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0db40a3d20 msgr2=0x7f0db40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0db40a3d20 0x7f0db40a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0dc009c5d0 tx=0x7f0dc0091d60 comp rx=0 tx=0).stop 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 msgr2=0x7f0dc8198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 0x7f0dc8198670 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000eaa0 tx=0x7f0dc000ef60 comp rx=0 tx=0).stop 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 shutdown_connections 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8103400 0x7f0dc8198670 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0db40a3d20 0x7f0db40a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8102a60 0x7f0dc819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 --2- 172.21.15.5:0/2418091464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0dc8101180 0x7f0dc819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 >> 172.21.15.5:0/2418091464 conn(0x7f0dc8077f70 msgr2=0x7f0dc806aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 shutdown_connections 2024-04-22T13:32:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.129+0000 7f0dcfdd5640 1 -- 172.21.15.5:0/2418091464 wait complete. 2024-04-22T13:32:44.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:32:44.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 -- 172.21.15.5:0/885795432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea0103b70 msgr2=0x7f7ea010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 --2- 172.21.15.5:0/885795432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea0103b70 0x7f7ea010bfc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009900 tx=0x7f7e9402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:44.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 -- 172.21.15.5:0/885795432 shutdown_connections 2024-04-22T13:32:44.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 --2- 172.21.15.5:0/885795432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea0103b70 0x7f7ea010bfc0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 --2- 172.21.15.5:0/885795432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea010c6d0 0x7f7ea010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 --2- 172.21.15.5:0/885795432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea01031c0 0x7f7ea01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.244+0000 7f7ea760f640 1 -- 172.21.15.5:0/885795432 >> 172.21.15.5:0/885795432 conn(0x7f7ea0077f70 msgr2=0x7f7ea0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 -- 172.21.15.5:0/885795432 shutdown_connections 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 -- 172.21.15.5:0/885795432 wait complete. 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 Processor -- start 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 -- start start 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 0x7f7ea019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea010c6d0 0x7f7ea019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea0198770 0x7f7ea0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea0112350 con 0x7f7ea01031c0 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01121d0 con 0x7f7ea0198770 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea760f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01124d0 con 0x7f7ea010c6d0 2024-04-22T13:32:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.245+0000 7f7ea53ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 0x7f7ea019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 0x7f7ea019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36194/0 (socket says 172.21.15.5:36194) 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 -- 172.21.15.5:0/609966973 learned_addr learned my addr 172.21.15.5:0/609966973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea010c6d0 0x7f7ea019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea0198770 0x7f7ea0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 -- 172.21.15.5:0/609966973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea010c6d0 msgr2=0x7f7ea019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea010c6d0 0x7f7ea019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 -- 172.21.15.5:0/609966973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea0198770 msgr2=0x7f7ea0198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea0198770 0x7f7ea0198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 -- 172.21.15.5:0/609966973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea0199370 con 0x7f7ea01031c0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 0x7f7ea019e660 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c920 tx=0x7f7e9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/609966973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98007c70 con 0x7f7ea01031c0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea01a5400 con 0x7f7ea01031c0 2024-04-22T13:32:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/609966973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e98007e10 con 0x7f7ea01031c0 2024-04-22T13:32:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/609966973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98018820 con 0x7f7ea01031c0 2024-04-22T13:32:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.246+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea01a57b0 con 0x7f7ea01031c0 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/609966973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7e9800f450 con 0x7f7ea01031c0 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 Processor -- start 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 -- start start 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6d330 0x555866a6d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a76820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5558669d21f0 con 0x555866a6dce0 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55586696f4b0 con 0x555866a6d330 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5558669ce1b0 con 0x555866a76e40 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea53ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea53ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a76820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36202/0 (socket says 172.21.15.5:36202) 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea53ab640 1 -- 172.21.15.5:0/3809194903 learned_addr learned my addr 172.21.15.5:0/3809194903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6d330 0x555866a6d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7e9e7fc640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e880a3d20 0x7f7e880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea53ab640 1 -- 172.21.15.5:0/3809194903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 msgr2=0x555866a772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.249+0000 7f7ea53ab640 1 -- 172.21.15.5:0/3809194903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6d330 msgr2=0x555866a6d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6d330 0x555866a6d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7ea53ab640 1 -- 172.21.15.5:0/3809194903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558669611b0 con 0x555866a6dce0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6d330 0x555866a6d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e880a3d20 0x7f7e880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/609966973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f7e980d0900 con 0x7f7ea01031c0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a76820 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7e980cdff0 tx=0x7f7e9800fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a29f4b6285a8d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7e877fe640 1 -- 172.21.15.5:0/3809194903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9809c410 con 0x555866a6dce0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e880a3d20 0x7f7e880a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005fd0 tx=0x7f7e90004a40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7e877fe640 1 -- 172.21.15.5:0/3809194903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9809cfd0 con 0x555866a6dce0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.250+0000 7f7e877fe640 1 -- 172.21.15.5:0/3809194903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98008230 con 0x555866a6dce0 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/3809194903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 msgr2=0x555866a76820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a76820 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7e980cdff0 tx=0x7f7e9800fe60 comp rx=0 tx=0).stop 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/3809194903 shutdown_connections 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a76820 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/3809194903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6d330 0x555866a6d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/3809194903 >> 172.21.15.5:0/3809194903 conn(0x555866a67ba0 msgr2=0x555866a67f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/3809194903 shutdown_connections 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/3809194903 wait complete. 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 -- start start 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.251+0000 7f7eaa1fd740 1 Processor -- start 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 0x555866a7d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6dce0 0x555866a7a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a7aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555866a7dd60 con 0x555866a6d330 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55586698b500 con 0x555866a6dce0 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555866a7dee0 con 0x555866a76e40 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 10 client.0 _ll_register_callbacks cb 0x555866959b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 0x555866a7d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 0x555866a7d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36212/0 (socket says 172.21.15.5:36212) 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 -- 172.21.15.5:0/2372447510 learned_addr learned my addr 172.21.15.5:0/2372447510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a7aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6dce0 0x555866a7a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 -- 172.21.15.5:0/2372447510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 msgr2=0x555866a7aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a7aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 -- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6dce0 msgr2=0x555866a7a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6dce0 0x555866a7a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 -- 172.21.15.5:0/2372447510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558666e43f0 con 0x555866a6d330 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 0x555866a7d1a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7e980ce680 tx=0x7f7e980d4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ddf51044c63c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7e84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7e84ff9640 1 -- 172.21.15.5:0/2372447510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980131b0 con 0x555866a6d330 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7e84ff9640 1 -- 172.21.15.5:0/2372447510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f7e9800fa90 con 0x555866a6d330 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.252+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9809cfd0 con 0x555866a6d330 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7e84ff9640 1 -- 172.21.15.5:0/2372447510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e9801b120 con 0x555866a6d330 2024-04-22T13:32:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 10 client.17814 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558667ae7d0 con 0x555866a6d330 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7e84ff9640 1 -- 172.21.15.5:0/2372447510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9809cfd0 con 0x555866a6d330 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7e84ff9640 1 -- 172.21.15.5:0/2372447510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7e98098f40 con 0x555866a6d330 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 10 client.17814 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 10 client.17814 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 20 client.17814 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866a8ae30 0x555866a8d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 4 client.17814 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555866971930 con 0x555866a8ae30 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.253+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866a8ae30 0x555866a8d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.254+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866a8ae30 0x555866a8d220 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=466173ccb66e6d67 server_cookie=ecbcef9d9d3ea0ff in_seq=0 out_seq=0 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.254+0000 7f7e84ff9640 10 client.17814 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.254+0000 7f7e84ff9640 1 -- 172.21.15.5:0/2372447510 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555866971930 con 0x555866a8ae30 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.254+0000 7f7e84ff9640 10 client.17814 handle_command_reply: tid=0 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.254+0000 7f7eaa1fd740 1 client.17814 shutdown 2024-04-22T13:32:44.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 msgr2=0x555866a7d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 0x555866a7d1a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7e980ce680 tx=0x7f7e980d4000 comp rx=0 tx=0).stop 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 shutdown_connections 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866a8ae30 0x555866a8d220 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a76e40 0x555866a7aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7e84ff9640 0 client.17814 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a6dce0 0x555866a7a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/2372447510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6d330 0x555866a7d1a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 >> 172.21.15.5:0/2372447510 conn(0x555866a67ba0 msgr2=0x555866a698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 shutdown_connections 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/2372447510 wait complete. 2024-04-22T13:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.255+0000 7f7eaa1fd740 20 client.17814 trim_cache size 0 max 16384 2024-04-22T13:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 Processor -- start 2024-04-22T13:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 -- start start 2024-04-22T13:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a86c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01ab840 con 0x555866a6dce0 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01ad380 con 0x555866a76e40 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01abf10 con 0x555866a6d330 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7ea4baa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7ea4baa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36214/0 (socket says 172.21.15.5:36214) 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.268+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1301116972 learned_addr learned my addr 172.21.15.5:0/1301116972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1301116972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 msgr2=0x555866a86c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a86c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a86c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1301116972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 msgr2=0x555866a7d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1301116972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555866a6c560 con 0x555866a6dce0 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7cc60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009850 tx=0x7f7e94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55006883c79b1ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7e86ffd640 1 -- 172.21.15.5:0/1301116972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94004480 con 0x555866a6dce0 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7e86ffd640 1 -- 172.21.15.5:0/1301116972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94050070 con 0x555866a6dce0 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.269+0000 7f7e86ffd640 1 -- 172.21.15.5:0/1301116972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9404b4b0 con 0x555866a6dce0 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1301116972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 msgr2=0x555866a7cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7cc60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009850 tx=0x7f7e94005fd0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1301116972 shutdown_connections 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7cc60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1301116972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a86c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1301116972 >> 172.21.15.5:0/1301116972 conn(0x555866a67ba0 msgr2=0x555866a66c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1301116972 shutdown_connections 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1301116972 wait complete. 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- start start 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 Processor -- start 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01accb0 con 0x555866a6dce0 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01ac5e0 con 0x555866a76e40 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea01a9860 con 0x555866a6d330 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.270+0000 7f7eaa1fd740 10 client.0 _ll_register_callbacks cb 0x7f7e980a0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36218/0 (socket says 172.21.15.5:36218) 2024-04-22T13:32:44.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1410880138 learned_addr learned my addr 172.21.15.5:0/1410880138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1410880138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 msgr2=0x555866a7c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a7c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 msgr2=0x555866a7cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 -- 172.21.15.5:0/1410880138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555866af0160 con 0x555866a6dce0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea5bac640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7ea4baa640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7c7a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7e94036040 tx=0x7f7e9404bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a6ca872fdc48c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7e84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7e84ff9640 1 -- 172.21.15.5:0/1410880138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9402c7e0 con 0x555866a6dce0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7e84ff9640 1 -- 172.21.15.5:0/1410880138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f7e94048040 con 0x555866a6dce0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e94050070 con 0x555866a6dce0 2024-04-22T13:32:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7e84ff9640 1 -- 172.21.15.5:0/1410880138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e9402c980 con 0x555866a6dce0 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7eaa1fd740 10 client.17820 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555866af1400 con 0x555866a6dce0 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.271+0000 7f7e84ff9640 1 -- 172.21.15.5:0/1410880138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94050070 con 0x555866a6dce0 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7e84ff9640 1 -- 172.21.15.5:0/1410880138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7e94003a10 con 0x555866a6dce0 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7eaa1fd740 10 client.17820 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7eaa1fd740 10 client.17820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7eaa1fd740 20 client.17820 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866af1b70 0x555866af3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7eaa1fd740 4 client.17820 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558669dfc10 con 0x555866af1b70 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866af1b70 0x555866af3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7e84ff9640 10 client.17820 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.272+0000 7f7ea53ab640 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866af1b70 0x555866af3f90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=780bc204b7b067d4 server_cookie=7d8f0eb0ffe09b33 in_seq=0 out_seq=0 2024-04-22T13:32:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7e84ff9640 1 -- 172.21.15.5:0/1410880138 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558669dfc10 con 0x555866af1b70 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7e84ff9640 10 client.17820 handle_command_reply: tid=0 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 client.17820 shutdown 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 msgr2=0x555866a7c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7c7a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7e94036040 tx=0x7f7e9404bc50 comp rx=0 tx=0).stop 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 shutdown_connections 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x555866af1b70 0x555866af3f90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555866a6dce0 0x555866a7c7a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555866a76e40 0x555866a7cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7e84ff9640 0 client.17820 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 --2- 172.21.15.5:0/1410880138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555866a6d330 0x555866a7c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 >> 172.21.15.5:0/1410880138 conn(0x555866a67ba0 msgr2=0x555866a68e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 shutdown_connections 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 1 -- 172.21.15.5:0/1410880138 wait complete. 2024-04-22T13:32:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7eaa1fd740 20 client.17820 trim_cache size 0 max 16384 2024-04-22T13:32:44.276 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e880a3d20 msgr2=0x7f7e880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e880a3d20 0x7f7e880a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005fd0 tx=0x7f7e90004a40 comp rx=0 tx=0).stop 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 msgr2=0x7f7ea019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 0x7f7ea019e660 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c920 tx=0x7f7e9800cde0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 shutdown_connections 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7e880a3d20 0x7f7e880a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ea0198770 0x7f7ea0198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea010c6d0 0x7f7ea019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 --2- 172.21.15.5:0/609966973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea01031c0 0x7f7ea019e660 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.273+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 >> 172.21.15.5:0/609966973 conn(0x7f7ea0077f70 msgr2=0x7f7ea00ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.274+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 shutdown_connections 2024-04-22T13:32:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.274+0000 7f7ea760f640 1 -- 172.21.15.5:0/609966973 wait complete. 2024-04-22T13:32:44.278 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:44 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3596141378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:44.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:44 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3596141378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:44.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:32:44.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:44 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3596141378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:44.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 -- 172.21.15.5:0/4050688686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 msgr2=0x7fb504101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 --2- 172.21.15.5:0/4050688686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb504101e10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009930 tx=0x7fb4f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:44.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 -- 172.21.15.5:0/4050688686 shutdown_connections 2024-04-22T13:32:44.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 --2- 172.21.15.5:0/4050688686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb504101e10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 --2- 172.21.15.5:0/4050688686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102350 0x7fb50410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 --2- 172.21.15.5:0/4050688686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb504102d70 0x7fb504103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 -- 172.21.15.5:0/4050688686 >> 172.21.15.5:0/4050688686 conn(0x7fb5040760d0 msgr2=0x7fb5040764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 -- 172.21.15.5:0/4050688686 shutdown_connections 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.387+0000 7fb509a11640 1 -- 172.21.15.5:0/4050688686 wait complete. 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 Processor -- start 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 -- start start 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb50410e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb504102350 0x7fb50410edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102d70 0x7fb50410f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb504112320 con 0x7fb504101990 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041121a0 con 0x7fb504102350 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb509a11640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041124a0 con 0x7fb504102d70 2024-04-22T13:32:44.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.388+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb50410e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb50410e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36230/0 (socket says 172.21.15.5:36230) 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 -- 172.21.15.5:0/1277348803 learned_addr learned my addr 172.21.15.5:0/1277348803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb5027fc640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb504102350 0x7fb50410edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102d70 0x7fb50410f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 -- 172.21.15.5:0/1277348803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102d70 msgr2=0x7fb50410f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102d70 0x7fb50410f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 -- 172.21.15.5:0/1277348803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb504102350 msgr2=0x7fb50410edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb504102350 0x7fb50410edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 -- 172.21.15.5:0/1277348803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a6d60 con 0x7fb504101990 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102d70 0x7fb50410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb50410e8b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400d950 tx=0x7fb4f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb508a0f640 1 -- 172.21.15.5:0/1277348803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4017070 con 0x7fb504101990 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb508a0f640 1 -- 172.21.15.5:0/1277348803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f4009c30 con 0x7fb504101990 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a6ff0 con 0x7fb504101990 2024-04-22T13:32:44.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb508a0f640 1 -- 172.21.15.5:0/1277348803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400f620 con 0x7fb504101990 2024-04-22T13:32:44.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.389+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a74c0 con 0x7fb504101990 2024-04-22T13:32:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb508a0f640 1 -- 172.21.15.5:0/1277348803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb4f4010460 con 0x7fb504101990 2024-04-22T13:32:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 Processor -- start 2024-04-22T13:32:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 -- start start 2024-04-22T13:32:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef7058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef710e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564aef66a1f0 con 0x564aef7054d0 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564aef6074b0 con 0x564aef70ea00 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb50c604740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564aef6661b0 con 0x564aef705e80 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb5037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef7058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb5037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef7058b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36246/0 (socket says 172.21.15.5:36246) 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb5037fe640 1 -- 172.21.15.5:0/2643220048 learned_addr learned my addr 172.21.15.5:0/2643220048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb502ffd640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.392+0000 7fb5027fc640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef710e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 -- 172.21.15.5:0/2643220048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 msgr2=0x564aef70e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb508a0f640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb4ec0a3d20 0x7fb4ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 -- 172.21.15.5:0/2643220048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 msgr2=0x564aef710e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef710e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 -- 172.21.15.5:0/2643220048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aef5f91b0 con 0x564aef7054d0 2024-04-22T13:32:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5027fc640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef710e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb502ffd640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb508a0f640 1 -- 172.21.15.5:0/1277348803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb4f40d0390 con 0x7fb504101990 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb4ec0a3d20 0x7fb4ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb5037fe640 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef7058b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009fd0 tx=0x7fb4fc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f3ad91f6423852 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb4e9ffb640 1 -- 172.21.15.5:0/2643220048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc011940 con 0x564aef7054d0 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb4e9ffb640 1 -- 172.21.15.5:0/2643220048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4fc011d00 con 0x564aef7054d0 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.393+0000 7fb4e9ffb640 1 -- 172.21.15.5:0/2643220048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc01a750 con 0x564aef7054d0 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 -- 172.21.15.5:0/2643220048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 msgr2=0x564aef7058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb4ec0a3d20 0x7fb4ec0a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc01d040 tx=0x7fb4fc01aaf0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef7058b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009fd0 tx=0x7fb4fc0108e0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 -- 172.21.15.5:0/2643220048 shutdown_connections 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef710e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 --2- 172.21.15.5:0/2643220048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef7058b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 -- 172.21.15.5:0/2643220048 >> 172.21.15.5:0/2643220048 conn(0x564aef6ffba0 msgr2=0x564aef6fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 -- 172.21.15.5:0/2643220048 shutdown_connections 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 -- 172.21.15.5:0/2643220048 wait complete. 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 -- start start 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.394+0000 7fb50c604740 1 Processor -- start 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef70e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564aef711970 con 0x564aef7054d0 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564aef623500 con 0x564aef70ea00 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564aef711af0 con 0x564aef705e80 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 10 client.0 _ll_register_callbacks cb 0x564aef5f1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef70e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef70e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36260/0 (socket says 172.21.15.5:36260) 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 -- 172.21.15.5:0/1433083184 learned_addr learned my addr 172.21.15.5:0/1433083184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb5027fc640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 -- 172.21.15.5:0/1433083184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 msgr2=0x564aef70b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 -- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 msgr2=0x564aef70bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 -- 172.21.15.5:0/1433083184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aef37c3f0 con 0x564aef7054d0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb5027fc640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef70e280 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1206120550fa5a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb4e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/1433083184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4020070 con 0x564aef7054d0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/1433083184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb4f40a4b40 con 0x564aef7054d0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc011d00 con 0x564aef7054d0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/1433083184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4f4009de0 con 0x564aef7054d0 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 10 client.17832 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.395+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564aef4467d0 con 0x564aef7054d0 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/1433083184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4fc011d00 con 0x564aef7054d0 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/1433083184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb4f40b5ae0 con 0x564aef7054d0 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb50c604740 10 client.17832 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb50c604740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb50c604740 20 client.17832 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb50c604740 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef71eb80 0x564aef720fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb50c604740 4 client.17832 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aef609930 con 0x564aef71eb80 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.396+0000 7fb5027fc640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef71eb80 0x564aef720fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb4e2ffd640 10 client.17832 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb5027fc640 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef71eb80 0x564aef720fb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58002f1188273f5 server_cookie=cd5c368a0c93c507 in_seq=0 out_seq=0 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/1433083184 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564aef609930 con 0x564aef71eb80 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb4e2ffd640 10 client.17832 handle_command_reply: tid=0 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb50c604740 1 client.17832 shutdown 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 msgr2=0x564aef70e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb50c604740 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef70e280 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f400bf60 comp rx=0 tx=0).stop 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 shutdown_connections 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb50c604740 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef71eb80 0x564aef720fb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb4e2ffd640 0 client.17832 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.397+0000 7fb50c604740 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb50c604740 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb50c604740 1 --2- 172.21.15.5:0/1433083184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef70e280 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 >> 172.21.15.5:0/1433083184 conn(0x564aef6ffba0 msgr2=0x564aef7018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 shutdown_connections 2024-04-22T13:32:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb50c604740 1 -- 172.21.15.5:0/1433083184 wait complete. 2024-04-22T13:32:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.398+0000 7fb50c604740 20 client.17832 trim_cache size 0 max 16384 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 Processor -- start 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 -- start start 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef71a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041afdc0 con 0x564aef7054d0 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b1900 con 0x564aef70ea00 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb50c604740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b0490 con 0x564aef705e80 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef71a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef71a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36274/0 (socket says 172.21.15.5:36274) 2024-04-22T13:32:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.411+0000 7fb502ffd640 1 -- 172.21.15.5:0/1528553358 learned_addr learned my addr 172.21.15.5:0/1528553358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb5027fc640 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb502ffd640 1 -- 172.21.15.5:0/1528553358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 msgr2=0x564aef70dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb502ffd640 1 -- 172.21.15.5:0/1528553358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 msgr2=0x564aef70e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb5037fe640 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb502ffd640 1 -- 172.21.15.5:0/1528553358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aef7215e0 con 0x564aef7054d0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb502ffd640 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef71a830 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb4f409a610 tx=0x7fb4f4098130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582d7bb5c37545a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb4e97fa640 1 -- 172.21.15.5:0/1528553358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4020070 con 0x564aef7054d0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb4e97fa640 1 -- 172.21.15.5:0/1528553358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f4015020 con 0x564aef7054d0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.412+0000 7fb4e97fa640 1 -- 172.21.15.5:0/1528553358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4098200 con 0x564aef7054d0 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- 172.21.15.5:0/1528553358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 msgr2=0x564aef71a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef71a830 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb4f409a610 tx=0x7fb4f4098130 comp rx=0 tx=0).stop 2024-04-22T13:32:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- 172.21.15.5:0/1528553358 shutdown_connections 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef70ea00 0x564aef70e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef70dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- 172.21.15.5:0/1528553358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef7054d0 0x564aef71a830 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- 172.21.15.5:0/1528553358 >> 172.21.15.5:0/1528553358 conn(0x564aef6ffba0 msgr2=0x564aef706af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- 172.21.15.5:0/1528553358 shutdown_connections 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- 172.21.15.5:0/1528553358 wait complete. 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- start start 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 Processor -- start 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef7054d0 0x564aef712fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef713500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 0x564aef77e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b1230 con 0x564aef70ea00 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b0b60 con 0x564aef7054d0 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041adda0 con 0x564aef705e80 2024-04-22T13:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.413+0000 7fb50c604740 10 client.0 _ll_register_callbacks cb 0x7fb4f409c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 0x564aef77e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 0x564aef77e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36278/0 (socket says 172.21.15.5:36278) 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 -- 172.21.15.5:0/2689146513 learned_addr learned my addr 172.21.15.5:0/2689146513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb502ffd640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef7054d0 0x564aef712fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5027fc640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef713500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 -- 172.21.15.5:0/2689146513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 msgr2=0x564aef713500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef713500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 -- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef7054d0 msgr2=0x564aef712fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef7054d0 0x564aef712fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 -- 172.21.15.5:0/2689146513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aef77e640 con 0x564aef70ea00 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb502ffd640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef7054d0 0x564aef712fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb5037fe640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 0x564aef77e050 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc01aff0 tx=0x7fb4fc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fbbaf0d76e1549 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb4e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/2689146513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004c10 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/2689146513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb4fc004db0 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f4015020 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/2689146513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc040050 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb50c604740 10 client.17838 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.414+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564aef7859e0 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/2689146513 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f4015020 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/2689146513 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb4fc00e890 con 0x564aef70ea00 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb50c604740 10 client.17838 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb50c604740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb50c604740 20 client.17838 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb50c604740 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef785f30 0x564aef788350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb50c604740 4 client.17838 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564aef677c10 con 0x564aef785f30 2024-04-22T13:32:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb502ffd640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef785f30 0x564aef788350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb502ffd640 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef785f30 0x564aef788350 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388203a7dd74c3d3 server_cookie=83372b4529192315 in_seq=0 out_seq=0 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.415+0000 7fb4e2ffd640 10 client.17838 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb4e2ffd640 1 -- 172.21.15.5:0/2689146513 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564aef677c10 con 0x564aef785f30 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb4e2ffd640 10 client.17838 handle_command_reply: tid=0 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 client.17838 shutdown 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 msgr2=0x564aef77e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 0x564aef77e050 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc01aff0 tx=0x7fb4fc023000 comp rx=0 tx=0).stop 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 shutdown_connections 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564aef785f30 0x564aef788350 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aef70ea00 0x564aef77e050 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564aef705e80 0x564aef713500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 --2- 172.21.15.5:0/2689146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564aef7054d0 0x564aef712fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb4e2ffd640 0 client.17838 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:44.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 >> 172.21.15.5:0/2689146513 conn(0x564aef6ffba0 msgr2=0x564aef70ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 shutdown_connections 2024-04-22T13:32:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 1 -- 172.21.15.5:0/2689146513 wait complete. 2024-04-22T13:32:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb50c604740 20 client.17838 trim_cache size 0 max 16384 2024-04-22T13:32:44.419 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb4ec0a3d20 msgr2=0x7fb4ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb509a11640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb4ec0a3d20 0x7fb4ec0a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc01d040 tx=0x7fb4fc01aaf0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 msgr2=0x7fb50410e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.416+0000 7fb509a11640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb50410e8b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400d950 tx=0x7fb4f400de10 comp rx=0 tx=0).stop 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 shutdown_connections 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb4ec0a3d20 0x7fb4ec0a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb504102d70 0x7fb50410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb504102350 0x7fb50410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 --2- 172.21.15.5:0/1277348803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb504101990 0x7fb50410e8b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 >> 172.21.15.5:0/1277348803 conn(0x7fb5040760d0 msgr2=0x7fb5040ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 shutdown_connections 2024-04-22T13:32:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:44.417+0000 7fb509a11640 1 -- 172.21.15.5:0/1277348803 wait complete. 2024-04-22T13:32:45.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:45 smithi049 ceph-mon[31379]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:45.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:45 smithi107 ceph-mon[31859]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:45.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:45 smithi005 ceph-mon[28006]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:45.424 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:32:45.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:32:45.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 -- 172.21.15.5:0/3450639723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6550103ae0 msgr2=0x7f655010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 --2- 172.21.15.5:0/3450639723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6550103ae0 0x7f655010bfc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 -- 172.21.15.5:0/3450639723 shutdown_connections 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 --2- 172.21.15.5:0/3450639723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6550103ae0 0x7f655010bfc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 --2- 172.21.15.5:0/3450639723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f655010c500 0x7f655010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 --2- 172.21.15.5:0/3450639723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65501031c0 0x7f65501035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 -- 172.21.15.5:0/3450639723 >> 172.21.15.5:0/3450639723 conn(0x7f6550078000 msgr2=0x7f6550078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 -- 172.21.15.5:0/3450639723 shutdown_connections 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.550+0000 7f6557dce640 1 -- 172.21.15.5:0/3450639723 wait complete. 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 Processor -- start 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 -- start start 2024-04-22T13:32:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 0x7f655019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550103ae0 0x7f655019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f655010c500 0x7f65501985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112190 con 0x7f65501031c0 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112010 con 0x7f6550103ae0 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6557dce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112310 con 0x7f655010c500 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 0x7f655019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 0x7f655019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36302/0 (socket says 172.21.15.5:36302) 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 -- 172.21.15.5:0/831415376 learned_addr learned my addr 172.21.15.5:0/831415376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f655636b640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f655010c500 0x7f65501985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555369640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550103ae0 0x7f655019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 -- 172.21.15.5:0/831415376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f655010c500 msgr2=0x7f65501985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f655010c500 0x7f65501985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 -- 172.21.15.5:0/831415376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550103ae0 msgr2=0x7f655019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550103ae0 0x7f655019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.551+0000 7f6555b6a640 1 -- 172.21.15.5:0/831415376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6550198d40 con 0x7f65501031c0 2024-04-22T13:32:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.552+0000 7f6555b6a640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 0x7f655019e410 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f654800d770 tx=0x7f654800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.552+0000 7f654effd640 1 -- 172.21.15.5:0/831415376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548017070 con 0x7f65501031c0 2024-04-22T13:32:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.552+0000 7f654effd640 1 -- 172.21.15.5:0/831415376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65480054d0 con 0x7f65501031c0 2024-04-22T13:32:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.552+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6550199030 con 0x7f65501031c0 2024-04-22T13:32:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.552+0000 7f654effd640 1 -- 172.21.15.5:0/831415376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548004c30 con 0x7f65501031c0 2024-04-22T13:32:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.552+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a52d0 con 0x7f65501031c0 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.554+0000 7f654effd640 1 -- 172.21.15.5:0/831415376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6548004e10 con 0x7f65501031c0 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 Processor -- start 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 -- start start 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8c8e0 0x55cffee8ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cffedf0f00 con 0x55cffee95e10 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cffed8e590 con 0x55cffee8d290 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f655a9c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cffedecec0 con 0x55cffee8c8e0 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f654effd640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f65380a3d20 0x7f65380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f6555369640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f6555369640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee98260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36304/0 (socket says 172.21.15.5:36304) 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.555+0000 7f6555369640 1 -- 172.21.15.5:0/3963721451 learned_addr learned my addr 172.21.15.5:0/3963721451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555b6a640 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f655636b640 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8c8e0 0x55cffee8ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f655636b640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f65380a3d20 0x7f65380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f654effd640 1 -- 172.21.15.5:0/831415376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f65480d01f0 con 0x7f65501031c0 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555369640 1 -- 172.21.15.5:0/3963721451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8c8e0 msgr2=0x55cffee8ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555369640 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8c8e0 0x55cffee8ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555369640 1 -- 172.21.15.5:0/3963721451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 msgr2=0x55cffee95840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555369640 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee95840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555369640 1 -- 172.21.15.5:0/3963721451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cffebcb7e0 con 0x55cffee95e10 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555b6a640 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6555369640 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee98260 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba79fa4e5274b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f655636b640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f65380a3d20 0x7f65380a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6540009fd0 tx=0x7f65400095f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6537fff640 1 -- 172.21.15.5:0/3963721451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544004180 con 0x55cffee95e10 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6537fff640 1 -- 172.21.15.5:0/3963721451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65440054f0 con 0x55cffee95e10 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.556+0000 7f6537fff640 1 -- 172.21.15.5:0/3963721451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440046c0 con 0x55cffee95e10 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 -- 172.21.15.5:0/3963721451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 msgr2=0x55cffee98260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee98260 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402c760 comp rx=0 tx=0).stop 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 -- 172.21.15.5:0/3963721451 shutdown_connections 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee98260 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3963721451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8c8e0 0x55cffee8ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 -- 172.21.15.5:0/3963721451 >> 172.21.15.5:0/3963721451 conn(0x55cffee86fb0 msgr2=0x55cffee87390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 -- 172.21.15.5:0/3963721451 shutdown_connections 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.557+0000 7f655a9c3740 1 -- 172.21.15.5:0/3963721451 wait complete. 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 -- start start 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 Processor -- start 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 0x55cffee92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee95e10 0x55cffee93800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cffee98d20 con 0x55cffee8c8e0 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cffedaa580 con 0x55cffee8d290 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cffee98ea0 con 0x55cffee95e10 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655a9c3740 10 client.0 _ll_register_callbacks cb 0x55cffedb0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f6555b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 0x55cffee92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f6555b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 0x55cffee92d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36310/0 (socket says 172.21.15.5:36310) 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f6555b6a640 1 -- 172.21.15.5:0/3135723967 learned_addr learned my addr 172.21.15.5:0/3135723967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f655636b640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee95e10 0x55cffee93800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.558+0000 7f6555369640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f6555b6a640 1 -- 172.21.15.5:0/3135723967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee95e10 msgr2=0x55cffee93800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f6555b6a640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee95e10 0x55cffee93800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f6555b6a640 1 -- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 msgr2=0x55cffee932c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f6555b6a640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee932c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f6555b6a640 1 -- 172.21.15.5:0/3135723967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cffec2bf70 con 0x55cffee8c8e0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f6555b6a640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 0x55cffee92d80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f654809bf70 tx=0x7f654809bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2823d10e6e55862a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f65357fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f65357fa640 1 -- 172.21.15.5:0/3135723967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65480a4e30 con 0x55cffee8c8e0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f65357fa640 1 -- 172.21.15.5:0/3135723967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f65480cdbf0 con 0x55cffee8c8e0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65440054f0 con 0x55cffee8c8e0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f65357fa640 1 -- 172.21.15.5:0/3135723967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65480cddf0 con 0x55cffee8c8e0 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f655a9c3740 10 client.17850 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cffeac3170 con 0x55cffee8c8e0 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f65357fa640 1 -- 172.21.15.5:0/3135723967 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65440054f0 con 0x55cffee8c8e0 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f65357fa640 1 -- 172.21.15.5:0/3135723967 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65480a49a0 con 0x55cffee8c8e0 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.559+0000 7f655a9c3740 10 client.17850 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f655a9c3740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f655a9c3740 20 client.17850 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffeea6270 0x55cffeea86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f655a9c3740 4 client.17850 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cffedfe920 con 0x55cffeea6270 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f6555369640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffeea6270 0x55cffeea86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f65357fa640 10 client.17850 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.560+0000 7f6555369640 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffeea6270 0x55cffeea86e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626f91bc8bd54334 server_cookie=4ed190f985c4ec11 in_seq=0 out_seq=0 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f65357fa640 1 -- 172.21.15.5:0/3135723967 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cffedfe920 con 0x55cffeea6270 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f65357fa640 10 client.17850 handle_command_reply: tid=0 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 client.17850 shutdown 2024-04-22T13:32:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 msgr2=0x55cffee92d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 0x55cffee92d80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f654809bf70 tx=0x7f654809bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 shutdown_connections 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffeea6270 0x55cffeea86e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee95e10 0x55cffee93800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8d290 0x55cffee932c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 --2- 172.21.15.5:0/3135723967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee8c8e0 0x55cffee92d80 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f65357fa640 0 client.17850 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 >> 172.21.15.5:0/3135723967 conn(0x55cffee86fb0 msgr2=0x55cffee88d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 shutdown_connections 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 1 -- 172.21.15.5:0/3135723967 wait complete. 2024-04-22T13:32:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.561+0000 7f655a9c3740 20 client.17850 trim_cache size 0 max 16384 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 Processor -- start 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 -- start start 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffee954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffeea1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee9e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ada70 con 0x55cffee95e10 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65501af5b0 con 0x55cffee8c8e0 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655a9c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ae140 con 0x55cffee8d290 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655636b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee9e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655636b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee9e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36320/0 (socket says 172.21.15.5:36320) 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.579+0000 7f655636b640 1 -- 172.21.15.5:0/237265775 learned_addr learned my addr 172.21.15.5:0/237265775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f6555369640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffeea1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f6555b6a640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffee954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f655636b640 1 -- 172.21.15.5:0/237265775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 msgr2=0x55cffeea1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f655636b640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffeea1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f655636b640 1 -- 172.21.15.5:0/237265775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 msgr2=0x55cffee954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f655636b640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffee954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f655636b640 1 -- 172.21.15.5:0/237265775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cffee8bc00 con 0x55cffee95e10 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f6555369640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffeea1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f6555b6a640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffee954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f655636b640 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee9e4d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f65400129c0 tx=0x7f6540009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3952991dd8e913 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f65377fe640 1 -- 172.21.15.5:0/237265775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540004b20 con 0x55cffee95e10 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f65377fe640 1 -- 172.21.15.5:0/237265775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6540003e20 con 0x55cffee95e10 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.580+0000 7f65377fe640 1 -- 172.21.15.5:0/237265775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540004590 con 0x55cffee95e10 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 -- 172.21.15.5:0/237265775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 msgr2=0x55cffee9e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee9e4d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f65400129c0 tx=0x7f6540009870 comp rx=0 tx=0).stop 2024-04-22T13:32:45.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 -- 172.21.15.5:0/237265775 shutdown_connections 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffee9e4d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffeea1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 --2- 172.21.15.5:0/237265775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffee954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 -- 172.21.15.5:0/237265775 >> 172.21.15.5:0/237265775 conn(0x55cffee86fb0 msgr2=0x55cffee85780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 -- 172.21.15.5:0/237265775 shutdown_connections 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 -- 172.21.15.5:0/237265775 wait complete. 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 -- start start 2024-04-22T13:32:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.581+0000 7f655a9c3740 1 Processor -- start 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffef047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffef04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffef052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65501aeee0 con 0x55cffee95e10 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ae810 con 0x55cffee8c8e0 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65501aba50 con 0x55cffee8d290 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655a9c3740 10 client.0 _ll_register_callbacks cb 0x7f65480046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffef052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffef052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36322/0 (socket says 172.21.15.5:36322) 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 -- 172.21.15.5:0/1784767924 learned_addr learned my addr 172.21.15.5:0/1784767924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f6555b6a640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffef047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f6555369640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffef04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 -- 172.21.15.5:0/1784767924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 msgr2=0x55cffef04d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffef04d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 -- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 msgr2=0x55cffef047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffef047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 -- 172.21.15.5:0/1784767924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cffef0fb50 con 0x55cffee95e10 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f6555b6a640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffef047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f655636b640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffef052a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f654000bf90 tx=0x7f6540003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e85ffc6649fea6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f65357fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.582+0000 7f65357fa640 1 -- 172.21.15.5:0/1784767924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540036060 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f65357fa640 1 -- 172.21.15.5:0/1784767924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f6540026040 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6540003e20 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f65357fa640 1 -- 172.21.15.5:0/1784767924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6540004c50 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 10 client.17856 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cffef10e00 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f65357fa640 1 -- 172.21.15.5:0/1784767924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6540003e20 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f65357fa640 1 -- 172.21.15.5:0/1784767924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65400054d0 con 0x55cffee95e10 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 10 client.17856 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 20 client.17856 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffef114f0 0x55cffef13910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 4 client.17856 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cffedf8790 con 0x55cffef114f0 2024-04-22T13:32:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.583+0000 7f6555b6a640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffef114f0 0x55cffef13910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.584+0000 7f65357fa640 10 client.17856 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.584+0000 7f6555b6a640 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffef114f0 0x55cffef13910 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73b9650f528b626 server_cookie=8c63a5f492bc71b9 in_seq=0 out_seq=0 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f65357fa640 1 -- 172.21.15.5:0/1784767924 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cffedf8790 con 0x55cffef114f0 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f65357fa640 10 client.17856 handle_command_reply: tid=0 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 client.17856 shutdown 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 msgr2=0x55cffef052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffef052a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f654000bf90 tx=0x7f6540003ba0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 shutdown_connections 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55cffef114f0 0x55cffef13910 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cffee95e10 0x55cffef052a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cffee8d290 0x55cffef04d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 --2- 172.21.15.5:0/1784767924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cffee8c8e0 0x55cffef047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f65357fa640 0 client.17856 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 >> 172.21.15.5:0/1784767924 conn(0x55cffee86fb0 msgr2=0x55cffee935d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 shutdown_connections 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 1 -- 172.21.15.5:0/1784767924 wait complete. 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.585+0000 7f655a9c3740 20 client.17856 trim_cache size 0 max 16384 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8e9587c8-0c7c-443f-b37f-d19645ebe72a", 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:32:45.587 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f65380a3d20 msgr2=0x7f65380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f65380a3d20 0x7f65380a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6540009fd0 tx=0x7f65400095f0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 msgr2=0x7f655019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 0x7f655019e410 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f654800d770 tx=0x7f654800dc30 comp rx=0 tx=0).stop 2024-04-22T13:32:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 shutdown_connections 2024-04-22T13:32:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f65380a3d20 0x7f65380a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f655010c500 0x7f65501985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6550103ae0 0x7f655019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 --2- 172.21.15.5:0/831415376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65501031c0 0x7f655019e410 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 >> 172.21.15.5:0/831415376 conn(0x7f6550078000 msgr2=0x7f65500fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 shutdown_connections 2024-04-22T13:32:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.586+0000 7f6557dce640 1 -- 172.21.15.5:0/831415376 wait complete. 2024-04-22T13:32:45.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:45.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 -- 172.21.15.5:0/1402993172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c102180 msgr2=0x7fec2c102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 --2- 172.21.15.5:0/1402993172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c102180 0x7fec2c102560 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fec20009900 tx=0x7fec2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 -- 172.21.15.5:0/1402993172 shutdown_connections 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 --2- 172.21.15.5:0/1402993172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2c107250 0x7fec2c10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 --2- 172.21.15.5:0/1402993172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c102b30 0x7fec2c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 --2- 172.21.15.5:0/1402993172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c102180 0x7fec2c102560 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 -- 172.21.15.5:0/1402993172 >> 172.21.15.5:0/1402993172 conn(0x7fec2c0fda00 msgr2=0x7fec2c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 -- 172.21.15.5:0/1402993172 shutdown_connections 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.701+0000 7fec32f66640 1 -- 172.21.15.5:0/1402993172 wait complete. 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 Processor -- start 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 -- start start 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2c102180 0x7fec2c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 0x7fec2c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c198640 0x7fec2c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c112190 con 0x7fec2c107250 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c112010 con 0x7fec2c198640 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec32f66640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c112310 con 0x7fec2c102180 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 0x7fec2c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 0x7fec2c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36338/0 (socket says 172.21.15.5:36338) 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec2bfff640 1 -- 172.21.15.5:0/976746706 learned_addr learned my addr 172.21.15.5:0/976746706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec30d02640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2c102180 0x7fec2c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.702+0000 7fec31503640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c198640 0x7fec2c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec2bfff640 1 -- 172.21.15.5:0/976746706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2c102180 msgr2=0x7fec2c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec2bfff640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2c102180 0x7fec2c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec2bfff640 1 -- 172.21.15.5:0/976746706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c198640 msgr2=0x7fec2c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec2bfff640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c198640 0x7fec2c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec2bfff640 1 -- 172.21.15.5:0/976746706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c199160 con 0x7fec2c107250 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec2bfff640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 0x7fec2c19e930 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e9c0 tx=0x7fec1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec29ffb640 1 -- 172.21.15.5:0/976746706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800cd80 con 0x7fec2c107250 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec29ffb640 1 -- 172.21.15.5:0/976746706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec180054d0 con 0x7fec2c107250 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c199450 con 0x7fec2c107250 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec29ffb640 1 -- 172.21.15.5:0/976746706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18010430 con 0x7fec2c107250 2024-04-22T13:32:45.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.703+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c1a5650 con 0x7fec2c107250 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.705+0000 7fec29ffb640 1 -- 172.21.15.5:0/976746706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fec18004fd0 con 0x7fec2c107250 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 Processor -- start 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 -- start start 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b3198960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198ea0 0x5630b31a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec29ffb640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fec1c0a3da0 0x7fec1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b31a1a80 0x5630b31a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630b30fd1e0 con 0x5630b3198580 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630b309a390 con 0x5630b31a1a80 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec35c31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630b30f91a0 con 0x5630b3198ea0 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b3198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b3198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36354/0 (socket says 172.21.15.5:36354) 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec30d02640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fec1c0a3da0 0x7fec1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 -- 172.21.15.5:0/2765400045 learned_addr learned my addr 172.21.15.5:0/2765400045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec2bfff640 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198ea0 0x5630b31a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec29ffb640 1 -- 172.21.15.5:0/976746706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fec180d0c30 con 0x7fec2c107250 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b31a1a80 0x5630b31a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 -- 172.21.15.5:0/2765400045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198ea0 msgr2=0x5630b31a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198ea0 0x5630b31a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 -- 172.21.15.5:0/2765400045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b31a1a80 msgr2=0x5630b31a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b31a1a80 0x5630b31a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.706+0000 7fec31503640 1 -- 172.21.15.5:0/2765400045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b2ed97d0 con 0x5630b3198580 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec30d02640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fec1c0a3da0 0x7fec1c0a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fec20006010 tx=0x7fec20005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec31503640 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b3198960 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fec24009370 tx=0x7fec24009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921fc5f77f7f4179 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec16ffd640 1 -- 172.21.15.5:0/2765400045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24019030 con 0x5630b3198580 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec16ffd640 1 -- 172.21.15.5:0/2765400045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec24013e50 con 0x5630b3198580 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec16ffd640 1 -- 172.21.15.5:0/2765400045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2400faa0 con 0x5630b3198580 2024-04-22T13:32:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec35c31740 1 -- 172.21.15.5:0/2765400045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 msgr2=0x5630b3198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.707+0000 7fec35c31740 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b3198960 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fec24009370 tx=0x7fec24009940 comp rx=0 tx=0).stop 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- 172.21.15.5:0/2765400045 shutdown_connections 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b31a1a80 0x5630b31a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198ea0 0x5630b31a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 --2- 172.21.15.5:0/2765400045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b3198960 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- 172.21.15.5:0/2765400045 >> 172.21.15.5:0/2765400045 conn(0x5630b3192cb0 msgr2=0x5630b3193090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- 172.21.15.5:0/2765400045 shutdown_connections 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- 172.21.15.5:0/2765400045 wait complete. 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- start start 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 Processor -- start 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198580 0x5630b319eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 0x5630b319f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630b31a4870 con 0x5630b31a1a80 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630b30b62f0 con 0x5630b3198ea0 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630b31a49f0 con 0x5630b3198580 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.708+0000 7fec35c31740 10 client.0 _ll_register_callbacks cb 0x5630b30c3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 0x5630b319f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 0x5630b319f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36364/0 (socket says 172.21.15.5:36364) 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 -- 172.21.15.5:0/3626511130 learned_addr learned my addr 172.21.15.5:0/3626511130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec30d02640 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198580 0x5630b319eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 -- 172.21.15.5:0/3626511130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198580 msgr2=0x5630b319eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198580 0x5630b319eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 -- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 msgr2=0x5630b319f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 -- 172.21.15.5:0/3626511130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b2e0f3f0 con 0x5630b31a1a80 2024-04-22T13:32:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec31503640 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 0x5630b319f570 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fec2400f6c0 tx=0x7fec24004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4b30a5a5a17426 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec0bfff640 1 -- 172.21.15.5:0/3626511130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24019040 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec24013e50 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec0bfff640 1 -- 172.21.15.5:0/3626511130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fec24011a80 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec0bfff640 1 -- 172.21.15.5:0/3626511130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec24011c80 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec35c31740 10 client.17868 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.709+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630b2dcf0f0 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec0bfff640 1 -- 172.21.15.5:0/3626511130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec24013e50 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec0bfff640 1 -- 172.21.15.5:0/3626511130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fec24025070 con 0x5630b31a1a80 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec35c31740 10 client.17868 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec35c31740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec35c31740 20 client.17868 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec35c31740 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b31b1740 0x5630b31b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec35c31740 4 client.17868 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b310ac00 con 0x5630b31b1740 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.710+0000 7fec30d02640 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b31b1740 0x5630b31b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec30d02640 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b31b1740 0x5630b31b3b30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81abc43e6114c479 server_cookie=42567fbbf05f2957 in_seq=0 out_seq=0 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec0bfff640 10 client.17868 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec0bfff640 1 -- 172.21.15.5:0/3626511130 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630b310ac00 con 0x5630b31b1740 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec0bfff640 10 client.17868 handle_command_reply: tid=0 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec35c31740 1 client.17868 shutdown 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 msgr2=0x5630b319f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.711+0000 7fec35c31740 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 0x5630b319f570 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fec2400f6c0 tx=0x7fec24004cf0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 shutdown_connections 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b31b1740 0x5630b31b3b30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b31a1a80 0x5630b319f570 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec0bfff640 0 client.17868 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 --2- 172.21.15.5:0/3626511130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b3198580 0x5630b319eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 >> 172.21.15.5:0/3626511130 conn(0x5630b3192cb0 msgr2=0x5630b3194a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 shutdown_connections 2024-04-22T13:32:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 1 -- 172.21.15.5:0/3626511130 wait complete. 2024-04-22T13:32:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.712+0000 7fec35c31740 20 client.17868 trim_cache size 0 max 16384 2024-04-22T13:32:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.729+0000 7fec35c31740 1 Processor -- start 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.729+0000 7fec35c31740 1 -- start start 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec35c31740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b31ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec35c31740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec35c31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 0x5630b319f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec35c31740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1adc10 con 0x5630b3198580 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec35c31740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1af750 con 0x5630b3198ea0 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec35c31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1ae2e0 con 0x5630b31a1a80 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b31ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b31ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36370/0 (socket says 172.21.15.5:36370) 2024-04-22T13:32:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 -- 172.21.15.5:0/679277823 learned_addr learned my addr 172.21.15.5:0/679277823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec2bfff640 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 -- 172.21.15.5:0/679277823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 msgr2=0x5630b319f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 0x5630b319f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 -- 172.21.15.5:0/679277823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 msgr2=0x5630b319e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 -- 172.21.15.5:0/679277823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b3198cd0 con 0x5630b3198580 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec2bfff640 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec30d02640 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b31ad730 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fec200096c0 tx=0x7fec2003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f1f5c350b577df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec167fc640 1 -- 172.21.15.5:0/679277823 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20059070 con 0x5630b3198580 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec167fc640 1 -- 172.21.15.5:0/679277823 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec200054f0 con 0x5630b3198580 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.730+0000 7fec167fc640 1 -- 172.21.15.5:0/679277823 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200035f0 con 0x5630b3198580 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 -- 172.21.15.5:0/679277823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 msgr2=0x5630b31ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b31ad730 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fec200096c0 tx=0x7fec2003e000 comp rx=0 tx=0).stop 2024-04-22T13:32:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 -- 172.21.15.5:0/679277823 shutdown_connections 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 0x5630b319f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198ea0 0x5630b319e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 --2- 172.21.15.5:0/679277823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198580 0x5630b31ad730 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 -- 172.21.15.5:0/679277823 >> 172.21.15.5:0/679277823 conn(0x5630b3192cb0 msgr2=0x5630b319eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 -- 172.21.15.5:0/679277823 shutdown_connections 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 -- 172.21.15.5:0/679277823 wait complete. 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 -- start start 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.731+0000 7fec35c31740 1 Processor -- start 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198580 0x5630b320fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 0x5630b3210410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 0x5630b31a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1af080 con 0x5630b3198ea0 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1ae9b0 con 0x5630b3198580 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1abc10 con 0x5630b31a1a80 2024-04-22T13:32:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 10 client.0 _ll_register_callbacks cb 0x7fec180048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 0x5630b3210410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 0x5630b3210410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36374/0 (socket says 172.21.15.5:36374) 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 -- 172.21.15.5:0/2989205773 learned_addr learned my addr 172.21.15.5:0/2989205773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec30d02640 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198580 0x5630b320fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 -- 172.21.15.5:0/2989205773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 msgr2=0x5630b31a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 0x5630b31a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 -- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198580 msgr2=0x5630b320fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198580 0x5630b320fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 -- 172.21.15.5:0/2989205773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b31a5700 con 0x5630b3198ea0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec2bfff640 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 0x5630b3210410 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fec1809cb30 tx=0x7fec1800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824e61b546aa2e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec0bfff640 1 -- 172.21.15.5:0/2989205773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1809d2a0 con 0x5630b3198ea0 2024-04-22T13:32:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec0bfff640 1 -- 172.21.15.5:0/2989205773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fec180ce5d0 con 0x5630b3198ea0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec200054f0 con 0x5630b3198ea0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec0bfff640 1 -- 172.21.15.5:0/2989205773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec180ce7d0 con 0x5630b3198ea0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 10 client.17874 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.732+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630b3217630 con 0x5630b3198ea0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec0bfff640 1 -- 172.21.15.5:0/2989205773 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec200054f0 con 0x5630b3198ea0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec0bfff640 1 -- 172.21.15.5:0/2989205773 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fec180972e0 con 0x5630b3198ea0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec35c31740 10 client.17874 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec35c31740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec35c31740 20 client.17874 populate_metadata read hostname 'smithi005' 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec35c31740 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b3217de0 0x5630b321a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec35c31740 4 client.17874 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630b3104a70 con 0x5630b3217de0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec30d02640 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b3217de0 0x5630b321a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec30d02640 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b3217de0 0x5630b321a1d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535f101e3ebba06f server_cookie=71da8ccc3f3b15a5 in_seq=0 out_seq=0 2024-04-22T13:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.733+0000 7fec0bfff640 10 client.17874 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec0bfff640 1 -- 172.21.15.5:0/2989205773 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630b3104a70 con 0x5630b3217de0 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec0bfff640 10 client.17874 handle_command_reply: tid=0 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 client.17874 shutdown 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 msgr2=0x5630b3210410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 0x5630b3210410 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fec1809cb30 tx=0x7fec1800a440 comp rx=0 tx=0).stop 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 shutdown_connections 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5630b3217de0 0x5630b321a1d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b3198ea0 0x5630b3210410 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630b31a1a80 0x5630b31a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec0bfff640 0 client.17874 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 --2- 172.21.15.5:0/2989205773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5630b3198580 0x5630b320fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 >> 172.21.15.5:0/2989205773 conn(0x5630b3192cb0 msgr2=0x5630b319f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 shutdown_connections 2024-04-22T13:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 1 -- 172.21.15.5:0/2989205773 wait complete. 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec35c31740 20 client.17874 trim_cache size 0 max 16384 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "8e9587c8-0c7c-443f-b37f-d19645ebe72a": { 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8e9587c8-0c7c-443f-b37f-d19645ebe72a", 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:32:45.738 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:32:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fec1c0a3da0 msgr2=0x7fec1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.734+0000 7fec32f66640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fec1c0a3da0 0x7fec1c0a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fec20006010 tx=0x7fec20005e40 comp rx=0 tx=0).stop 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 msgr2=0x7fec2c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 0x7fec2c19e930 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e9c0 tx=0x7fec1800ee80 comp rx=0 tx=0).stop 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 shutdown_connections 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c107250 0x7fec2c19e930 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fec1c0a3da0 0x7fec1c0a6260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c198640 0x7fec2c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 --2- 172.21.15.5:0/976746706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2c102180 0x7fec2c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 >> 172.21.15.5:0/976746706 conn(0x7fec2c0fda00 msgr2=0x7fec2c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 shutdown_connections 2024-04-22T13:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:32:45.735+0000 7fec32f66640 1 -- 172.21.15.5:0/976746706 wait complete. 2024-04-22T13:32:45.741 INFO:tasks.cephfs.filesystem:scrub status for tag:8e9587c8-0c7c-443f-b37f-d19645ebe72a - {'path': '/', 'tag': '8e9587c8-0c7c-443f-b37f-d19645ebe72a', 'options': 'recursive,force'} 2024-04-22T13:32:46.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:46 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:32:46.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:46 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:46 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:32:46.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:46 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:46.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:46 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:32:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:46 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:46 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:32:46.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:46 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:46 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:32:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:46 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:46 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:32:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:46 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:47.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:47 smithi049 ceph-mon[31379]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:47.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:47 smithi107 ceph-mon[31859]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:47.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:47 smithi005 ceph-mon[28006]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:49.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:49 smithi049 ceph-mon[31379]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:49.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:49 smithi107 ceph-mon[31859]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:49.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:49 smithi005 ceph-mon[28006]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:51.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:51 smithi049 ceph-mon[31379]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:51.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:51 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:32:51.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:51 smithi107 ceph-mon[31859]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:51.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:51 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:32:51.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:51 smithi005 ceph-mon[28006]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:51.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:51 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:32:53.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:53 smithi049 ceph-mon[31379]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:53.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:53 smithi107 ceph-mon[31859]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:53.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:53 smithi005 ceph-mon[28006]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:55.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:55 smithi107 ceph-mon[31859]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:55.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:55 smithi005 ceph-mon[28006]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:55.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:55 smithi049 ceph-mon[31379]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:57.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:57 smithi107 ceph-mon[31859]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:57.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:57 smithi005 ceph-mon[28006]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:57.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:57 smithi049 ceph-mon[31379]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:59.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:32:59 smithi107 ceph-mon[31859]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:59.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:32:59 smithi005 ceph-mon[28006]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:59.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:32:59 smithi049 ceph-mon[31379]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:01.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:01 smithi107 ceph-mon[31859]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:01.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:01 smithi005 ceph-mon[28006]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:01.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:01 smithi049 ceph-mon[31379]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:03.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:03 smithi107 ceph-mon[31859]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:03.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:03 smithi005 ceph-mon[28006]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:03.323 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:03 smithi049 ceph-mon[31379]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:05.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:05 smithi005 ceph-mon[28006]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:05.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:05 smithi049 ceph-mon[31379]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:05.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:05 smithi107 ceph-mon[31859]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:07.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:07 smithi005 ceph-mon[28006]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:07.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:07 smithi049 ceph-mon[31379]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:07.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:07 smithi107 ceph-mon[31859]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:09.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:09 smithi005 ceph-mon[28006]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:09.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:09 smithi049 ceph-mon[31379]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:09.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:09 smithi107 ceph-mon[31859]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:11.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:11 smithi005 ceph-mon[28006]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:11.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:11 smithi049 ceph-mon[31379]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:11.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:11 smithi107 ceph-mon[31859]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:13 smithi005 ceph-mon[28006]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:13.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:13 smithi049 ceph-mon[31379]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:13.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:13 smithi107 ceph-mon[31859]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:15.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:15 smithi005 ceph-mon[28006]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:15.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:15 smithi049 ceph-mon[31379]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:15.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:15 smithi107 ceph-mon[31859]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:33:15.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:15.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 -- 172.21.15.5:0/3281948088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694107250 msgr2=0x7f469410ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 --2- 172.21.15.5:0/3281948088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694107250 0x7f469410ee20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f468c009ca0 tx=0x7f468c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 -- 172.21.15.5:0/3281948088 shutdown_connections 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 --2- 172.21.15.5:0/3281948088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694107250 0x7f469410ee20 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 --2- 172.21.15.5:0/3281948088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694102b30 0x7f4694106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 --2- 172.21.15.5:0/3281948088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4694102180 0x7f4694102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.863+0000 7f469ac81640 1 -- 172.21.15.5:0/3281948088 >> 172.21.15.5:0/3281948088 conn(0x7f46940fda00 msgr2=0x7f46940ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 -- 172.21.15.5:0/3281948088 shutdown_connections 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 -- 172.21.15.5:0/3281948088 wait complete. 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 Processor -- start 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 -- start start 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 0x7f46941986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694107250 0x7f4694198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f469419cf60 0x7f469419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46941121a0 con 0x7f4694102180 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4694112020 con 0x7f469419cf60 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f469ac81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4694112320 con 0x7f4694107250 2024-04-22T13:33:15.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.864+0000 7f4698a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 0x7f46941986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 0x7f46941986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35150/0 (socket says 172.21.15.5:35150) 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 -- 172.21.15.5:0/368027183 learned_addr learned my addr 172.21.15.5:0/368027183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f469921e640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f469419cf60 0x7f469419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4693fff640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694107250 0x7f4694198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 -- 172.21.15.5:0/368027183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694107250 msgr2=0x7f4694198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694107250 0x7f4694198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 -- 172.21.15.5:0/368027183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f469419cf60 msgr2=0x7f469419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f469419cf60 0x7f469419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 -- 172.21.15.5:0/368027183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469419da50 con 0x7f4694102180 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4693fff640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694107250 0x7f4694198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:15.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4698a1d640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 0x7f46941986a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f468800e9f0 tx=0x7f468800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:15.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4691ffb640 1 -- 172.21.15.5:0/368027183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468800cd80 con 0x7f4694102180 2024-04-22T13:33:15.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4691ffb640 1 -- 172.21.15.5:0/368027183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46880108e0 con 0x7f4694102180 2024-04-22T13:33:15.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469419dd40 con 0x7f4694102180 2024-04-22T13:33:15.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f4691ffb640 1 -- 172.21.15.5:0/368027183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688018980 con 0x7f4694102180 2024-04-22T13:33:15.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.865+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a5580 con 0x7f4694102180 2024-04-22T13:33:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f4691ffb640 1 -- 172.21.15.5:0/368027183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f4688010430 con 0x7f4694102180 2024-04-22T13:33:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 Processor -- start 2024-04-22T13:33:15.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 -- start start 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533588960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x556533591540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x556533593e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565334ed1e0 con 0x556533588580 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55653348a390 con 0x556533591a80 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469d876740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5565334e91a0 con 0x556533588ea0 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f4691ffb640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46840a3da0 0x7f46840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469921e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533588960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469921e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533588960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35162/0 (socket says 172.21.15.5:35162) 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.868+0000 7f469921e640 1 -- 172.21.15.5:0/2550878012 learned_addr learned my addr 172.21.15.5:0/2550878012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:15.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f4693fff640 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x556533593e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46840a3da0 0x7f46840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f4691ffb640 1 -- 172.21.15.5:0/368027183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f46880d0b50 con 0x7f4694102180 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f4698a1d640 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x556533591540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 -- 172.21.15.5:0/2550878012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 msgr2=0x556533591540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x556533591540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 -- 172.21.15.5:0/2550878012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 msgr2=0x556533593e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x556533593e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 -- 172.21.15.5:0/2550878012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565332c97d0 con 0x556533588580 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f4693fff640 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x556533593e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533588960 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f468c037040 tx=0x7f468c0324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e93ddc5613c8962 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f469921e640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46840a3da0 0x7f46840a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f468c009ca0 tx=0x7f468c048000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:15.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f467effd640 1 -- 172.21.15.5:0/2550878012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c051070 con 0x556533588580 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f467effd640 1 -- 172.21.15.5:0/2550878012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f468c032830 con 0x556533588580 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.869+0000 7f467effd640 1 -- 172.21.15.5:0/2550878012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c039040 con 0x556533588580 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 -- 172.21.15.5:0/2550878012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 msgr2=0x556533588960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533588960 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f468c037040 tx=0x7f468c0324c0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 -- 172.21.15.5:0/2550878012 shutdown_connections 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x556533593e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x556533591540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 --2- 172.21.15.5:0/2550878012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533588960 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 -- 172.21.15.5:0/2550878012 >> 172.21.15.5:0/2550878012 conn(0x556533582cb0 msgr2=0x556533583090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 -- 172.21.15.5:0/2550878012 shutdown_connections 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 -- 172.21.15.5:0/2550878012 wait complete. 2024-04-22T13:33:15.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.870+0000 7f469d876740 1 -- start start 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 Processor -- start 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x55653358eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 0x55653358f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533591a80 0x55653358f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556533594810 con 0x556533588ea0 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5565334a62f0 con 0x556533588580 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556533594990 con 0x556533591a80 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469d876740 10 client.0 _ll_register_callbacks cb 0x5565334b3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 0x55653358f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 0x55653358f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35178/0 (socket says 172.21.15.5:35178) 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4693fff640 1 -- 172.21.15.5:0/3872784902 learned_addr learned my addr 172.21.15.5:0/3872784902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f469921e640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533591a80 0x55653358f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4698a1d640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x55653358eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4693fff640 1 -- 172.21.15.5:0/3872784902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533591a80 msgr2=0x55653358f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4693fff640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533591a80 0x55653358f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.871+0000 7f4693fff640 1 -- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 msgr2=0x55653358eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4693fff640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x55653358eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4693fff640 1 -- 172.21.15.5:0/3872784902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565331ff3f0 con 0x556533588ea0 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f469921e640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533591a80 0x55653358f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:15.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4693fff640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 0x55653358f0a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4680009e30 tx=0x7f46800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a5261d7840d62e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4673fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4673fff640 1 -- 172.21.15.5:0/3872784902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680012860 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4673fff640 1 -- 172.21.15.5:0/3872784902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f4680012a00 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c032830 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4673fff640 1 -- 172.21.15.5:0/3872784902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4680012c40 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f469d876740 10 client.17886 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565331bf0f0 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4673fff640 1 -- 172.21.15.5:0/3872784902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f468c032830 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.872+0000 7f4673fff640 1 -- 172.21.15.5:0/3872784902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f468000f040 con 0x556533588ea0 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f469d876740 10 client.17886 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f469d876740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f469d876740 20 client.17886 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f469d876740 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5565335a1c20 0x5565335a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f469d876740 4 client.17886 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565334fac00 con 0x5565335a1c20 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f4698a1d640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5565335a1c20 0x5565335a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f4698a1d640 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5565335a1c20 0x5565335a4060 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9541736d22fe8926 server_cookie=f6dcb93f4e9cda36 in_seq=0 out_seq=0 2024-04-22T13:33:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.873+0000 7f4673fff640 10 client.17886 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f4673fff640 1 -- 172.21.15.5:0/3872784902 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565334fac00 con 0x5565335a1c20 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f4673fff640 10 client.17886 handle_command_reply: tid=0 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 client.17886 shutdown 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 msgr2=0x55653358f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 0x55653358f0a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4680009e30 tx=0x7f46800117b0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 shutdown_connections 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5565335a1c20 0x5565335a4060 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588ea0 0x55653358f0a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533591a80 0x55653358f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 --2- 172.21.15.5:0/3872784902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x55653358eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f4673fff640 0 client.17886 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 >> 172.21.15.5:0/3872784902 conn(0x556533582cb0 msgr2=0x556533584a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 shutdown_connections 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 1 -- 172.21.15.5:0/3872784902 wait complete. 2024-04-22T13:33:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.874+0000 7f469d876740 20 client.17886 trim_cache size 0 max 16384 2024-04-22T13:33:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 Processor -- start 2024-04-22T13:33:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 -- start start 2024-04-22T13:33:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x556533591290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x55653359d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 0x556533599fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46941adb80 con 0x556533591a80 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46941af6c0 con 0x556533588580 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469d876740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46941ae250 con 0x556533588ea0 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469921e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 0x556533599fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469921e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 0x556533599fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35192/0 (socket says 172.21.15.5:35192) 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f469921e640 1 -- 172.21.15.5:0/3533359844 learned_addr learned my addr 172.21.15.5:0/3533359844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.892+0000 7f4693fff640 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x55653359d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469921e640 1 -- 172.21.15.5:0/3533359844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 msgr2=0x55653359d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469921e640 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x55653359d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469921e640 1 -- 172.21.15.5:0/3533359844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 msgr2=0x556533591290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469921e640 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x556533591290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469921e640 1 -- 172.21.15.5:0/3533359844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565335a0720 con 0x556533591a80 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469921e640 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 0x556533599fc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f468c037040 tx=0x7f468c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b519971871b9fd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f467e7fc640 1 -- 172.21.15.5:0/3533359844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c05a070 con 0x556533591a80 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f467e7fc640 1 -- 172.21.15.5:0/3533359844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f468c032830 con 0x556533591a80 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f467e7fc640 1 -- 172.21.15.5:0/3533359844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c051040 con 0x556533591a80 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469d876740 1 -- 172.21.15.5:0/3533359844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 msgr2=0x556533599fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469d876740 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 0x556533599fc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f468c037040 tx=0x7f468c02c8b0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469d876740 1 -- 172.21.15.5:0/3533359844 shutdown_connections 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469d876740 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533591a80 0x556533599fc0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469d876740 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x55653359d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.893+0000 7f469d876740 1 --2- 172.21.15.5:0/3533359844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533588580 0x556533591290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- 172.21.15.5:0/3533359844 >> 172.21.15.5:0/3533359844 conn(0x556533582cb0 msgr2=0x5565335814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- 172.21.15.5:0/3533359844 shutdown_connections 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- 172.21.15.5:0/3533359844 wait complete. 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- start start 2024-04-22T13:33:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 Processor -- start 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533600160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x5565336006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x55653360b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46941aeff0 con 0x556533588580 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46941ae920 con 0x556533591a80 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46941abb60 con 0x556533588ea0 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f469d876740 10 client.0 _ll_register_callbacks cb 0x7f4680009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f4698a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533600160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f4698a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533600160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35196/0 (socket says 172.21.15.5:35196) 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.894+0000 7f4698a1d640 1 -- 172.21.15.5:0/2340319900 learned_addr learned my addr 172.21.15.5:0/2340319900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4693fff640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x5565336006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f469921e640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x55653360b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4698a1d640 1 -- 172.21.15.5:0/2340319900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 msgr2=0x5565336006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4698a1d640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x5565336006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4698a1d640 1 -- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 msgr2=0x55653360b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4698a1d640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x55653360b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4698a1d640 1 -- 172.21.15.5:0/2340319900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556533600f90 con 0x556533588580 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4698a1d640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533600160 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f46880ce8d0 tx=0x7f4688000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac14c4d65ee83a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4673fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4673fff640 1 -- 172.21.15.5:0/2340319900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46880b6060 con 0x556533588580 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4673fff640 1 -- 172.21.15.5:0/2340319900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f4688010a90 con 0x556533588580 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c032830 con 0x556533588580 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4673fff640 1 -- 172.21.15.5:0/2340319900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46880a5790 con 0x556533588580 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f469d876740 10 client.17892 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55653360c890 con 0x556533588580 2024-04-22T13:33:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.895+0000 7f4673fff640 1 -- 172.21.15.5:0/2340319900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f468c032830 con 0x556533588580 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f4673fff640 1 -- 172.21.15.5:0/2340319900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f46880bbce0 con 0x556533588580 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f469d876740 10 client.17892 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f469d876740 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f469d876740 20 client.17892 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f469d876740 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55653360cf10 0x55653360f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f469d876740 4 client.17892 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565334f4a70 con 0x55653360cf10 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.896+0000 7f4693fff640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55653360cf10 0x55653360f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f4693fff640 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55653360cf10 0x55653360f330 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f972e361e86509 server_cookie=f2408cd98ebfcdff in_seq=0 out_seq=0 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f4673fff640 10 client.17892 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f4673fff640 1 -- 172.21.15.5:0/2340319900 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5565334f4a70 con 0x55653360cf10 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f4673fff640 10 client.17892 handle_command_reply: tid=0 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 client.17892 shutdown 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 msgr2=0x556533600160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533600160 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f46880ce8d0 tx=0x7f4688000bc0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 shutdown_connections 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55653360cf10 0x55653360f330 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556533591a80 0x55653360b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f4673fff640 0 client.17892 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556533588ea0 0x5565336006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 --2- 172.21.15.5:0/2340319900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556533588580 0x556533600160 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 >> 172.21.15.5:0/2340319900 conn(0x556533582cb0 msgr2=0x55653358f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 shutdown_connections 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 1 -- 172.21.15.5:0/2340319900 wait complete. 2024-04-22T13:33:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.897+0000 7f469d876740 20 client.17892 trim_cache size 0 max 16384 2024-04-22T13:33:15.903 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:33:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:33:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:33:15.903 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46840a3da0 msgr2=0x7f46840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46840a3da0 0x7f46840a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f468c009ca0 tx=0x7f468c048000 comp rx=0 tx=0).stop 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 msgr2=0x7f46941986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 0x7f46941986a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f468800e9f0 tx=0x7f468800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 shutdown_connections 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46840a3da0 0x7f46840a6260 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f469419cf60 0x7f469419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4694107250 0x7f4694198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 --2- 172.21.15.5:0/368027183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4694102180 0x7f46941986a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 >> 172.21.15.5:0/368027183 conn(0x7f46940fda00 msgr2=0x7f46940ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 shutdown_connections 2024-04-22T13:33:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:15.898+0000 7f469ac81640 1 -- 172.21.15.5:0/368027183 wait complete. 2024-04-22T13:33:15.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:33:15.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 -- 172.21.15.5:0/2952545891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c101070 msgr2=0x7fd44c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 --2- 172.21.15.5:0/2952545891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c101070 0x7fd44c101450 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd448009900 tx=0x7fd44802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 -- 172.21.15.5:0/2952545891 shutdown_connections 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 --2- 172.21.15.5:0/2952545891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 0x7fd44c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 --2- 172.21.15.5:0/2952545891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101a20 0x7fd44c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 --2- 172.21.15.5:0/2952545891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c101070 0x7fd44c101450 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 -- 172.21.15.5:0/2952545891 >> 172.21.15.5:0/2952545891 conn(0x7fd44c0fc910 msgr2=0x7fd44c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 -- 172.21.15.5:0/2952545891 shutdown_connections 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.012+0000 7fd453ee4640 1 -- 172.21.15.5:0/2952545891 wait complete. 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 Processor -- start 2024-04-22T13:33:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 -- start start 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 0x7fd44c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 0x7fd44c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 0x7fd44c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1121c0 con 0x7fd44c19cfa0 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c112040 con 0x7fd44c106140 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd453ee4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c112340 con 0x7fd44c101070 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd452481640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 0x7fd44c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 0x7fd44c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 0x7fd44c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:58700/0 (socket says 172.21.15.5:58700) 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 -- 172.21.15.5:0/4194074038 learned_addr learned my addr 172.21.15.5:0/4194074038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd452481640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 0x7fd44c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35202/0 (socket says 172.21.15.5:35202) 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd45147f640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 0x7fd44c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 -- 172.21.15.5:0/4194074038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 msgr2=0x7fd44c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 0x7fd44c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 -- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 msgr2=0x7fd44c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 0x7fd44c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd451c80640 1 -- 172.21.15.5:0/4194074038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c19da90 con 0x7fd44c101070 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd45147f640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 0x7fd44c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.013+0000 7fd452481640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 0x7fd44c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.014+0000 7fd451c80640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 0x7fd44c1986e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd448005940 tx=0x7fd44802c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.014+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448043020 con 0x7fd44c101070 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.014+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c19dd20 con 0x7fd44c101070 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.014+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1a5570 con 0x7fd44c101070 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.014+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44802cbe0 con 0x7fd44c101070 2024-04-22T13:33:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.014+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448003c20 con 0x7fd44c101070 2024-04-22T13:33:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.015+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd430005290 con 0x7fd44c101070 2024-04-22T13:33:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.016+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd448046020 con 0x7fd44c101070 2024-04-22T13:33:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.017+0000 7fd442ffd640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4380a3d20 0x7fd4380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.017+0000 7fd45147f640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4380a3d20 0x7fd4380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.017+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fd4480f26d0 con 0x7fd44c101070 2024-04-22T13:33:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.017+0000 7fd45147f640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4380a3d20 0x7fd4380a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0fcea0 tx=0x7fd43c005e50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.021+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd44804d030 con 0x7fd44c101070 2024-04-22T13:33:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.248+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd430005580 con 0x7fd44c101070 2024-04-22T13:33:16.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.249+0000 7fd442ffd640 1 -- 172.21.15.5:0/4194074038 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7fd4480beeb0 con 0x7fd44c101070 2024-04-22T13:33:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:33:16.246 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:33:16.246 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:33:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4380a3d20 msgr2=0x7fd4380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4380a3d20 0x7fd4380a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0fcea0 tx=0x7fd43c005e50 comp rx=0 tx=0).stop 2024-04-22T13:33:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 msgr2=0x7fd44c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 0x7fd44c1986e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd448005940 tx=0x7fd44802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 shutdown_connections 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd44c19cfa0 0x7fd44c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4380a3d20 0x7fd4380a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd44c106140 0x7fd44c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 --2- 172.21.15.5:0/4194074038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c101070 0x7fd44c1986e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.251+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 >> 172.21.15.5:0/4194074038 conn(0x7fd44c0fc910 msgr2=0x7fd44c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.252+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 shutdown_connections 2024-04-22T13:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.252+0000 7fd453ee4640 1 -- 172.21.15.5:0/4194074038 wait complete. 2024-04-22T13:33:16.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:33:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 -- 172.21.15.5:0/912615853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c106160 msgr2=0x7f5d9c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 --2- 172.21.15.5:0/912615853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c106160 0x7f5d9c10dd30 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009ca0 tx=0x7f5d9402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 -- 172.21.15.5:0/912615853 shutdown_connections 2024-04-22T13:33:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 --2- 172.21.15.5:0/912615853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c106160 0x7f5d9c10dd30 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 --2- 172.21.15.5:0/912615853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c101a40 0x7f5d9c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 --2- 172.21.15.5:0/912615853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 0x7f5d9c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 -- 172.21.15.5:0/912615853 >> 172.21.15.5:0/912615853 conn(0x7f5d9c078000 msgr2=0x7f5d9c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 -- 172.21.15.5:0/912615853 shutdown_connections 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 -- 172.21.15.5:0/912615853 wait complete. 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 Processor -- start 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.367+0000 7f5da37f4640 1 -- start start 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 0x7f5d9c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c106160 0x7f5d9c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 0x7f5d9c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1121e0 con 0x7f5d9c19cf20 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c112060 con 0x7f5d9c101090 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c112360 con 0x7f5d9c106160 2024-04-22T13:33:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 0x7f5d9c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 0x7f5d9c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35230/0 (socket says 172.21.15.5:35230) 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 -- 172.21.15.5:0/182954287 learned_addr learned my addr 172.21.15.5:0/182954287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c106160 0x7f5d9c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1590640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 0x7f5d9c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 -- 172.21.15.5:0/182954287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c106160 msgr2=0x7f5d9c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c106160 0x7f5d9c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 -- 172.21.15.5:0/182954287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 msgr2=0x7f5d9c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 0x7f5d9c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 -- 172.21.15.5:0/182954287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c19da10 con 0x7f5d9c19cf20 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1590640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 0x7f5d9c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da1d91640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 0x7f5d9c19d3d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005e30 tx=0x7f5d940346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5d927fc640 1 -- 172.21.15.5:0/182954287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94041020 con 0x7f5d9c19cf20 2024-04-22T13:33:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5d927fc640 1 -- 172.21.15.5:0/182954287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d94032970 con 0x7f5d9c19cf20 2024-04-22T13:33:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c19dca0 con 0x7f5d9c19cf20 2024-04-22T13:33:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.368+0000 7f5d927fc640 1 -- 172.21.15.5:0/182954287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94039040 con 0x7f5d9c19cf20 2024-04-22T13:33:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.369+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a5460 con 0x7f5d9c19cf20 2024-04-22T13:33:16.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5d927fc640 1 -- 172.21.15.5:0/182954287 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5d94044020 con 0x7f5d9c19cf20 2024-04-22T13:33:16.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 Processor -- start 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 -- start start 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5d927fc640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d880a3d20 0x7f5d880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6330 0x55aae55c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 0x55aae55cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aae552b1f0 con 0x55aae55c6ce0 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aae54c84b0 con 0x55aae55cfe40 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.371+0000 7f5da63e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55aae55271b0 con 0x55aae55c6330 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5d927fc640 1 -- 172.21.15.5:0/182954287 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5d940f2350 con 0x7f5d9c19cf20 2024-04-22T13:33:16.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d880a3d20 0x7f5d880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da0d8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 0x55aae55cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da0d8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 0x55aae55cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35238/0 (socket says 172.21.15.5:35238) 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:44396/0 (socket says 172.21.15.5:44396) 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da0d8f640 1 -- 172.21.15.5:0/718944141 learned_addr learned my addr 172.21.15.5:0/718944141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1590640 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6330 0x55aae55c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 -- 172.21.15.5:0/718944141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6330 msgr2=0x55aae55c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6330 0x55aae55c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 -- 172.21.15.5:0/718944141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 msgr2=0x55aae55cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 0x55aae55cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 -- 172.21.15.5:0/718944141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae54ba1b0 con 0x55aae55cfe40 2024-04-22T13:33:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 0x55aae55cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d880a3d20 0x7f5d880a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0118f0 tx=0x7f5d8c009b20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5da1d91640 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d02c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d940efbe0 tx=0x7f5d94009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12450b5b9cc56f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.372+0000 7f5d877fe640 1 -- 172.21.15.5:0/718944141 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940bdd90 con 0x55aae55cfe40 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5d877fe640 1 -- 172.21.15.5:0/718944141 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d9402de50 con 0x55aae55cfe40 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5d877fe640 1 -- 172.21.15.5:0/718944141 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940c70a0 con 0x55aae55cfe40 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 -- 172.21.15.5:0/718944141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 msgr2=0x55aae55d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d02c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d940efbe0 tx=0x7f5d94009a20 comp rx=0 tx=0).stop 2024-04-22T13:33:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 -- 172.21.15.5:0/718944141 shutdown_connections 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6ce0 0x55aae55cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d02c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 --2- 172.21.15.5:0/718944141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6330 0x55aae55c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 -- 172.21.15.5:0/718944141 >> 172.21.15.5:0/718944141 conn(0x55aae55c0ba0 msgr2=0x55aae55c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 -- 172.21.15.5:0/718944141 shutdown_connections 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.373+0000 7f5da63e8740 1 -- 172.21.15.5:0/718944141 wait complete. 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 -- start start 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 Processor -- start 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aae55d6b80 con 0x55aae55c6330 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55aae54e4500 con 0x55aae55cfe40 2024-04-22T13:33:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55aae55d6d00 con 0x55aae55c6ce0 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da1590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da63e8740 10 client.0 _ll_register_callbacks cb 0x55aae54b2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da1590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35240/0 (socket says 172.21.15.5:35240) 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da1590640 1 -- 172.21.15.5:0/3698099745 learned_addr learned my addr 172.21.15.5:0/3698099745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.374+0000 7f5da1d91640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da1590640 1 -- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 msgr2=0x55aae55d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da1590640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da1590640 1 -- 172.21.15.5:0/3698099745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 msgr2=0x55aae55d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da1590640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da1590640 1 -- 172.21.15.5:0/3698099745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae523d3f0 con 0x55aae55c6330 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da1590640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d6130 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5d98011a30 tx=0x7f5d98011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b981815964a2fbd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5d84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5d84ff9640 1 -- 172.21.15.5:0/3698099745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98004460 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5d84ff9640 1 -- 172.21.15.5:0/3698099745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5d9800f920 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9402de50 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5d84ff9640 1 -- 172.21.15.5:0/3698099745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d9800fb20 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da63e8740 10 client.17910 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aae53077d0 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5d84ff9640 1 -- 172.21.15.5:0/3698099745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d9402de50 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5d84ff9640 1 -- 172.21.15.5:0/3698099745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d98018070 con 0x55aae55c6330 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da63e8740 10 client.17910 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.375+0000 7f5da63e8740 10 client.17910 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:33:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5da63e8740 20 client.17910 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5da63e8740 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae55e3d50 0x55aae55e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5da63e8740 4 client.17910 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae54ca930 con 0x55aae55e3d50 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae55e3d50 0x55aae55e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae55e3d50 0x55aae55e61d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67bde641a9b9c617 server_cookie=49072d1548261893 in_seq=0 out_seq=0 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.376+0000 7f5d84ff9640 10 client.17910 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5d84ff9640 1 -- 172.21.15.5:0/3698099745 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aae54ca930 con 0x55aae55e3d50 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5d84ff9640 10 client.17910 handle_command_reply: tid=0 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 client.17910 shutdown 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 msgr2=0x55aae55d6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d6130 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5d98011a30 tx=0x7f5d98011ef0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 shutdown_connections 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae55e3d50 0x55aae55e61d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 --2- 172.21.15.5:0/3698099745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d6130 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5d84ff9640 0 client.17910 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 >> 172.21.15.5:0/3698099745 conn(0x55aae55c0ba0 msgr2=0x55aae55c28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 shutdown_connections 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 1 -- 172.21.15.5:0/3698099745 wait complete. 2024-04-22T13:33:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.377+0000 7f5da63e8740 20 client.17910 trim_cache size 0 max 16384 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 Processor -- start 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 -- start start 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1ab840 con 0x55aae55c6330 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1ad380 con 0x55aae55cfe40 2024-04-22T13:33:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da63e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1abf10 con 0x55aae55c6ce0 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35248/0 (socket says 172.21.15.5:35248) 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 -- 172.21.15.5:0/135891163 learned_addr learned my addr 172.21.15.5:0/135891163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 -- 172.21.15.5:0/135891163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 msgr2=0x55aae55d6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 -- 172.21.15.5:0/135891163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 msgr2=0x55aae55ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 -- 172.21.15.5:0/135891163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae55cd7f0 con 0x55aae55c6330 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5da1590640 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d5bf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5d980195b0 tx=0x7f5d9800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2025f37c36917a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.391+0000 7f5d86ffd640 1 -- 172.21.15.5:0/135891163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802f070 con 0x55aae55c6330 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5d86ffd640 1 -- 172.21.15.5:0/135891163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d980197a0 con 0x55aae55c6330 2024-04-22T13:33:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5d86ffd640 1 -- 172.21.15.5:0/135891163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98026070 con 0x55aae55c6330 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 -- 172.21.15.5:0/135891163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 msgr2=0x55aae55d5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d5bf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5d980195b0 tx=0x7f5d9800fab0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 -- 172.21.15.5:0/135891163 shutdown_connections 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 --2- 172.21.15.5:0/135891163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55d5bf0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 -- 172.21.15.5:0/135891163 >> 172.21.15.5:0/135891163 conn(0x55aae55c0ba0 msgr2=0x55aae55bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 -- 172.21.15.5:0/135891163 shutdown_connections 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 -- 172.21.15.5:0/135891163 wait complete. 2024-04-22T13:33:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 -- start start 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.392+0000 7f5da63e8740 1 Processor -- start 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1accb0 con 0x55aae55c6330 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1ac5e0 con 0x55aae55cfe40 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1a9840 con 0x55aae55c6ce0 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da63e8740 10 client.0 _ll_register_callbacks cb 0x55aae54b2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55dc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35252/0 (socket says 172.21.15.5:35252) 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 -- 172.21.15.5:0/704390700 learned_addr learned my addr 172.21.15.5:0/704390700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1d91640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 -- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 msgr2=0x55aae55d50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 -- 172.21.15.5:0/704390700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 msgr2=0x55aae55d57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 -- 172.21.15.5:0/704390700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae5649310 con 0x55aae55c6330 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1d91640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5da1590640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55dc450 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5d98011f30 tx=0x7f5d98005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddb010132d47c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.393+0000 7f5d84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5d84ff9640 1 -- 172.21.15.5:0/704390700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98026070 con 0x55aae55c6330 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5d84ff9640 1 -- 172.21.15.5:0/704390700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5d98019b60 con 0x55aae55c6330 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d980197a0 con 0x55aae55c6330 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5d84ff9640 1 -- 172.21.15.5:0/704390700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d98042050 con 0x55aae55c6330 2024-04-22T13:33:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 10 client.17916 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aae564a480 con 0x55aae55c6330 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5d84ff9640 1 -- 172.21.15.5:0/704390700 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d980197a0 con 0x55aae55c6330 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5d84ff9640 1 -- 172.21.15.5:0/704390700 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d98018030 con 0x55aae55c6330 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 10 client.17916 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 10 client.17916 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 20 client.17916 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae564ab10 0x55aae564cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 4 client.17916 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aae5538c10 con 0x55aae564ab10 2024-04-22T13:33:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.394+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae564ab10 0x55aae564cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5d84ff9640 10 client.17916 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da0d8f640 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae564ab10 0x55aae564cf30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ad800290a86cd1a server_cookie=bffd92fe418a5d6 in_seq=0 out_seq=0 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5d84ff9640 1 -- 172.21.15.5:0/704390700 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aae5538c10 con 0x55aae564ab10 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5d84ff9640 10 client.17916 handle_command_reply: tid=0 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da63e8740 1 client.17916 shutdown 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 msgr2=0x55aae55dc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da63e8740 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55dc450 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5d98011f30 tx=0x7f5d98005590 comp rx=0 tx=0).stop 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 shutdown_connections 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da63e8740 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aae55cfe40 0x55aae55d57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.395+0000 7f5da63e8740 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aae55c6ce0 0x55aae55d50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da63e8740 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55aae564ab10 0x55aae564cf30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da63e8740 1 --2- 172.21.15.5:0/704390700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aae55c6330 0x55aae55dc450 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 >> 172.21.15.5:0/704390700 conn(0x55aae55c0ba0 msgr2=0x55aae55cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5d84ff9640 0 client.17916 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 shutdown_connections 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da63e8740 1 -- 172.21.15.5:0/704390700 wait complete. 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da63e8740 20 client.17916 trim_cache size 0 max 16384 2024-04-22T13:33:16.401 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d880a3d20 msgr2=0x7f5d880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d880a3d20 0x7f5d880a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0118f0 tx=0x7f5d8c009b20 comp rx=0 tx=0).stop 2024-04-22T13:33:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 msgr2=0x7f5d9c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 0x7f5d9c19d3d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005e30 tx=0x7f5d940346d0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 shutdown_connections 2024-04-22T13:33:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d9c19cf20 0x7f5d9c19d3d0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d880a3d20 0x7f5d880a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c106160 0x7f5d9c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 --2- 172.21.15.5:0/182954287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9c101090 0x7f5d9c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 >> 172.21.15.5:0/182954287 conn(0x7f5d9c078000 msgr2=0x7f5d9c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 shutdown_connections 2024-04-22T13:33:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.396+0000 7f5da37f4640 1 -- 172.21.15.5:0/182954287 wait complete. 2024-04-22T13:33:16.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:33:16.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 -- 172.21.15.5:0/906517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 msgr2=0x7f3c0c102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 --2- 172.21.15.5:0/906517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c102560 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009900 tx=0x7f3c0802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 -- 172.21.15.5:0/906517840 shutdown_connections 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 --2- 172.21.15.5:0/906517840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c107250 0x7f3c0c10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 --2- 172.21.15.5:0/906517840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c0c102b30 0x7f3c0c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 --2- 172.21.15.5:0/906517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c102560 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.511+0000 7f3c136c8640 1 -- 172.21.15.5:0/906517840 >> 172.21.15.5:0/906517840 conn(0x7f3c0c0fda00 msgr2=0x7f3c0c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 -- 172.21.15.5:0/906517840 shutdown_connections 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 -- 172.21.15.5:0/906517840 wait complete. 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 Processor -- start 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 -- start start 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c0c107250 0x7f3c0c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c198590 0x7f3c0c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1121b0 con 0x7f3c0c102180 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c112030 con 0x7f3c0c198590 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c136c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c112330 con 0x7f3c0c107250 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c11464640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c11464640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35278/0 (socket says 172.21.15.5:35278) 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.512+0000 7f3c11464640 1 -- 172.21.15.5:0/1601757364 learned_addr learned my addr 172.21.15.5:0/1601757364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c10c63640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c0c107250 0x7f3c0c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c198590 0x7f3c0c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11464640 1 -- 172.21.15.5:0/1601757364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c0c107250 msgr2=0x7f3c0c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11464640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c0c107250 0x7f3c0c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11464640 1 -- 172.21.15.5:0/1601757364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c198590 msgr2=0x7f3c0c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11464640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c198590 0x7f3c0c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11464640 1 -- 172.21.15.5:0/1601757364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c1990b0 con 0x7f3c0c102180 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c198590 0x7f3c0c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c11464640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c19e3e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005940 tx=0x7f3c0802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c027fc640 1 -- 172.21.15.5:0/1601757364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08043020 con 0x7f3c0c102180 2024-04-22T13:33:16.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c199340 con 0x7f3c0c102180 2024-04-22T13:33:16.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c027fc640 1 -- 172.21.15.5:0/1601757364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0802cbe0 con 0x7f3c0c102180 2024-04-22T13:33:16.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c027fc640 1 -- 172.21.15.5:0/1601757364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08003bf0 con 0x7f3c0c102180 2024-04-22T13:33:16.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.513+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1a55e0 con 0x7f3c0c102180 2024-04-22T13:33:16.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c027fc640 1 -- 172.21.15.5:0/1601757364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3c08046020 con 0x7f3c0c102180 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 Processor -- start 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 -- start start 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19af4d0 0x5569c19af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 0x5569c19b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569c19141f0 con 0x5569c19afe80 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5569c18b14b0 con 0x5569c19af4d0 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c16397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569c19101b0 con 0x5569c19b8a00 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c11464640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 0x5569c19b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c11464640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 0x5569c19b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35282/0 (socket says 172.21.15.5:35282) 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c11464640 1 -- 172.21.15.5:0/895891742 learned_addr learned my addr 172.21.15.5:0/895891742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.516+0000 7f3c027fc640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11c65640 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19af4d0 0x5569c19af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c10c63640 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c027fc640 1 -- 172.21.15.5:0/1601757364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3c080f2940 con 0x7f3c0c102180 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11464640 1 -- 172.21.15.5:0/895891742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 msgr2=0x5569c19bae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11464640 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11464640 1 -- 172.21.15.5:0/895891742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19af4d0 msgr2=0x5569c19af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11464640 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19af4d0 0x5569c19af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11464640 1 -- 172.21.15.5:0/895891742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c18a31b0 con 0x5569c19afe80 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c10c63640 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11464640 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 0x5569c19b8430 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3c080f0170 tx=0x7f3c0802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77206adc0636f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3bf77fe640 1 -- 172.21.15.5:0/895891742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080be380 con 0x5569c19afe80 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3bf77fe640 1 -- 172.21.15.5:0/895891742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c080bf120 con 0x5569c19afe80 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3bf77fe640 1 -- 172.21.15.5:0/895891742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080054d0 con 0x5569c19afe80 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.517+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3c04005c40 tx=0x7f3c04009200 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 -- 172.21.15.5:0/895891742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 msgr2=0x5569c19b8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 0x5569c19b8430 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3c080f0170 tx=0x7f3c0802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 -- 172.21.15.5:0/895891742 shutdown_connections 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19afe80 0x5569c19b8430 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 --2- 172.21.15.5:0/895891742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19af4d0 0x5569c19af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 -- 172.21.15.5:0/895891742 >> 172.21.15.5:0/895891742 conn(0x5569c19a9ba0 msgr2=0x5569c19a9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 -- 172.21.15.5:0/895891742 shutdown_connections 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 -- 172.21.15.5:0/895891742 wait complete. 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 -- start start 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.518+0000 7f3c16397740 1 Processor -- start 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569c19141f0 con 0x5569c19b8a00 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5569c18cd500 con 0x5569c19afe80 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569c19bb730 con 0x5569c19af4d0 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 10 client.0 _ll_register_callbacks cb 0x5569c189bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35290/0 (socket says 172.21.15.5:35290) 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 -- 172.21.15.5:0/3409673551 learned_addr learned my addr 172.21.15.5:0/3409673551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c10c63640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11464640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 -- 172.21.15.5:0/3409673551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 msgr2=0x5569c19b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19b8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 -- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 msgr2=0x5569c19b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 -- 172.21.15.5:0/3409673551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c16263f0 con 0x5569c19b8a00 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c11c65640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b5ce0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400f980 tx=0x7f3c04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d1cdbab4eef1c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3bf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/3409673551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04023070 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/3409673551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3c04004d20 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.519+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c080bf120 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/3409673551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c0402e050 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 10 client.17928 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569c16f07d0 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/3409673551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c080bf120 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/3409673551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3c0401e040 con 0x5569c19b8a00 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 10 client.17928 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 10 client.17928 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 20 client.17928 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c19c88e0 0x5569c19cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 4 client.17928 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:16.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569c18b3930 con 0x5569c19c88e0 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.520+0000 7f3c11464640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c19c88e0 0x5569c19cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.521+0000 7f3bf4ff9640 10 client.17928 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.521+0000 7f3c11464640 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c19c88e0 0x5569c19cacd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4d29d2a0b50b2cf server_cookie=a37cc073988d70c6 in_seq=0 out_seq=0 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.521+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/3409673551 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569c18b3930 con 0x5569c19c88e0 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.521+0000 7f3bf4ff9640 10 client.17928 handle_command_reply: tid=0 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 client.17928 shutdown 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 msgr2=0x5569c19b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b5ce0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400f980 tx=0x7f3c04004a40 comp rx=0 tx=0).stop 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 shutdown_connections 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c19c88e0 0x5569c19cacd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b5ce0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3bf4ff9640 0 client.17928 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 --2- 172.21.15.5:0/3409673551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 >> 172.21.15.5:0/3409673551 conn(0x5569c19a9ba0 msgr2=0x5569c19ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 shutdown_connections 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 1 -- 172.21.15.5:0/3409673551 wait complete. 2024-04-22T13:33:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.522+0000 7f3c16397740 20 client.17928 trim_cache size 0 max 16384 2024-04-22T13:33:16.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.535+0000 7f3c16397740 1 Processor -- start 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.535+0000 7f3c16397740 1 -- start start 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c16397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c16397740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c16397740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c16397740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1adb60 con 0x5569c19b8a00 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c16397740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1af6a0 con 0x5569c19afe80 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c16397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1ae230 con 0x5569c19af4d0 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35306/0 (socket says 172.21.15.5:35306) 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 -- 172.21.15.5:0/1656954392 learned_addr learned my addr 172.21.15.5:0/1656954392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11464640 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 -- 172.21.15.5:0/1656954392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 msgr2=0x5569c19c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 -- 172.21.15.5:0/1656954392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 msgr2=0x5569c19b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 -- 172.21.15.5:0/1656954392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c19afc20 con 0x5569c19b8a00 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3c11c65640 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b8340 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f3c04014e30 tx=0x7f3c04011260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9f1287e27857e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3bf6ffd640 1 -- 172.21.15.5:0/1656954392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04023040 con 0x5569c19b8a00 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3bf6ffd640 1 -- 172.21.15.5:0/1656954392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c04037070 con 0x5569c19b8a00 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.536+0000 7f3bf6ffd640 1 -- 172.21.15.5:0/1656954392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04018040 con 0x5569c19b8a00 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 -- 172.21.15.5:0/1656954392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 msgr2=0x5569c19b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b8340 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f3c04014e30 tx=0x7f3c04011260 comp rx=0 tx=0).stop 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 -- 172.21.15.5:0/1656954392 shutdown_connections 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19b8a00 0x5569c19b8340 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 --2- 172.21.15.5:0/1656954392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19af4d0 0x5569c19c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 -- 172.21.15.5:0/1656954392 >> 172.21.15.5:0/1656954392 conn(0x5569c19a9ba0 msgr2=0x5569c19a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 -- 172.21.15.5:0/1656954392 shutdown_connections 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 -- 172.21.15.5:0/1656954392 wait complete. 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.537+0000 7f3c16397740 1 -- start start 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 Processor -- start 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 0x5569c19c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1aefd0 con 0x5569c19af4d0 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1ae900 con 0x5569c19afe80 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1abb60 con 0x5569c19b8a00 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c16397740 10 client.0 _ll_register_callbacks cb 0x7f3c0400f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 0x5569c19c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 0x5569c19c0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35314/0 (socket says 172.21.15.5:35314) 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 -- 172.21.15.5:0/4215122610 learned_addr learned my addr 172.21.15.5:0/4215122610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11c65640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 -- 172.21.15.5:0/4215122610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 msgr2=0x5569c19bc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c10c63640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 -- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 msgr2=0x5569c19bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19bc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 -- 172.21.15.5:0/4215122610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c19bcda0 con 0x5569c19af4d0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c10c63640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3c11464640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 0x5569c19c0ca0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f3c080be960 tx=0x7f3c080bad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0fb93f3258d362 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.538+0000 7f3bf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/4215122610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080be380 con 0x5569c19af4d0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/4215122610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3c080bcd20 con 0x5569c19af4d0 2024-04-22T13:33:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c04037070 con 0x5569c19af4d0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/4215122610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c080bf120 con 0x5569c19af4d0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 10 client.17934 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569c1a2eda0 con 0x5569c19af4d0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/4215122610 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c04037070 con 0x5569c19af4d0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/4215122610 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3c080f0280 con 0x5569c19af4d0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 10 client.17934 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 10 client.17934 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 20 client.17934 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c1a2f1a0 0x5569c1a31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 4 client.17934 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569c1921c10 con 0x5569c1a2f1a0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.539+0000 7f3c10c63640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c1a2f1a0 0x5569c1a31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c10c63640 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c1a2f1a0 0x5569c1a31590 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b78b66bd63eaf1c server_cookie=870418c19fdc8278 in_seq=0 out_seq=0 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3bf4ff9640 10 client.17934 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3bf4ff9640 1 -- 172.21.15.5:0/4215122610 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569c1921c10 con 0x5569c1a2f1a0 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3bf4ff9640 10 client.17934 handle_command_reply: tid=0 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 client.17934 shutdown 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 msgr2=0x5569c19c0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 0x5569c19c0ca0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f3c080be960 tx=0x7f3c080bad40 comp rx=0 tx=0).stop 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 shutdown_connections 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5569c1a2f1a0 0x5569c1a31590 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569c19b8a00 0x5569c19bc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3bf4ff9640 0 client.17934 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569c19afe80 0x5569c19bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 --2- 172.21.15.5:0/4215122610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569c19af4d0 0x5569c19c0ca0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 >> 172.21.15.5:0/4215122610 conn(0x5569c19a9ba0 msgr2=0x5569c19b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 shutdown_connections 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 1 -- 172.21.15.5:0/4215122610 wait complete. 2024-04-22T13:33:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.540+0000 7f3c16397740 20 client.17934 trim_cache size 0 max 16384 2024-04-22T13:33:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bf80a3d20 msgr2=0x7f3bf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3c04005c40 tx=0x7f3c04009200 comp rx=0 tx=0).stop 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 msgr2=0x7f3c0c19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c19e3e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005940 tx=0x7f3c0802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 shutdown_connections 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c198590 0x7f3c0c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c0c107250 0x7f3c0c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 --2- 172.21.15.5:0/1601757364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c0c102180 0x7f3c0c19e3e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 >> 172.21.15.5:0/1601757364 conn(0x7f3c0c0fda00 msgr2=0x7f3c0c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 shutdown_connections 2024-04-22T13:33:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.541+0000 7f3c136c8640 1 -- 172.21.15.5:0/1601757364 wait complete. 2024-04-22T13:33:16.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:33:16.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 -- 172.21.15.5:0/1042069728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee94103b70 msgr2=0x7fee9410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/1042069728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee94103b70 0x7fee9410bfc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fee80009900 tx=0x7fee8002bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:16.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 -- 172.21.15.5:0/1042069728 shutdown_connections 2024-04-22T13:33:16.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/1042069728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee94103b70 0x7fee9410bfc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/1042069728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee9410c6d0 0x7fee9410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/1042069728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 0x7fee941035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 -- 172.21.15.5:0/1042069728 >> 172.21.15.5:0/1042069728 conn(0x7fee94077f70 msgr2=0x7fee94078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 -- 172.21.15.5:0/1042069728 shutdown_connections 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.655+0000 7fee9b1f4640 1 -- 172.21.15.5:0/1042069728 wait complete. 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 Processor -- start 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 -- start start 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 0x7fee9419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 0x7fee9419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94198710 0x7fee94198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee94112350 con 0x7fee9410c6d0 2024-04-22T13:33:16.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee941121d0 con 0x7fee941031c0 2024-04-22T13:33:16.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee9b1f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee941124d0 con 0x7fee94198710 2024-04-22T13:33:16.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 0x7fee9419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 0x7fee9419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35336/0 (socket says 172.21.15.5:35336) 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee93fff640 1 -- 172.21.15.5:0/2284750802 learned_addr learned my addr 172.21.15.5:0/2284750802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.656+0000 7fee98f90640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 0x7fee9419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee99791640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94198710 0x7fee94198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee93fff640 1 -- 172.21.15.5:0/2284750802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94198710 msgr2=0x7fee94198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee93fff640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94198710 0x7fee94198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee93fff640 1 -- 172.21.15.5:0/2284750802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 msgr2=0x7fee9419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee93fff640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 0x7fee9419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee93fff640 1 -- 172.21.15.5:0/2284750802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee94199310 con 0x7fee9410c6d0 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee98f90640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 0x7fee9419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee99791640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94198710 0x7fee94198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee93fff640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 0x7fee9419eb90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fee80036040 tx=0x7fee8002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee91ffb640 1 -- 172.21.15.5:0/2284750802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80043020 con 0x7fee9410c6d0 2024-04-22T13:33:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee91ffb640 1 -- 172.21.15.5:0/2284750802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8002cbe0 con 0x7fee9410c6d0 2024-04-22T13:33:16.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee941995a0 con 0x7fee9410c6d0 2024-04-22T13:33:16.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee91ffb640 1 -- 172.21.15.5:0/2284750802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80003c30 con 0x7fee9410c6d0 2024-04-22T13:33:16.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.657+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee941a5740 con 0x7fee9410c6d0 2024-04-22T13:33:16.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee91ffb640 1 -- 172.21.15.5:0/2284750802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fee80046020 con 0x7fee9410c6d0 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 Processor -- start 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 -- start start 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 0x55c5d497d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4986430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d4986a00 0x55c5d4988e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5d48e21f0 con 0x55c5d497d4d0 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c5d487f4b0 con 0x55c5d497de80 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee9dde6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c5d48de1b0 con 0x55c5d4986a00 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 0x55c5d497d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 0x55c5d497d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35350/0 (socket says 172.21.15.5:35350) 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee99791640 1 -- 172.21.15.5:0/3689304553 learned_addr learned my addr 172.21.15.5:0/3689304553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee91ffb640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fee840a3d20 0x7fee840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee98f90640 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4986430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee93fff640 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d4986a00 0x55c5d4988e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.660+0000 7fee99791640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fee840a3d20 0x7fee840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee99791640 1 -- 172.21.15.5:0/3689304553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d4986a00 msgr2=0x55c5d4988e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee99791640 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d4986a00 0x55c5d4988e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee99791640 1 -- 172.21.15.5:0/3689304553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 msgr2=0x55c5d4986430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee99791640 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4986430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee99791640 1 -- 172.21.15.5:0/3689304553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d48711b0 con 0x55c5d497d4d0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee98f90640 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4986430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee91ffb640 1 -- 172.21.15.5:0/2284750802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fee800f29d0 con 0x7fee9410c6d0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee99791640 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 0x55c5d497d8b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fee8c013910 tx=0x7fee8c013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18259ada397ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee7effd640 1 -- 172.21.15.5:0/3689304553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c004b20 con 0x55c5d497d4d0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee7effd640 1 -- 172.21.15.5:0/3689304553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c004cc0 con 0x55c5d497d4d0 2024-04-22T13:33:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.661+0000 7fee7effd640 1 -- 172.21.15.5:0/3689304553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0056f0 con 0x55c5d497d4d0 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 -- 172.21.15.5:0/3689304553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 msgr2=0x55c5d497d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 0x55c5d497d8b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fee8c013910 tx=0x7fee8c013dd0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee99791640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fee840a3d20 0x7fee840a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee8c020000 tx=0x7fee8c01f000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 -- 172.21.15.5:0/3689304553 shutdown_connections 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d4986a00 0x55c5d4988e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4986430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 --2- 172.21.15.5:0/3689304553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d497d4d0 0x55c5d497d8b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 -- 172.21.15.5:0/3689304553 >> 172.21.15.5:0/3689304553 conn(0x55c5d4977ba0 msgr2=0x55c5d4977f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 -- 172.21.15.5:0/3689304553 shutdown_connections 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 -- 172.21.15.5:0/3689304553 wait complete. 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 -- start start 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.662+0000 7fee9dde6740 1 Processor -- start 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497d4d0 0x55c5d4986360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497de80 0x55c5d49836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4983d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5d4989970 con 0x55c5d4986a00 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c5d489b500 con 0x55c5d497d4d0 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c5d4989af0 con 0x55c5d497de80 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 10 client.0 _ll_register_callbacks cb 0x55c5d4869b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4983d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4983d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35358/0 (socket says 172.21.15.5:35358) 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 -- 172.21.15.5:0/383008161 learned_addr learned my addr 172.21.15.5:0/383008161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee93fff640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497de80 0x55c5d49836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee98f90640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497d4d0 0x55c5d4986360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 -- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497de80 msgr2=0x55c5d49836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497de80 0x55c5d49836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 -- 172.21.15.5:0/383008161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497d4d0 msgr2=0x55c5d4986360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497d4d0 0x55c5d4986360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 -- 172.21.15.5:0/383008161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d45f43f0 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee99791640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4983d00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fee8c013600 tx=0x7fee8c013630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92eba7cd0f35ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee73fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee73fff640 1 -- 172.21.15.5:0/383008161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0054b0 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee73fff640 1 -- 172.21.15.5:0/383008161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fee8c022070 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.663+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c004cc0 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee73fff640 1 -- 172.21.15.5:0/383008161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee8c005650 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 10 client.17946 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5d46be7d0 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee73fff640 1 -- 172.21.15.5:0/383008161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c004cc0 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee73fff640 1 -- 172.21.15.5:0/383008161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fee8c026500 con 0x55c5d4986a00 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 10 client.17946 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 10 client.17946 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:33:16.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 20 client.17946 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d4996cd0 0x55c5d49990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 4 client.17946 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5d4881930 con 0x55c5d4996cd0 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.664+0000 7fee98f90640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d4996cd0 0x55c5d49990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.665+0000 7fee73fff640 10 client.17946 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.665+0000 7fee98f90640 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d4996cd0 0x55c5d49990c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29a4ac2e4db5fd4f server_cookie=d03acfc849570f8a in_seq=0 out_seq=0 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.665+0000 7fee73fff640 1 -- 172.21.15.5:0/383008161 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5d4881930 con 0x55c5d4996cd0 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.665+0000 7fee73fff640 10 client.17946 handle_command_reply: tid=0 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.665+0000 7fee9dde6740 1 client.17946 shutdown 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 msgr2=0x55c5d4983d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4983d00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fee8c013600 tx=0x7fee8c013630 comp rx=0 tx=0).stop 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 shutdown_connections 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d4996cd0 0x55c5d49990c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4983d00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497de80 0x55c5d49836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 --2- 172.21.15.5:0/383008161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497d4d0 0x55c5d4986360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee73fff640 0 client.17946 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 >> 172.21.15.5:0/383008161 conn(0x55c5d4977ba0 msgr2=0x55c5d49798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 shutdown_connections 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 1 -- 172.21.15.5:0/383008161 wait complete. 2024-04-22T13:33:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.666+0000 7fee9dde6740 20 client.17946 trim_cache size 0 max 16384 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 Processor -- start 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 -- start start 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d4992860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4976a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4985e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee941adb90 con 0x55c5d4986a00 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee941af6d0 con 0x55c5d497de80 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee9dde6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee941ae260 con 0x55c5d497d4d0 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4985e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4985e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35366/0 (socket says 172.21.15.5:35366) 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.679+0000 7fee99791640 1 -- 172.21.15.5:0/825753680 learned_addr learned my addr 172.21.15.5:0/825753680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee98f90640 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d4992860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee99791640 1 -- 172.21.15.5:0/825753680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 msgr2=0x55c5d4992860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee99791640 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d4992860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee99791640 1 -- 172.21.15.5:0/825753680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 msgr2=0x55c5d4976a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee99791640 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4976a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee99791640 1 -- 172.21.15.5:0/825753680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d497d0b0 con 0x55c5d4986a00 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee98f90640 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d4992860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee99791640 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4985e20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fee8c012010 tx=0x7fee8c013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f1f1d470b1881 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee7e7fc640 1 -- 172.21.15.5:0/825753680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c03e020 con 0x55c5d4986a00 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee7e7fc640 1 -- 172.21.15.5:0/825753680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c026bb0 con 0x55c5d4986a00 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee7e7fc640 1 -- 172.21.15.5:0/825753680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c004c10 con 0x55c5d4986a00 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee9dde6740 1 -- 172.21.15.5:0/825753680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 msgr2=0x55c5d4985e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee9dde6740 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4985e20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fee8c012010 tx=0x7fee8c013dd0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.680+0000 7fee9dde6740 1 -- 172.21.15.5:0/825753680 shutdown_connections 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4985e20 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d4976a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 --2- 172.21.15.5:0/825753680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d4992860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- 172.21.15.5:0/825753680 >> 172.21.15.5:0/825753680 conn(0x55c5d4977ba0 msgr2=0x55c5d4976370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- 172.21.15.5:0/825753680 shutdown_connections 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- 172.21.15.5:0/825753680 wait complete. 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- start start 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 Processor -- start 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d498ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d497ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4980480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee941af000 con 0x55c5d4986a00 2024-04-22T13:33:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fee941ae930 con 0x55c5d497de80 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee941abb90 con 0x55c5d497d4d0 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.681+0000 7fee9dde6740 10 client.0 _ll_register_callbacks cb 0x7fee8c013710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4980480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4980480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35372/0 (socket says 172.21.15.5:35372) 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 -- 172.21.15.5:0/1828925621 learned_addr learned my addr 172.21.15.5:0/1828925621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee98f90640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d498ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee93fff640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d497ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 -- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 msgr2=0x55c5d498ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d498ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 -- 172.21.15.5:0/1828925621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 msgr2=0x55c5d497ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d497ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 -- 172.21.15.5:0/1828925621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d49809c0 con 0x55c5d4986a00 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee99791640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4980480 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0136e0 tx=0x7fee8c0132e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b5a292a4b458e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee73fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee73fff640 1 -- 172.21.15.5:0/1828925621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c005950 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c026bb0 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee73fff640 1 -- 172.21.15.5:0/1828925621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fee8c002030 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee73fff640 1 -- 172.21.15.5:0/1828925621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee8c026470 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee9dde6740 10 client.17952 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.682+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5d49fcd10 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee73fff640 1 -- 172.21.15.5:0/1828925621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c026bb0 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee73fff640 1 -- 172.21.15.5:0/1828925621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fee8c03a450 con 0x55c5d4986a00 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee9dde6740 10 client.17952 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee9dde6740 10 client.17952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee9dde6740 20 client.17952 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee9dde6740 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d49fd310 0x55c5d49ff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee9dde6740 4 client.17952 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5d48efc10 con 0x55c5d49fd310 2024-04-22T13:33:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee98f90640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d49fd310 0x55c5d49ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee98f90640 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d49fd310 0x55c5d49ff730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b97ecb4280ac942f server_cookie=3eb5a050118a4c65 in_seq=0 out_seq=0 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.683+0000 7fee73fff640 10 client.17952 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee73fff640 1 -- 172.21.15.5:0/1828925621 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5d48efc10 con 0x55c5d49fd310 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee73fff640 10 client.17952 handle_command_reply: tid=0 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 client.17952 shutdown 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 msgr2=0x55c5d4980480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4980480 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0136e0 tx=0x7fee8c0132e0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 shutdown_connections 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c5d49fd310 0x55c5d49ff730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5d4986a00 0x55c5d4980480 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee73fff640 0 client.17952 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c5d497de80 0x55c5d497ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 --2- 172.21.15.5:0/1828925621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5d497d4d0 0x55c5d498ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 >> 172.21.15.5:0/1828925621 conn(0x55c5d4977ba0 msgr2=0x55c5d49797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 shutdown_connections 2024-04-22T13:33:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 1 -- 172.21.15.5:0/1828925621 wait complete. 2024-04-22T13:33:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9dde6740 20 client.17952 trim_cache size 0 max 16384 2024-04-22T13:33:16.687 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fee840a3d20 msgr2=0x7fee840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fee840a3d20 0x7fee840a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee8c020000 tx=0x7fee8c01f000 comp rx=0 tx=0).stop 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 msgr2=0x7fee9419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.684+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 0x7fee9419eb90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fee80036040 tx=0x7fee8002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 shutdown_connections 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee9410c6d0 0x7fee9419eb90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fee840a3d20 0x7fee840a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94198710 0x7fee94198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 --2- 172.21.15.5:0/2284750802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee941031c0 0x7fee9419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 >> 172.21.15.5:0/2284750802 conn(0x7fee94077f70 msgr2=0x7fee940fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 shutdown_connections 2024-04-22T13:33:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.685+0000 7fee9b1f4640 1 -- 172.21.15.5:0/2284750802 wait complete. 2024-04-22T13:33:16.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:33:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.799+0000 7f1e03b4a640 1 -- 172.21.15.5:0/3163830918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 msgr2=0x7f1dfc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.799+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/3163830918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc1035a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009900 tx=0x7f1df802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- 172.21.15.5:0/3163830918 shutdown_connections 2024-04-22T13:33:16.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/3163830918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc10c6d0 0x7f1dfc10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/3163830918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc103b70 0x7f1dfc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/3163830918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc1035a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- 172.21.15.5:0/3163830918 >> 172.21.15.5:0/3163830918 conn(0x7f1dfc077f70 msgr2=0x7f1dfc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- 172.21.15.5:0/3163830918 shutdown_connections 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- 172.21.15.5:0/3163830918 wait complete. 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 Processor -- start 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- start start 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc10c6d0 0x7f1dfc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc1988c0 0x7f1dfc198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112390 con 0x7f1dfc1031c0 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112210 con 0x7f1dfc1988c0 2024-04-22T13:33:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.800+0000 7f1e03b4a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112510 con 0x7f1dfc10c6d0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35392/0 (socket says 172.21.15.5:35392) 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 -- 172.21.15.5:0/480532923 learned_addr learned my addr 172.21.15.5:0/480532923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e020e7640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc1988c0 0x7f1dfc198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e010e5640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc10c6d0 0x7f1dfc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 -- 172.21.15.5:0/480532923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc10c6d0 msgr2=0x7f1dfc19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc10c6d0 0x7f1dfc19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 -- 172.21.15.5:0/480532923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc1988c0 msgr2=0x7f1dfc198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc1988c0 0x7f1dfc198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 -- 172.21.15.5:0/480532923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc1993e0 con 0x7f1dfc1031c0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e020e7640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc1988c0 0x7f1dfc198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e010e5640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc10c6d0 0x7f1dfc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e018e6640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc19e6c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1df8005940 tx=0x7f1df802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1df2ffd640 1 -- 172.21.15.5:0/480532923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8043020 con 0x7f1dfc1031c0 2024-04-22T13:33:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1df2ffd640 1 -- 172.21.15.5:0/480532923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df802ca70 con 0x7f1dfc1031c0 2024-04-22T13:33:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1df2ffd640 1 -- 172.21.15.5:0/480532923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80039f0 con 0x7f1dfc1031c0 2024-04-22T13:33:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc199670 con 0x7f1dfc1031c0 2024-04-22T13:33:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.801+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1a5870 con 0x7f1dfc1031c0 2024-04-22T13:33:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.803+0000 7f1df2ffd640 1 -- 172.21.15.5:0/480532923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1df8046020 con 0x7f1dfc1031c0 2024-04-22T13:33:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 Processor -- start 2024-04-22T13:33:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 -- start start 2024-04-22T13:33:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x5601734048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560173404e80 0x56017340d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56017340da00 0x56017340fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601733691f0 con 0x5601734044d0 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5601733064b0 con 0x56017340da00 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1e06810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5601733651b0 con 0x560173404e80 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.804+0000 7f1df2ffd640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1de80a3d20 0x7f1de80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x5601734048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x5601734048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35400/0 (socket says 172.21.15.5:35400) 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 -- 172.21.15.5:0/3496425418 learned_addr learned my addr 172.21.15.5:0/3496425418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e010e5640 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56017340da00 0x56017340fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1de80a3d20 0x7f1de80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1df2ffd640 1 -- 172.21.15.5:0/480532923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1df80f2850 con 0x7f1dfc1031c0 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e018e6640 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560173404e80 0x56017340d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 -- 172.21.15.5:0/3496425418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560173404e80 msgr2=0x56017340d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560173404e80 0x56017340d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 -- 172.21.15.5:0/3496425418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56017340da00 msgr2=0x56017340fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56017340da00 0x56017340fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 -- 172.21.15.5:0/3496425418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601732f81b0 con 0x5601734044d0 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e010e5640 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56017340da00 0x56017340fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x5601734048b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1df4014ea0 tx=0x7f1df4014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822e23c7bda7bf3e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1e020e7640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1de80a3d20 0x7f1de80a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011c40 tx=0x7f1df401c000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1de7fff640 1 -- 172.21.15.5:0/3496425418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4022070 con 0x5601734044d0 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1de7fff640 1 -- 172.21.15.5:0/3496425418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4003e20 con 0x5601734044d0 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.805+0000 7f1de7fff640 1 -- 172.21.15.5:0/3496425418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4004290 con 0x5601734044d0 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 -- 172.21.15.5:0/3496425418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 msgr2=0x5601734048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x5601734048b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1df4014ea0 tx=0x7f1df4014f90 comp rx=0 tx=0).stop 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 -- 172.21.15.5:0/3496425418 shutdown_connections 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56017340da00 0x56017340fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560173404e80 0x56017340d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 --2- 172.21.15.5:0/3496425418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x5601734048b0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 -- 172.21.15.5:0/3496425418 >> 172.21.15.5:0/3496425418 conn(0x5601733feba0 msgr2=0x5601733fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 -- 172.21.15.5:0/3496425418 shutdown_connections 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.806+0000 7f1e06810740 1 -- 172.21.15.5:0/3496425418 wait complete. 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 -- start start 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 Processor -- start 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601734044d0 0x56017340d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 0x56017340a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x56017340ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560173410910 con 0x560173404e80 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560173322500 con 0x5601734044d0 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560173410a90 con 0x56017340da00 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e06810740 10 client.0 _ll_register_callbacks cb 0x5601732f0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 0x56017340a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 0x56017340a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35410/0 (socket says 172.21.15.5:35410) 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 -- 172.21.15.5:0/4123854126 learned_addr learned my addr 172.21.15.5:0/4123854126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e020e7640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x56017340ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e018e6640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601734044d0 0x56017340d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 -- 172.21.15.5:0/4123854126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 msgr2=0x56017340ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x56017340ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 -- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601734044d0 msgr2=0x56017340d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601734044d0 0x56017340d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e010e5640 1 -- 172.21.15.5:0/4123854126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017307b3f0 con 0x560173404e80 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.807+0000 7f1e018e6640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601734044d0 0x56017340d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e010e5640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 0x56017340a710 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0136e0 tx=0x7f1dec013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6a530cf33c6c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1de57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1de57fa640 1 -- 172.21.15.5:0/4123854126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec004460 con 0x560173404e80 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1de57fa640 1 -- 172.21.15.5:0/4123854126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1dec0119c0 con 0x560173404e80 2024-04-22T13:33:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df4003e20 con 0x560173404e80 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1de57fa640 1 -- 172.21.15.5:0/4123854126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1dec011bc0 con 0x560173404e80 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 10 client.17964 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601731457d0 con 0x560173404e80 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1de57fa640 1 -- 172.21.15.5:0/4123854126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4003e20 con 0x560173404e80 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1de57fa640 1 -- 172.21.15.5:0/4123854126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1dec01a020 con 0x560173404e80 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 10 client.17964 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 10 client.17964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 20 client.17964 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x56017341dbe0 0x56017341fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 4 client.17964 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.808+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560173308930 con 0x56017341dbe0 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.809+0000 7f1e018e6640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x56017341dbe0 0x56017341fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.809+0000 7f1de57fa640 10 client.17964 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.809+0000 7f1e018e6640 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x56017341dbe0 0x56017341fff0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c8c4e38ac998c5 server_cookie=fd54073b3a015fb0 in_seq=0 out_seq=0 2024-04-22T13:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1de57fa640 1 -- 172.21.15.5:0/4123854126 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560173308930 con 0x56017341dbe0 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1de57fa640 10 client.17964 handle_command_reply: tid=0 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 client.17964 shutdown 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 msgr2=0x56017340a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 0x56017340a710 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0136e0 tx=0x7f1dec013ba0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 shutdown_connections 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x56017341dbe0 0x56017341fff0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560173404e80 0x56017340a710 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x56017340ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 --2- 172.21.15.5:0/4123854126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601734044d0 0x56017340d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1de57fa640 0 client.17964 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 >> 172.21.15.5:0/4123854126 conn(0x5601733feba0 msgr2=0x5601734008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 shutdown_connections 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 1 -- 172.21.15.5:0/4123854126 wait complete. 2024-04-22T13:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.810+0000 7f1e06810740 20 client.17964 trim_cache size 0 max 16384 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 Processor -- start 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 -- start start 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5601734044d0 0x5601734197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 0x56017340ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 0x56017340d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1adc30 con 0x56017340da00 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1af770 con 0x560173404e80 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e06810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1ae300 con 0x5601734044d0 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 0x56017340d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 0x56017340d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35412/0 (socket says 172.21.15.5:35412) 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 -- 172.21.15.5:0/4161088238 learned_addr learned my addr 172.21.15.5:0/4161088238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e018e6640 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5601734044d0 0x5601734197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 -- 172.21.15.5:0/4161088238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5601734044d0 msgr2=0x5601734197d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5601734044d0 0x5601734197d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 -- 172.21.15.5:0/4161088238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 msgr2=0x56017340ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 0x56017340ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 -- 172.21.15.5:0/4161088238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601733fa470 con 0x56017340da00 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.824+0000 7f1e020e7640 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 0x56017340d380 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f1df4014e70 tx=0x7f1df4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec4f6d48f3f6e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1de77fe640 1 -- 172.21.15.5:0/4161088238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4022070 con 0x56017340da00 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1de77fe640 1 -- 172.21.15.5:0/4161088238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4003e20 con 0x56017340da00 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1de77fe640 1 -- 172.21.15.5:0/4161088238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df40050b0 con 0x56017340da00 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 -- 172.21.15.5:0/4161088238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 msgr2=0x56017340d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 0x56017340d380 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f1df4014e70 tx=0x7f1df4005040 comp rx=0 tx=0).stop 2024-04-22T13:33:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 -- 172.21.15.5:0/4161088238 shutdown_connections 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56017340da00 0x56017340d380 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 0x56017340ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 --2- 172.21.15.5:0/4161088238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5601734044d0 0x5601734197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 -- 172.21.15.5:0/4161088238 >> 172.21.15.5:0/4161088238 conn(0x5601733feba0 msgr2=0x5601733fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 -- 172.21.15.5:0/4161088238 shutdown_connections 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.825+0000 7f1e06810740 1 -- 172.21.15.5:0/4161088238 wait complete. 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 -- start start 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 Processor -- start 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x560173411b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 0x560173412040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x560173483130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1af0a0 con 0x5601734044d0 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1ae9d0 con 0x560173404e80 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1abc30 con 0x56017340da00 2024-04-22T13:33:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e06810740 10 client.0 _ll_register_callbacks cb 0x7f1dec009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x560173411b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x560173411b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35422/0 (socket says 172.21.15.5:35422) 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 -- 172.21.15.5:0/477431080 learned_addr learned my addr 172.21.15.5:0/477431080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e020e7640 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x560173483130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 -- 172.21.15.5:0/477431080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 msgr2=0x560173483130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x560173483130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 -- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 msgr2=0x560173412040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 0x560173412040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.826+0000 7f1e018e6640 1 -- 172.21.15.5:0/477431080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560173483670 con 0x5601734044d0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e018e6640 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x560173411b00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1df80be8c0 tx=0x7f1df8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2780b56048413b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1de57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1de57fa640 1 -- 172.21.15.5:0/477431080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80c2ce0 con 0x5601734044d0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1de57fa640 1 -- 172.21.15.5:0/477431080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f1df80bafa0 con 0x5601734044d0 2024-04-22T13:33:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df4003e20 con 0x5601734044d0 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1de57fa640 1 -- 172.21.15.5:0/477431080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1df80c75a0 con 0x5601734044d0 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 10 client.17970 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560173484830 con 0x5601734044d0 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1de57fa640 1 -- 172.21.15.5:0/477431080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4003e20 con 0x5601734044d0 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1de57fa640 1 -- 172.21.15.5:0/477431080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1df80c7110 con 0x5601734044d0 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 10 client.17970 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 10 client.17970 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 20 client.17970 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560173484f30 0x560173487370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 4 client.17970 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.827+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560173376c10 con 0x560173484f30 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e010e5640 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560173484f30 0x560173487370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1de57fa640 10 client.17970 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e010e5640 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560173484f30 0x560173487370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6191d6d5d5506aa server_cookie=3ffb5bc21e9dd2a7 in_seq=0 out_seq=0 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1de57fa640 1 -- 172.21.15.5:0/477431080 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560173376c10 con 0x560173484f30 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1de57fa640 10 client.17970 handle_command_reply: tid=0 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 client.17970 shutdown 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 msgr2=0x560173411b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x560173411b00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1df80be8c0 tx=0x7f1df8047000 comp rx=0 tx=0).stop 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 shutdown_connections 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x560173484f30 0x560173487370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56017340da00 0x560173483130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560173404e80 0x560173412040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 --2- 172.21.15.5:0/477431080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601734044d0 0x560173411b00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1de57fa640 0 client.17970 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 >> 172.21.15.5:0/477431080 conn(0x5601733feba0 msgr2=0x5601733fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 shutdown_connections 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.828+0000 7f1e06810740 1 -- 172.21.15.5:0/477431080 wait complete. 2024-04-22T13:33:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e06810740 20 client.17970 trim_cache size 0 max 16384 2024-04-22T13:33:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1de80a3d20 msgr2=0x7f1de80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1de80a3d20 0x7f1de80a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011c40 tx=0x7f1df401c000 comp rx=0 tx=0).stop 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 msgr2=0x7f1dfc19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc19e6c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1df8005940 tx=0x7f1df802c5f0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 shutdown_connections 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1de80a3d20 0x7f1de80a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc1988c0 0x7f1dfc198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dfc10c6d0 0x7f1dfc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 --2- 172.21.15.5:0/480532923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dfc1031c0 0x7f1dfc19e6c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 >> 172.21.15.5:0/480532923 conn(0x7f1dfc077f70 msgr2=0x7f1dfc0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 shutdown_connections 2024-04-22T13:33:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.829+0000 7f1e03b4a640 1 -- 172.21.15.5:0/480532923 wait complete. 2024-04-22T13:33:16.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:33:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 -- 172.21.15.5:0/2703776644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9090102270 msgr2=0x7f9090102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 --2- 172.21.15.5:0/2703776644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9090102270 0x7f9090102650 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f908c009930 tx=0x7f908c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 -- 172.21.15.5:0/2703776644 shutdown_connections 2024-04-22T13:33:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 --2- 172.21.15.5:0/2703776644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90901044f0 0x7f909010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 --2- 172.21.15.5:0/2703776644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9090103b50 0x7f9090103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 --2- 172.21.15.5:0/2703776644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9090102270 0x7f9090102650 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 -- 172.21.15.5:0/2703776644 >> 172.21.15.5:0/2703776644 conn(0x7f90900fda10 msgr2=0x7f90900ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 -- 172.21.15.5:0/2703776644 shutdown_connections 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.944+0000 7f9097b0e640 1 -- 172.21.15.5:0/2703776644 wait complete. 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 Processor -- start 2024-04-22T13:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 -- start start 2024-04-22T13:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9090102270 0x7f909019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9090103b50 0x7f909019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 0x7f9090198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9090112290 con 0x7f90901044f0 2024-04-22T13:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9090112110 con 0x7f9090102270 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f9097b0e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9090112410 con 0x7f9090103b50 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f90960ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 0x7f9090198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f90960ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 0x7f9090198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35436/0 (socket says 172.21.15.5:35436) 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f90960ab640 1 -- 172.21.15.5:0/1930471449 learned_addr learned my addr 172.21.15.5:0/1930471449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.945+0000 7f90950a9640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9090103b50 0x7f909019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90958aa640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9090102270 0x7f909019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90960ab640 1 -- 172.21.15.5:0/1930471449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9090103b50 msgr2=0x7f909019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90960ab640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9090103b50 0x7f909019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90960ab640 1 -- 172.21.15.5:0/1930471449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9090102270 msgr2=0x7f909019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90960ab640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9090102270 0x7f909019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90960ab640 1 -- 172.21.15.5:0/1930471449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9090198d10 con 0x7f90901044f0 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90958aa640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9090102270 0x7f909019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f90960ab640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 0x7f9090198590 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f908800cce0 tx=0x7f908800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f9086ffd640 1 -- 172.21.15.5:0/1930471449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908800f450 con 0x7f90901044f0 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f9086ffd640 1 -- 172.21.15.5:0/1930471449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908800faa0 con 0x7f90901044f0 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9090199000 con 0x7f90901044f0 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f9086ffd640 1 -- 172.21.15.5:0/1930471449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088018b70 con 0x7f90901044f0 2024-04-22T13:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.946+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90901a52a0 con 0x7f90901044f0 2024-04-22T13:33:16.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.948+0000 7f9086ffd640 1 -- 172.21.15.5:0/1930471449 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9088018d10 con 0x7f90901044f0 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 Processor -- start 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 -- start start 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 0x56210eaee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eaf9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56210ea531f0 con 0x56210eaee4d0 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56210e9f04b0 con 0x56210eaeee80 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f909a7db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56210ea4f1b0 con 0x56210eaf7a00 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f9086ffd640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f907c0a3d20 0x7f907c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f90950a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 0x56210eaee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f90960ab640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f907c0a3d20 0x7f907c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f90950a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 0x56210eaee8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35444/0 (socket says 172.21.15.5:35444) 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f90950a9640 1 -- 172.21.15.5:0/3047833852 learned_addr learned my addr 172.21.15.5:0/3047833852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f9086ffd640 1 -- 172.21.15.5:0/1930471449 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f90880d0c90 con 0x7f90901044f0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.949+0000 7f90958aa640 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eaf9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 -- 172.21.15.5:0/3047833852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 msgr2=0x56210eaf9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eaf9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 -- 172.21.15.5:0/3047833852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 msgr2=0x56210eaf7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 -- 172.21.15.5:0/3047833852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210e9e21b0 con 0x56210eaee4d0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90958aa640 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90950a9640 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 0x56210eaee8b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f908000bb70 tx=0x7f9080004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65347042169be922 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f90960ab640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f907c0a3d20 0x7f907c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90880995d0 tx=0x7f9088092090 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f907bfff640 1 -- 172.21.15.5:0/3047833852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080005020 con 0x56210eaee4d0 2024-04-22T13:33:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f907bfff640 1 -- 172.21.15.5:0/3047833852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9080005670 con 0x56210eaee4d0 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.950+0000 7f907bfff640 1 -- 172.21.15.5:0/3047833852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90800106c0 con 0x56210eaee4d0 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 -- 172.21.15.5:0/3047833852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 msgr2=0x56210eaee8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 0x56210eaee8b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f908000bb70 tx=0x7f9080004610 comp rx=0 tx=0).stop 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 -- 172.21.15.5:0/3047833852 shutdown_connections 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eaf9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 --2- 172.21.15.5:0/3047833852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaee4d0 0x56210eaee8b0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 -- 172.21.15.5:0/3047833852 >> 172.21.15.5:0/3047833852 conn(0x56210eae8ba0 msgr2=0x56210eae8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 -- 172.21.15.5:0/3047833852 shutdown_connections 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 -- 172.21.15.5:0/3047833852 wait complete. 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 -- start start 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.951+0000 7f909a7db740 1 Processor -- start 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eaf7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 0x56210eaf4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56210eafa910 con 0x56210eaf7a00 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56210ea0c500 con 0x56210eaeee80 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56210eafaa90 con 0x56210eaee4d0 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f909a7db740 10 client.0 _ll_register_callbacks cb 0x56210e9dab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 0x56210eaf4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 0x56210eaf4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35448/0 (socket says 172.21.15.5:35448) 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 -- 172.21.15.5:0/1485971858 learned_addr learned my addr 172.21.15.5:0/1485971858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eaf7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90958aa640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 -- 172.21.15.5:0/1485971858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 msgr2=0x56210eaf7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eaf7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 -- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 msgr2=0x56210eaf4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 -- 172.21.15.5:0/1485971858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210e7653f0 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90950a9640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 0x56210eaf4b40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9080005fd0 tx=0x7f9080004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe454b3af81d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90797fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.952+0000 7f90797fa640 1 -- 172.21.15.5:0/1485971858 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080005000 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f90797fa640 1 -- 172.21.15.5:0/1485971858 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f90800051a0 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9080005670 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f90797fa640 1 -- 172.21.15.5:0/1485971858 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90800053a0 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 10 client.17982 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56210e82f7d0 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f90797fa640 1 -- 172.21.15.5:0/1485971858 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9080005670 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f90797fa640 1 -- 172.21.15.5:0/1485971858 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f908001f410 con 0x56210eaf7a00 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 10 client.17982 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 20 client.17982 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb079a0 0x56210eb09d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 4 client.17982 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56210e9f2930 con 0x56210eb079a0 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.953+0000 7f90958aa640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb079a0 0x56210eb09d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.954+0000 7f90797fa640 10 client.17982 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.954+0000 7f90958aa640 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb079a0 0x56210eb09d90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f72a438e12d5772 server_cookie=1d2461a20a7d509 in_seq=0 out_seq=0 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.954+0000 7f90797fa640 1 -- 172.21.15.5:0/1485971858 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56210e9f2930 con 0x56210eb079a0 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.954+0000 7f90797fa640 10 client.17982 handle_command_reply: tid=0 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.954+0000 7f909a7db740 1 client.17982 shutdown 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 msgr2=0x56210eaf4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 0x56210eaf4b40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9080005fd0 tx=0x7f9080004cc0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 shutdown_connections 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb079a0 0x56210eb09d90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaf7a00 0x56210eaf4b40 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaeee80 0x56210eaf4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f90797fa640 0 client.17982 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 --2- 172.21.15.5:0/1485971858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eaf7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 >> 172.21.15.5:0/1485971858 conn(0x56210eae8ba0 msgr2=0x56210eaea8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 shutdown_connections 2024-04-22T13:33:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 1 -- 172.21.15.5:0/1485971858 wait complete. 2024-04-22T13:33:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.955+0000 7f909a7db740 20 client.17982 trim_cache size 0 max 16384 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.968+0000 7f909a7db740 1 Processor -- start 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.968+0000 7f909a7db740 1 -- start start 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.968+0000 7f909a7db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eb037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.968+0000 7f909a7db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eae7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.968+0000 7f909a7db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaf7a00 0x56210eaf6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f909a7db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90901ada70 con 0x56210eaeee80 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f909a7db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90901af5b0 con 0x56210eaf7a00 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f909a7db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90901ae140 con 0x56210eaee4d0 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eae7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eae7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35460/0 (socket says 172.21.15.5:35460) 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 -- 172.21.15.5:0/531727503 learned_addr learned my addr 172.21.15.5:0/531727503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 -- 172.21.15.5:0/531727503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 msgr2=0x56210eb037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eb037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 -- 172.21.15.5:0/531727503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaf7a00 msgr2=0x56210eaf6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaf7a00 0x56210eaf6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 -- 172.21.15.5:0/531727503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210eae89d0 con 0x56210eaeee80 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f90958aa640 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eae7e40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f908c009900 tx=0x7f908c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9ce8aa6e6df6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f907b7fe640 1 -- 172.21.15.5:0/531727503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c02c7e0 con 0x56210eaeee80 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f907b7fe640 1 -- 172.21.15.5:0/531727503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908c005040 con 0x56210eaeee80 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.969+0000 7f907b7fe640 1 -- 172.21.15.5:0/531727503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c004b80 con 0x56210eaeee80 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 -- 172.21.15.5:0/531727503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 msgr2=0x56210eae7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eae7e40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f908c009900 tx=0x7f908c009670 comp rx=0 tx=0).stop 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 -- 172.21.15.5:0/531727503 shutdown_connections 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eae7e40 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaf7a00 0x56210eaf6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 --2- 172.21.15.5:0/531727503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaee4d0 0x56210eb037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 -- 172.21.15.5:0/531727503 >> 172.21.15.5:0/531727503 conn(0x56210eae8ba0 msgr2=0x56210eaefaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 -- 172.21.15.5:0/531727503 shutdown_connections 2024-04-22T13:33:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 -- 172.21.15.5:0/531727503 wait complete. 2024-04-22T13:33:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 -- start start 2024-04-22T13:33:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.970+0000 7f909a7db740 1 Processor -- start 2024-04-22T13:33:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaee4d0 0x56210eb000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eafb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eafb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90901aeee0 con 0x56210eaeee80 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90901ae810 con 0x56210eaee4d0 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90901aba50 con 0x56210eaf7a00 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 10 client.0 _ll_register_callbacks cb 0x7f90800046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eafb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eafb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35468/0 (socket says 172.21.15.5:35468) 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 -- 172.21.15.5:0/862013489 learned_addr learned my addr 172.21.15.5:0/862013489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90950a9640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaee4d0 0x56210eb000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90950a9640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eafb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 -- 172.21.15.5:0/862013489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 msgr2=0x56210eafb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eafb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 -- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaee4d0 msgr2=0x56210eb000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaee4d0 0x56210eb000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 -- 172.21.15.5:0/862013489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210eafc170 con 0x56210eaeee80 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90958aa640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eafb430 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f908c009850 tx=0x7f908c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce888f3196590f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90797fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90797fa640 1 -- 172.21.15.5:0/862013489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c055070 con 0x56210eaeee80 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f90797fa640 1 -- 172.21.15.5:0/862013489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f908c0054b0 con 0x56210eaeee80 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.971+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908c005040 con 0x56210eaeee80 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f90797fa640 1 -- 172.21.15.5:0/862013489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f908c02c7e0 con 0x56210eaeee80 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 10 client.17988 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56210eb6de10 con 0x56210eaeee80 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f90797fa640 1 -- 172.21.15.5:0/862013489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908c005040 con 0x56210eaeee80 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f90797fa640 1 -- 172.21.15.5:0/862013489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f908c0035f0 con 0x56210eaeee80 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 10 client.17988 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 20 client.17988 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb6e4a0 0x56210eb708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 4 client.17988 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56210ea60c10 con 0x56210eb6e4a0 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.972+0000 7f90950a9640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb6e4a0 0x56210eb708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f90797fa640 10 client.17988 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f90950a9640 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb6e4a0 0x56210eb708c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8535c560058d81ef server_cookie=a0a90c1b721202ca in_seq=0 out_seq=0 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f90797fa640 1 -- 172.21.15.5:0/862013489 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56210ea60c10 con 0x56210eb6e4a0 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f90797fa640 10 client.17988 handle_command_reply: tid=0 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 client.17988 shutdown 2024-04-22T13:33:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 msgr2=0x56210eafb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eafb430 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f908c009850 tx=0x7f908c004470 comp rx=0 tx=0).stop 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 shutdown_connections 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56210eb6e4a0 0x56210eb708c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210eaeee80 0x56210eafb430 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f90797fa640 0 client.17988 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56210eaf7a00 0x56210eafb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 --2- 172.21.15.5:0/862013489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56210eaee4d0 0x56210eb000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 >> 172.21.15.5:0/862013489 conn(0x56210eae8ba0 msgr2=0x56210eb027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 shutdown_connections 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 1 -- 172.21.15.5:0/862013489 wait complete. 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.973+0000 7f909a7db740 20 client.17988 trim_cache size 0 max 16384 2024-04-22T13:33:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f907c0a3d20 msgr2=0x7f907c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f907c0a3d20 0x7f907c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90880995d0 tx=0x7f9088092090 comp rx=0 tx=0).stop 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 msgr2=0x7f9090198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 0x7f9090198590 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f908800cce0 tx=0x7f908800a620 comp rx=0 tx=0).stop 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 shutdown_connections 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90901044f0 0x7f9090198590 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f907c0a3d20 0x7f907c0a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9090103b50 0x7f909019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 --2- 172.21.15.5:0/1930471449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9090102270 0x7f909019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 >> 172.21.15.5:0/1930471449 conn(0x7f90900fda10 msgr2=0x7f90900fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 shutdown_connections 2024-04-22T13:33:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:16.974+0000 7f9097b0e640 1 -- 172.21.15.5:0/1930471449 wait complete. 2024-04-22T13:33:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:17 smithi005 ceph-mon[28006]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:17.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:17 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4194074038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:17.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:17 smithi049 ceph-mon[31379]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:17.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:17 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4194074038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:17.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:17 smithi107 ceph-mon[31859]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:33:17.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:17 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4194074038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:17.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:33:17.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:33:17.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:33:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3294759053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 msgr2=0x7f5df410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3294759053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df410eb60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009ca0 tx=0x7f5dec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3294759053 shutdown_connections 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3294759053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df410eb60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3294759053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4103b70 0x7f5df410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3294759053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41031c0 0x7f5df41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3294759053 >> 172.21.15.5:0/3294759053 conn(0x7f5df4077f70 msgr2=0x7f5df4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.105+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3294759053 shutdown_connections 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3294759053 wait complete. 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 Processor -- start 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 -- start start 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41031c0 0x7f5df419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4198740 0x7f5df4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4112380 con 0x7f5df410c6d0 2024-04-22T13:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4112200 con 0x7f5df41031c0 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.106+0000 7f5dfa3ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4112500 con 0x7f5df4198740 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35482/0 (socket says 172.21.15.5:35482) 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 -- 172.21.15.5:0/3189891571 learned_addr learned my addr 172.21.15.5:0/3189891571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df3fff640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41031c0 0x7f5df419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df896b640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4198740 0x7f5df4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 -- 172.21.15.5:0/3189891571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4198740 msgr2=0x7f5df4198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4198740 0x7f5df4198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 -- 172.21.15.5:0/3189891571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41031c0 msgr2=0x7f5df419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41031c0 0x7f5df419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 -- 172.21.15.5:0/3189891571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df4199340 con 0x7f5df410c6d0 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df896b640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4198740 0x7f5df4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df37fe640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df419eb70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d950 tx=0x7f5de400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df17fa640 1 -- 172.21.15.5:0/3189891571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5df410c6d0 2024-04-22T13:33:18.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df41a53d0 con 0x7f5df410c6d0 2024-04-22T13:33:18.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df17fa640 1 -- 172.21.15.5:0/3189891571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de4009c30 con 0x7f5df410c6d0 2024-04-22T13:33:18.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5df17fa640 1 -- 172.21.15.5:0/3189891571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40047a0 con 0x7f5df410c6d0 2024-04-22T13:33:18.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.107+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df41a57d0 con 0x7f5df410c6d0 2024-04-22T13:33:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.110+0000 7f5df17fa640 1 -- 172.21.15.5:0/3189891571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5de4004020 con 0x7f5df410c6d0 2024-04-22T13:33:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.110+0000 7f5dfcfbe740 1 Processor -- start 2024-04-22T13:33:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.110+0000 7f5dfcfbe740 1 -- start start 2024-04-22T13:33:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.110+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 0x5562ece03cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.110+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece0c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece0ce10 0x5562ece0f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562ecd67f00 con 0x5562ece038e0 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5562ecd05590 con 0x5562ece04290 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562ecd63ec0 con 0x5562ece0ce10 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 0x5562ece03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 0x5562ece03cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35490/0 (socket says 172.21.15.5:35490) 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 -- 172.21.15.5:0/393428599 learned_addr learned my addr 172.21.15.5:0/393428599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df17fa640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5ddc0a3d20 0x7f5ddc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df3fff640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece0c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df37fe640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece0ce10 0x5562ece0f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 -- 172.21.15.5:0/393428599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece0ce10 msgr2=0x5562ece0f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece0ce10 0x5562ece0f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 -- 172.21.15.5:0/393428599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 msgr2=0x5562ece0c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece0c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 -- 172.21.15.5:0/393428599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ecb427e0 con 0x5562ece038e0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df17fa640 1 -- 172.21.15.5:0/3189891571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5de40d04c0 con 0x7f5df410c6d0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df37fe640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece0ce10 0x5562ece0f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df3fff640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece0c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5ddc0a3d20 0x7f5ddc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.111+0000 7f5df896b640 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 0x5562ece03cc0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009430 tx=0x7f5dec0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfafbcd05691bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dda7fc640 1 -- 172.21.15.5:0/393428599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec039050 con 0x5562ece038e0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dda7fc640 1 -- 172.21.15.5:0/393428599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec046030 con 0x5562ece038e0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dda7fc640 1 -- 172.21.15.5:0/393428599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec034440 con 0x5562ece038e0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/393428599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 msgr2=0x5562ece03cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 0x5562ece03cc0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009430 tx=0x7f5dec0098a0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5df896b640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5ddc0a3d20 0x7f5ddc0a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5dec04a000 tx=0x7f5dec04b000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/393428599 shutdown_connections 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece0ce10 0x5562ece0f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece0c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/393428599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece038e0 0x5562ece03cc0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/393428599 >> 172.21.15.5:0/393428599 conn(0x5562ecdfdfb0 msgr2=0x5562ecdfe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/393428599 shutdown_connections 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.112+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/393428599 wait complete. 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 -- start start 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 Processor -- start 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece09e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece0a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece0a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562ece0fd80 con 0x5562ece0ce10 2024-04-22T13:33:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5562ecd21580 con 0x5562ece038e0 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562ece0ff00 con 0x5562ece04290 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5dfcfbe740 10 client.0 _ll_register_callbacks cb 0x5562ecd27650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece0a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece0a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35500/0 (socket says 172.21.15.5:35500) 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.113+0000 7f5df896b640 1 -- 172.21.15.5:0/52305007 learned_addr learned my addr 172.21.15.5:0/52305007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df3fff640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece09e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df37fe640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece0a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df896b640 1 -- 172.21.15.5:0/52305007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 msgr2=0x5562ece0a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df896b640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece0a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df896b640 1 -- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 msgr2=0x5562ece09e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df896b640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece09e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df896b640 1 -- 172.21.15.5:0/52305007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ecba2f70 con 0x5562ece0ce10 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5df896b640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece0a8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5dec02c630 tx=0x7f5dec032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8e89579b7c2bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dd37fe640 1 -- 172.21.15.5:0/52305007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec051040 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dd37fe640 1 -- 172.21.15.5:0/52305007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5dec003970 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec046030 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dd37fe640 1 -- 172.21.15.5:0/52305007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5dec063050 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dfcfbe740 10 client.18000 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562eca3a170 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.114+0000 7f5dd37fe640 1 -- 172.21.15.5:0/52305007 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec046030 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dd37fe640 1 -- 172.21.15.5:0/52305007 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec054070 con 0x5562ece0ce10 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dfcfbe740 10 client.18000 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dfcfbe740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dfcfbe740 20 client.18000 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece1d260 0x5562ece1f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dfcfbe740 4 client.18000 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562ecd75920 con 0x5562ece1d260 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5df3fff640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece1d260 0x5562ece1f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5dd37fe640 10 client.18000 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.115+0000 7f5df3fff640 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece1d260 0x5562ece1f650 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78823dcafb468189 server_cookie=c6fe6dfbb7d6d9ce in_seq=0 out_seq=0 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dd37fe640 1 -- 172.21.15.5:0/52305007 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562ecd75920 con 0x5562ece1d260 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dd37fe640 10 client.18000 handle_command_reply: tid=0 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 client.18000 shutdown 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 msgr2=0x5562ece0a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece0a8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5dec02c630 tx=0x7f5dec032be0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 shutdown_connections 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece1d260 0x5562ece1f650 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece0a8c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece0a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dd37fe640 0 client.18000 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/52305007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece09e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.116+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 >> 172.21.15.5:0/52305007 conn(0x5562ecdfdfb0 msgr2=0x5562ecdffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.117+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 shutdown_connections 2024-04-22T13:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.117+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/52305007 wait complete. 2024-04-22T13:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.117+0000 7f5dfcfbe740 20 client.18000 trim_cache size 0 max 16384 2024-04-22T13:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.134+0000 7f5dfcfbe740 1 Processor -- start 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.134+0000 7f5dfcfbe740 1 -- start start 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece0c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41adbe0 con 0x5562ece0ce10 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41af720 con 0x5562ece038e0 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41ae2b0 con 0x5562ece04290 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35508/0 (socket says 172.21.15.5:35508) 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 -- 172.21.15.5:0/2433134491 learned_addr learned my addr 172.21.15.5:0/2433134491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df3fff640 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece0c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df37fe640 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 -- 172.21.15.5:0/2433134491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 msgr2=0x5562ece22a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece22a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 -- 172.21.15.5:0/2433134491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 msgr2=0x5562ece0c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece0c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 -- 172.21.15.5:0/2433134491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ece08390 con 0x5562ece0ce10 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5df896b640 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece070a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5dec02c6d0 tx=0x7f5dec032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85d48b4672f2229 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dd9ffb640 1 -- 172.21.15.5:0/2433134491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec06c070 con 0x5562ece0ce10 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dd9ffb640 1 -- 172.21.15.5:0/2433134491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec046040 con 0x5562ece0ce10 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.135+0000 7f5dd9ffb640 1 -- 172.21.15.5:0/2433134491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec039040 con 0x5562ece0ce10 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/2433134491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 msgr2=0x5562ece070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece070a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5dec02c6d0 tx=0x7f5dec032c50 comp rx=0 tx=0).stop 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/2433134491 shutdown_connections 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece070a0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece04290 0x5562ece22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/2433134491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece038e0 0x5562ece0c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/2433134491 >> 172.21.15.5:0/2433134491 conn(0x5562ecdfdfb0 msgr2=0x5562ecdfd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/2433134491 shutdown_connections 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/2433134491 wait complete. 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.136+0000 7f5dfcfbe740 1 -- start start 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 Processor -- start 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece038e0 0x5562ece81300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece81840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece7b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41af050 con 0x5562ece0ce10 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41ae980 con 0x5562ece04290 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5df41abbe0 con 0x5562ece038e0 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5dfcfbe740 10 client.0 _ll_register_callbacks cb 0x7f5de8009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece7b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece7b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35518/0 (socket says 172.21.15.5:35518) 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 -- 172.21.15.5:0/3135003906 learned_addr learned my addr 172.21.15.5:0/3135003906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df37fe640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece81840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df3fff640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece038e0 0x5562ece81300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 -- 172.21.15.5:0/3135003906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece038e0 msgr2=0x5562ece81300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece038e0 0x5562ece81300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 -- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 msgr2=0x5562ece81840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece81840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df896b640 1 -- 172.21.15.5:0/3135003906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ece7be00 con 0x5562ece0ce10 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.137+0000 7f5df3fff640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece038e0 0x5562ece81300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5df896b640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece7b710 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5dec034760 tx=0x7f5dec009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3d286c35600324 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dd37fe640 1 -- 172.21.15.5:0/3135003906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec06c070 con 0x5562ece0ce10 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dd37fe640 1 -- 172.21.15.5:0/3135003906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5dec003b70 con 0x5562ece0ce10 2024-04-22T13:33:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec046040 con 0x5562ece0ce10 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dd37fe640 1 -- 172.21.15.5:0/3135003906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5dec039040 con 0x5562ece0ce10 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 10 client.18006 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562ece88b40 con 0x5562ece0ce10 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dd37fe640 1 -- 172.21.15.5:0/3135003906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec046040 con 0x5562ece0ce10 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dd37fe640 1 -- 172.21.15.5:0/3135003906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec054040 con 0x5562ece0ce10 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 10 client.18006 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 20 client.18006 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece890d0 0x5562ece8b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 4 client.18006 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.138+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562ecd6f790 con 0x5562ece890d0 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.139+0000 7f5df3fff640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece890d0 0x5562ece8b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.139+0000 7f5df3fff640 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece890d0 0x5562ece8b4f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4b2b750efa2dc4 server_cookie=484b736401dcd7a9 in_seq=0 out_seq=0 2024-04-22T13:33:18.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.139+0000 7f5dd37fe640 10 client.18006 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dd37fe640 1 -- 172.21.15.5:0/3135003906 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562ecd6f790 con 0x5562ece890d0 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dd37fe640 10 client.18006 handle_command_reply: tid=0 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dfcfbe740 1 client.18006 shutdown 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 msgr2=0x5562ece7b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece7b710 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5dec034760 tx=0x7f5dec009880 comp rx=0 tx=0).stop 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 shutdown_connections 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.140+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5562ece890d0 0x5562ece8b4f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562ece0ce10 0x5562ece7b710 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dd37fe640 0 client.18006 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5562ece04290 0x5562ece81840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 1 --2- 172.21.15.5:0/3135003906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562ece038e0 0x5562ece81300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 >> 172.21.15.5:0/3135003906 conn(0x5562ecdfdfb0 msgr2=0x5562ece085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 shutdown_connections 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 1 -- 172.21.15.5:0/3135003906 wait complete. 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfcfbe740 20 client.18006 trim_cache size 0 max 16384 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d9462225-0716-4a03-a1c7-082b2824e021", 2024-04-22T13:33:18.143 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:33:18.144 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:33:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5ddc0a3d20 msgr2=0x7f5ddc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5ddc0a3d20 0x7f5ddc0a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5dec04a000 tx=0x7f5dec04b000 comp rx=0 tx=0).stop 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 msgr2=0x7f5df419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df419eb70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d950 tx=0x7f5de400de10 comp rx=0 tx=0).stop 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 shutdown_connections 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5df410c6d0 0x7f5df419eb70 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5ddc0a3d20 0x7f5ddc0a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df4198740 0x7f5df4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 --2- 172.21.15.5:0/3189891571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5df41031c0 0x7f5df419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 >> 172.21.15.5:0/3189891571 conn(0x7f5df4077f70 msgr2=0x7f5df40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 shutdown_connections 2024-04-22T13:33:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.141+0000 7f5dfa3ce640 1 -- 172.21.15.5:0/3189891571 wait complete. 2024-04-22T13:33:18.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:18.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:18.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 -- 172.21.15.5:0/147923211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101a00 msgr2=0x7f3108105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 --2- 172.21.15.5:0/147923211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101a00 0x7f3108105a40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:18.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 -- 172.21.15.5:0/147923211 shutdown_connections 2024-04-22T13:33:18.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 --2- 172.21.15.5:0/147923211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101a00 0x7f3108105a40 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 --2- 172.21.15.5:0/147923211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 0x7f310810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 --2- 172.21.15.5:0/147923211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3108101050 0x7f3108101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 -- 172.21.15.5:0/147923211 >> 172.21.15.5:0/147923211 conn(0x7f3108077fa0 msgr2=0x7f31080783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 -- 172.21.15.5:0/147923211 shutdown_connections 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.257+0000 7f3110655640 1 -- 172.21.15.5:0/147923211 wait complete. 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 Processor -- start 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 -- start start 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 0x7f310810e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 0x7f310810e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f310810ef00 0x7f31081a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3108112150 con 0x7f3108101050 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3108111fd0 con 0x7f3108106120 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f3110655640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31081122d0 con 0x7f310810ef00 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 0x7f310810e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 0x7f310810e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35540/0 (socket says 172.21.15.5:35540) 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 -- 172.21.15.5:0/4239448300 learned_addr learned my addr 172.21.15.5:0/4239448300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310dbf0640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 0x7f310810e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310ebf2640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f310810ef00 0x7f31081a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 -- 172.21.15.5:0/4239448300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f310810ef00 msgr2=0x7f31081a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f310810ef00 0x7f31081a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 -- 172.21.15.5:0/4239448300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 msgr2=0x7f310810e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 0x7f310810e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 -- 172.21.15.5:0/4239448300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31081a7000 con 0x7f3108101050 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310dbf0640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 0x7f310810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.258+0000 7f310e3f1640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 0x7f310810e480 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f310400d950 tx=0x7f310400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.259+0000 7f30ff7fe640 1 -- 172.21.15.5:0/4239448300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104017030 con 0x7f3108101050 2024-04-22T13:33:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.259+0000 7f30ff7fe640 1 -- 172.21.15.5:0/4239448300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3104004020 con 0x7f3108101050 2024-04-22T13:33:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.259+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31081a72f0 con 0x7f3108101050 2024-04-22T13:33:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.259+0000 7f30ff7fe640 1 -- 172.21.15.5:0/4239448300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040047a0 con 0x7f3108101050 2024-04-22T13:33:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.259+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31081a7860 con 0x7f3108101050 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.261+0000 7f30ff7fe640 1 -- 172.21.15.5:0/4239448300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3104009a30 con 0x7f3108101050 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.261+0000 7f3113248740 1 Processor -- start 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 -- start start 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 0x556829635960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x556829640e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55682959a1e0 con 0x556829635ea0 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556829537390 con 0x55682963ea80 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f3113248740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568295961a0 con 0x556829635580 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f30ff7fe640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f30f40a3dd0 0x7f30f40a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 0x556829635960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 0x556829635960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:59044/0 (socket says 172.21.15.5:59044) 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310dbf0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x556829640e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 -- 172.21.15.5:0/2400045324 learned_addr learned my addr 172.21.15.5:0/2400045324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35548/0 (socket says 172.21.15.5:35548) 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f30f40a3dd0 0x7f30f40a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 -- 172.21.15.5:0/2400045324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 msgr2=0x556829640e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x556829640e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f30ff7fe640 1 -- 172.21.15.5:0/4239448300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f31040d0210 con 0x7f3108101050 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 -- 172.21.15.5:0/2400045324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 msgr2=0x55682963e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.262+0000 7f310ebf2640 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.263+0000 7f310ebf2640 1 -- 172.21.15.5:0/2400045324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568293767d0 con 0x556829635580 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.263+0000 7f310ebf2640 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 0x556829635960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31000129f0 tx=0x7f3100012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99742ebacfb39d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.263+0000 7f310ebf2640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f30f40a3dd0 0x7f30f40a6290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f310001c000 tx=0x7f3100020000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.263+0000 7f30fcbfe640 1 -- 172.21.15.5:0/2400045324 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000120d0 con 0x556829635580 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.263+0000 7f30fcbfe640 1 -- 172.21.15.5:0/2400045324 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3100023020 con 0x556829635580 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.263+0000 7f30fcbfe640 1 -- 172.21.15.5:0/2400045324 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100003de0 con 0x556829635580 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 -- 172.21.15.5:0/2400045324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 msgr2=0x556829635960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 0x556829635960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31000129f0 tx=0x7f3100012eb0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 -- 172.21.15.5:0/2400045324 shutdown_connections 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x556829640e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 --2- 172.21.15.5:0/2400045324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635580 0x556829635960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 -- 172.21.15.5:0/2400045324 >> 172.21.15.5:0/2400045324 conn(0x55682962fcb0 msgr2=0x556829630090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 -- 172.21.15.5:0/2400045324 shutdown_connections 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 -- 172.21.15.5:0/2400045324 wait complete. 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 -- start start 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.264+0000 7f3113248740 1 Processor -- start 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635580 0x55682963bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x55682963c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568296417b0 con 0x556829635ea0 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5568295532f0 con 0x556829635580 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556829641930 con 0x55682963ea80 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 10 client.0 _ll_register_callbacks cb 0x556829560580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35554/0 (socket says 172.21.15.5:35554) 2024-04-22T13:33:18.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 -- 172.21.15.5:0/3747849466 learned_addr learned my addr 172.21.15.5:0/3747849466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310ebf2640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x55682963c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310e3f1640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635580 0x55682963bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 -- 172.21.15.5:0/3747849466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 msgr2=0x55682963c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x55682963c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 -- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635580 msgr2=0x55682963bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635580 0x55682963bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 -- 172.21.15.5:0/3747849466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568292ac3f0 con 0x556829635ea0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310ebf2640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x55682963c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f310dbf0640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963c060 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f310806b720 tx=0x7f30f802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a15710da319940 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f30f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f30f1ffb640 1 -- 172.21.15.5:0/3747849466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8005d50 con 0x556829635ea0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f30f1ffb640 1 -- 172.21.15.5:0/3747849466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f30f8005450 con 0x556829635ea0 2024-04-22T13:33:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.265+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3100023020 con 0x556829635ea0 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f30f1ffb640 1 -- 172.21.15.5:0/3747849466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30f802cc80 con 0x556829635ea0 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 10 client.18018 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55682926c0f0 con 0x556829635ea0 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f30f1ffb640 1 -- 172.21.15.5:0/3747849466 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3100023020 con 0x556829635ea0 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f30f1ffb640 1 -- 172.21.15.5:0/3747849466 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f30f8003710 con 0x556829635ea0 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 10 client.18018 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 20 client.18018 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55682964ea10 0x556829650e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 4 client.18018 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568295a7c00 con 0x55682964ea10 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.266+0000 7f310e3f1640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55682964ea10 0x556829650e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.267+0000 7f30f1ffb640 10 client.18018 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.267+0000 7f310e3f1640 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55682964ea10 0x556829650e50 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e14f8b20d66a61 server_cookie=2e07ad80bc019b28 in_seq=0 out_seq=0 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.267+0000 7f30f1ffb640 1 -- 172.21.15.5:0/3747849466 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568295a7c00 con 0x55682964ea10 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.267+0000 7f30f1ffb640 10 client.18018 handle_command_reply: tid=0 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.267+0000 7f3113248740 1 client.18018 shutdown 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 msgr2=0x55682963c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963c060 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f310806b720 tx=0x7f30f802ca90 comp rx=0 tx=0).stop 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 shutdown_connections 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55682964ea10 0x556829650e50 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635ea0 0x55682963c060 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x55682963c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 --2- 172.21.15.5:0/3747849466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635580 0x55682963bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f30f1ffb640 0 client.18018 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 >> 172.21.15.5:0/3747849466 conn(0x55682962fcb0 msgr2=0x556829631a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 shutdown_connections 2024-04-22T13:33:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 1 -- 172.21.15.5:0/3747849466 wait complete. 2024-04-22T13:33:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.268+0000 7f3113248740 20 client.18018 trim_cache size 0 max 16384 2024-04-22T13:33:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.285+0000 7f3113248740 1 Processor -- start 2024-04-22T13:33:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.285+0000 7f3113248740 1 -- start start 2024-04-22T13:33:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.285+0000 7f3113248740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x55682963e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.285+0000 7f3113248740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635ea0 0x55682964a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f3113248740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x55682963ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f3113248740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31081afda0 con 0x556829635580 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f3113248740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b18e0 con 0x55682963ea80 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f3113248740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b0470 con 0x556829635ea0 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x55682963e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x55682963e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35566/0 (socket says 172.21.15.5:35566) 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 -- 172.21.15.5:0/2637430674 learned_addr learned my addr 172.21.15.5:0/2637430674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 -- 172.21.15.5:0/2637430674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635ea0 msgr2=0x55682964a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635ea0 0x55682964a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 -- 172.21.15.5:0/2637430674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 msgr2=0x55682963ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x55682963ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 -- 172.21.15.5:0/2637430674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556829634900 con 0x556829635580 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f310e3f1640 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x55682963e250 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f310409bee0 tx=0x7f310400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b7166b37673e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f30f3fff640 1 -- 172.21.15.5:0/2637430674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040a0590 con 0x556829635580 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f30f3fff640 1 -- 172.21.15.5:0/2637430674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31040a4e50 con 0x556829635580 2024-04-22T13:33:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.286+0000 7f30f3fff640 1 -- 172.21.15.5:0/2637430674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104098850 con 0x556829635580 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- 172.21.15.5:0/2637430674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 msgr2=0x55682963e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x55682963e250 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f310409bee0 tx=0x7f310400df40 comp rx=0 tx=0).stop 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- 172.21.15.5:0/2637430674 shutdown_connections 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55682963ea80 0x55682963ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829635ea0 0x55682964a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- 172.21.15.5:0/2637430674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x55682963e250 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- 172.21.15.5:0/2637430674 >> 172.21.15.5:0/2637430674 conn(0x55682962fcb0 msgr2=0x5568296310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- 172.21.15.5:0/2637430674 shutdown_connections 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- 172.21.15.5:0/2637430674 wait complete. 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- start start 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 Processor -- start 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x556829642910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635ea0 0x556829642e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x5568296b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b1210 con 0x556829635580 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b0b40 con 0x556829635ea0 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31081adda0 con 0x55682963ea80 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f3113248740 10 client.0 _ll_register_callbacks cb 0x7f30f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x556829642910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f310e3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x556829642910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35576/0 (socket says 172.21.15.5:35576) 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f310e3f1640 1 -- 172.21.15.5:0/1992902788 learned_addr learned my addr 172.21.15.5:0/1992902788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.287+0000 7f310dbf0640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635ea0 0x556829642e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310ebf2640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x5568296b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310e3f1640 1 -- 172.21.15.5:0/1992902788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 msgr2=0x5568296b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310e3f1640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x5568296b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310e3f1640 1 -- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635ea0 msgr2=0x556829642e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310e3f1640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635ea0 0x556829642e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310e3f1640 1 -- 172.21.15.5:0/1992902788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556829643750 con 0x556829635580 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310ebf2640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x5568296b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f310e3f1640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x556829642910 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3104005820 tx=0x7f310409c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc1c4eeb5cc5e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f30f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1992902788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040cdc10 con 0x556829635580 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1992902788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f3104098850 con 0x556829635580 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31040a4e50 con 0x556829635580 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1992902788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31040a0150 con 0x556829635580 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 10 client.18024 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568296b5610 con 0x556829635580 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1992902788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31040a4e50 con 0x556829635580 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1992902788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f31040a49c0 con 0x556829635580 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 10 client.18024 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 20 client.18024 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568296b5d30 0x5568296b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 4 client.18024 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.288+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568295a1a70 con 0x5568296b5d30 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f310dbf0640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568296b5d30 0x5568296b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f310dbf0640 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568296b5d30 0x5568296b8120 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a09392076e647cec server_cookie=113d235d44c564dd in_seq=0 out_seq=0 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f30f1ffb640 10 client.18024 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1992902788 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5568295a1a70 con 0x5568296b5d30 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f30f1ffb640 10 client.18024 handle_command_reply: tid=0 2024-04-22T13:33:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 client.18024 shutdown 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 msgr2=0x556829642910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x556829642910 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3104005820 tx=0x7f310409c1e0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 shutdown_connections 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568296b5d30 0x5568296b8120 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682963ea80 0x5568296b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556829635ea0 0x556829642e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 --2- 172.21.15.5:0/1992902788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556829635580 0x556829642910 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f30f1ffb640 0 client.18024 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.289+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 >> 172.21.15.5:0/1992902788 conn(0x55682962fcb0 msgr2=0x556829631970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 shutdown_connections 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3113248740 1 -- 172.21.15.5:0/1992902788 wait complete. 2024-04-22T13:33:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3113248740 20 client.18024 trim_cache size 0 max 16384 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: "d9462225-0716-4a03-a1c7-082b2824e021": { 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d9462225-0716-4a03-a1c7-082b2824e021", 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:33:18.297 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f30f40a3dd0 msgr2=0x7f30f40a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f30f40a3dd0 0x7f30f40a6290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f310001c000 tx=0x7f3100020000 comp rx=0 tx=0).stop 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 msgr2=0x7f310810e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 0x7f310810e480 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f310400d950 tx=0x7f310400de10 comp rx=0 tx=0).stop 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 shutdown_connections 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f30f40a3dd0 0x7f30f40a6290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f310810ef00 0x7f31081a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3108106120 0x7f310810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 --2- 172.21.15.5:0/4239448300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108101050 0x7f310810e480 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 >> 172.21.15.5:0/4239448300 conn(0x7f3108077fa0 msgr2=0x7f31081072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 shutdown_connections 2024-04-22T13:33:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:18.290+0000 7f3110655640 1 -- 172.21.15.5:0/4239448300 wait complete. 2024-04-22T13:33:18.300 INFO:tasks.cephfs.filesystem:scrub status for tag:d9462225-0716-4a03-a1c7-082b2824e021 - {'path': '/', 'tag': 'd9462225-0716-4a03-a1c7-082b2824e021', 'options': 'recursive,force'} 2024-04-22T13:33:19.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:19 smithi005 ceph-mon[28006]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:33:19.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:19 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:33:19.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:19 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:19.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:19 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:33:19.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:19 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:19.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:19 smithi049 ceph-mon[31379]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:33:19.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:19 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:33:19.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:19 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:19.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:19 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:33:19.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:19 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:19.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:19 smithi107 ceph-mon[31859]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:33:19.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:19 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:33:19.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:19 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:19.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:19 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:33:19.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:19 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:20.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:20 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:33:20.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:20 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:33:20.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:20 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:33:21.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:21 smithi049 ceph-mon[31379]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:33:21.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:33:21.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:33:21.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:33:21.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:21 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:33:21.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:21 smithi107 ceph-mon[31859]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:33:21.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:33:21.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:33:21.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:33:21.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:21 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:33:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:21 smithi005 ceph-mon[28006]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:33:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:33:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:33:21.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:33:21.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:21 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:33:23.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:23 smithi049 ceph-mon[31379]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-22T13:33:23.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:23 smithi107 ceph-mon[31859]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-22T13:33:23.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:23 smithi005 ceph-mon[28006]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-22T13:33:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:25 smithi049 ceph-mon[31379]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:25 smithi107 ceph-mon[31859]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:33:25.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:25 smithi005 ceph-mon[28006]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:33:27.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:27 smithi049 ceph-mon[31379]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:33:27.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:27 smithi107 ceph-mon[31859]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:33:27.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:27 smithi005 ceph-mon[28006]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:33:29.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:29 smithi049 ceph-mon[31379]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:29.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:29 smithi107 ceph-mon[31859]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:29 smithi005 ceph-mon[28006]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:31.336 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:31 smithi107 ceph-mon[31859]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:31.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:31 smithi049 ceph-mon[31379]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:31.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:31 smithi005 ceph-mon[28006]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:33.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:33 smithi049 ceph-mon[31379]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:33.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:33 smithi107 ceph-mon[31859]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:33.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:33 smithi005 ceph-mon[28006]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:35.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:35 smithi049 ceph-mon[31379]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:35.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:35 smithi107 ceph-mon[31859]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:35.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:35 smithi005 ceph-mon[28006]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:37.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:37 smithi049 ceph-mon[31379]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:37.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:37 smithi107 ceph-mon[31859]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:37 smithi005 ceph-mon[28006]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:39.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:39 smithi005 ceph-mon[28006]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:39.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:39 smithi049 ceph-mon[31379]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:39.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:39 smithi107 ceph-mon[31859]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:41.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:41 smithi049 ceph-mon[31379]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:41.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:41 smithi107 ceph-mon[31859]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:41.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:41 smithi005 ceph-mon[28006]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:43.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:43 smithi049 ceph-mon[31379]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:43.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:43 smithi107 ceph-mon[31859]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:43 smithi005 ceph-mon[28006]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:45.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:45 smithi049 ceph-mon[31379]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:45.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:45 smithi107 ceph-mon[31859]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:45.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:45 smithi005 ceph-mon[28006]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:47.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:47 smithi049 ceph-mon[31379]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:47.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:47 smithi107 ceph-mon[31859]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:47.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:47 smithi005 ceph-mon[28006]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:48.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:48.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:48.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 -- 172.21.15.5:0/1956644358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 msgr2=0x7fbc101014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 --2- 172.21.15.5:0/1956644358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc101014a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009900 tx=0x7fbc0002bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 -- 172.21.15.5:0/1956644358 shutdown_connections 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 --2- 172.21.15.5:0/1956644358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc10106190 0x7fbc1010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 --2- 172.21.15.5:0/1956644358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc10101a70 0x7fbc10105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 --2- 172.21.15.5:0/1956644358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc101014a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 -- 172.21.15.5:0/1956644358 >> 172.21.15.5:0/1956644358 conn(0x7fbc10077fa0 msgr2=0x7fbc100783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 -- 172.21.15.5:0/1956644358 shutdown_connections 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.424+0000 7fbc16f59640 1 -- 172.21.15.5:0/1956644358 wait complete. 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 Processor -- start 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 -- start start 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc10198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc10106190 0x7fbc10198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc1019cff0 0x7fbc1019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112210 con 0x7fbc101010c0 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112090 con 0x7fbc1019cff0 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc16f59640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112390 con 0x7fbc10106190 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc14cf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc10198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc14cf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc10198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58702/0 (socket says 172.21.15.5:58702) 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc14cf5640 1 -- 172.21.15.5:0/603159392 learned_addr learned my addr 172.21.15.5:0/603159392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc154f6640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc1019cff0 0x7fbc1019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc10106190 0x7fbc10198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc14cf5640 1 -- 172.21.15.5:0/603159392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc10106190 msgr2=0x7fbc10198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc10106190 0x7fbc10198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.425+0000 7fbc14cf5640 1 -- 172.21.15.5:0/603159392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc1019cff0 msgr2=0x7fbc1019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc1019cff0 0x7fbc1019d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc14cf5640 1 -- 172.21.15.5:0/603159392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1019dae0 con 0x7fbc101010c0 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc154f6640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc1019cff0 0x7fbc1019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:48.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc10198730 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005940 tx=0x7fbc0002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc0dffb640 1 -- 172.21.15.5:0/603159392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00043020 con 0x7fbc101010c0 2024-04-22T13:33:48.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc0dffb640 1 -- 172.21.15.5:0/603159392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0002cbe0 con 0x7fbc101010c0 2024-04-22T13:33:48.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1019dd70 con 0x7fbc101010c0 2024-04-22T13:33:48.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc0dffb640 1 -- 172.21.15.5:0/603159392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00003bf0 con 0x7fbc101010c0 2024-04-22T13:33:48.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.426+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101a5590 con 0x7fbc101010c0 2024-04-22T13:33:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc0dffb640 1 -- 172.21.15.5:0/603159392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbc00046020 con 0x7fbc101010c0 2024-04-22T13:33:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 Processor -- start 2024-04-22T13:33:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 -- start start 2024-04-22T13:33:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a33540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56306498f1e0 con 0x563064a33a80 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56306492c390 con 0x563064a2aea0 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.429+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56306498b1a0 con 0x563064a2a580 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0dffb640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbbfc0a3d20 0x7fbbfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc14cf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a33540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a35e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58704/0 (socket says 172.21.15.5:58704) 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 -- 172.21.15.5:0/3742942524 learned_addr learned my addr 172.21.15.5:0/3742942524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc154f6640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbbfc0a3d20 0x7fbbfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc154f6640 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0dffb640 1 -- 172.21.15.5:0/603159392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbc000f2a70 con 0x7fbc101010c0 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 -- 172.21.15.5:0/3742942524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 msgr2=0x563064a2a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a2a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 -- 172.21.15.5:0/3742942524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 msgr2=0x563064a33540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a33540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 -- 172.21.15.5:0/3742942524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306476b7d0 con 0x563064a33a80 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a35e70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800bc50 tx=0x7fbbf800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af78257b3407bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc154f6640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbbfc0a3d20 0x7fbbfc0a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc08009540 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc06ffd640 1 -- 172.21.15.5:0/3742942524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8015070 con 0x563064a33a80 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.430+0000 7fbc06ffd640 1 -- 172.21.15.5:0/3742942524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf801a910 con 0x563064a33a80 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc06ffd640 1 -- 172.21.15.5:0/3742942524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8011940 con 0x563064a33a80 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3742942524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 msgr2=0x563064a35e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a35e70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800bc50 tx=0x7fbbf800f730 comp rx=0 tx=0).stop 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3742942524 shutdown_connections 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a35e70 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a33540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3742942524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a2a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3742942524 >> 172.21.15.5:0/3742942524 conn(0x563064a24cb0 msgr2=0x563064a25090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3742942524 shutdown_connections 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.431+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3742942524 wait complete. 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 -- start start 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 Processor -- start 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a30ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a31010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a31550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563064a36870 con 0x563064a33a80 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5630649482f0 con 0x563064a2aea0 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563064a369f0 con 0x563064a2a580 2024-04-22T13:33:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc19c2c740 10 client.0 _ll_register_callbacks cb 0x563064955580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc154f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a31550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc154f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a31550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58714/0 (socket says 172.21.15.5:58714) 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc154f6640 1 -- 172.21.15.5:0/1583225932 learned_addr learned my addr 172.21.15.5:0/1583225932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.432+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a30ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a31010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc154f6640 1 -- 172.21.15.5:0/1583225932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 msgr2=0x563064a30ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc154f6640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a30ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc154f6640 1 -- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 msgr2=0x563064a31010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc154f6640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a31010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc154f6640 1 -- 172.21.15.5:0/1583225932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630646a13f0 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a31010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc154f6640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a31550 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbc08010b10 tx=0x7fbc08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8caef83e33d269c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbbeffff640 1 -- 172.21.15.5:0/1583225932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc080115e0 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbbeffff640 1 -- 172.21.15.5:0/1583225932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbc08011780 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf801a910 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbbeffff640 1 -- 172.21.15.5:0/1583225932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc08011980 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc19c2c740 10 client.18036 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630646610f0 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.433+0000 7fbbeffff640 1 -- 172.21.15.5:0/1583225932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf801a910 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbbeffff640 1 -- 172.21.15.5:0/1583225932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc0801fc30 con 0x563064a33a80 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc19c2c740 10 client.18036 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc19c2c740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:48.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc19c2c740 20 client.18036 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064a43a90 0x563064a45f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc19c2c740 4 client.18036 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56306499cc00 con 0x563064a43a90 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064a43a90 0x563064a45f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064a43a90 0x563064a45f10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1299451dceaca967 server_cookie=5ceb9e3f6e1af84 in_seq=0 out_seq=0 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.434+0000 7fbbeffff640 10 client.18036 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbbeffff640 1 -- 172.21.15.5:0/1583225932 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56306499cc00 con 0x563064a43a90 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbbeffff640 10 client.18036 handle_command_reply: tid=0 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 client.18036 shutdown 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 msgr2=0x563064a31550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a31550 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbc08010b10 tx=0x7fbc08004a40 comp rx=0 tx=0).stop 2024-04-22T13:33:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 shutdown_connections 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064a43a90 0x563064a45f10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064a31550 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2aea0 0x563064a31010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbbeffff640 0 client.18036 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1583225932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2a580 0x563064a30ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 >> 172.21.15.5:0/1583225932 conn(0x563064a24cb0 msgr2=0x563064a26a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 shutdown_connections 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1583225932 wait complete. 2024-04-22T13:33:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.435+0000 7fbc19c2c740 20 client.18036 trim_cache size 0 max 16384 2024-04-22T13:33:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 Processor -- start 2024-04-22T13:33:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 -- start start 2024-04-22T13:33:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 0x563064a33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a33a80 0x563064a3f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101ada50 con 0x563064a2a580 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101af590 con 0x563064a33a80 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101ae120 con 0x563064a2aea0 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.453+0000 7fbc14cf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 0x563064a33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 0x563064a33200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58716/0 (socket says 172.21.15.5:58716) 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 -- 172.21.15.5:0/1933462308 learned_addr learned my addr 172.21.15.5:0/1933462308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc154f6640 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a33a80 0x563064a3f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 -- 172.21.15.5:0/1933462308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 msgr2=0x563064a2fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 -- 172.21.15.5:0/1933462308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a33a80 msgr2=0x563064a3f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a33a80 0x563064a3f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 -- 172.21.15.5:0/1933462308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563064a24ae0 con 0x563064a2a580 2024-04-22T13:33:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc154f6640 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a33a80 0x563064a3f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 0x563064a33200 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbc000bea80 tx=0x7fbc00090040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85637ee778b579c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc067fc640 1 -- 172.21.15.5:0/1933462308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000f2e60 con 0x563064a2a580 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc067fc640 1 -- 172.21.15.5:0/1933462308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0003c350 con 0x563064a2a580 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.454+0000 7fbc067fc640 1 -- 172.21.15.5:0/1933462308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00003d90 con 0x563064a2a580 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1933462308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 msgr2=0x563064a33200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 0x563064a33200 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbc000bea80 tx=0x7fbc00090040 comp rx=0 tx=0).stop 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1933462308 shutdown_connections 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a33a80 0x563064a3f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/1933462308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a2a580 0x563064a33200 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1933462308 >> 172.21.15.5:0/1933462308 conn(0x563064a24cb0 msgr2=0x563064a23d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1933462308 shutdown_connections 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 -- 172.21.15.5:0/1933462308 wait complete. 2024-04-22T13:33:48.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 -- start start 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.455+0000 7fbc19c2c740 1 Processor -- start 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2a580 0x563064a2db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064aa9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101aeec0 con 0x563064a33a80 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101ae7f0 con 0x563064a2a580 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101aba30 con 0x563064a2aea0 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 10 client.0 _ll_register_callbacks cb 0x7fbc08009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064aa9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064aa9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58726/0 (socket says 172.21.15.5:58726) 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 -- 172.21.15.5:0/3553356672 learned_addr learned my addr 172.21.15.5:0/3553356672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2a580 0x563064a2db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 -- 172.21.15.5:0/3553356672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 msgr2=0x563064a2e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 -- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2a580 msgr2=0x563064a2db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2a580 0x563064a2db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 -- 172.21.15.5:0/3553356672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563064aa9590 con 0x563064a33a80 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc0ffff640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc154f6640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064aa9050 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbc08010ae0 tx=0x7fbc080124d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b2f21d83a2e123 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbbeffff640 1 -- 172.21.15.5:0/3553356672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08011b20 con 0x563064a33a80 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbbeffff640 1 -- 172.21.15.5:0/3553356672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fbc08011cc0 con 0x563064a33a80 2024-04-22T13:33:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc0003c350 con 0x563064a33a80 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.456+0000 7fbbeffff640 1 -- 172.21.15.5:0/3553356672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc08004ab0 con 0x563064a33a80 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 10 client.18042 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563064aaa6e0 con 0x563064a33a80 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbbeffff640 1 -- 172.21.15.5:0/3553356672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0003c350 con 0x563064a33a80 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbbeffff640 1 -- 172.21.15.5:0/3553356672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc08005150 con 0x563064a33a80 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 10 client.18042 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 20 client.18042 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064aaae80 0x563064aad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 4 client.18042 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563064996a70 con 0x563064aaae80 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.457+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064aaae80 0x563064aad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc14cf5640 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064aaae80 0x563064aad2a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f522449ce507f9f7 server_cookie=83b151bcce17689d in_seq=0 out_seq=0 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbbeffff640 10 client.18042 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbbeffff640 1 -- 172.21.15.5:0/3553356672 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563064996a70 con 0x563064aaae80 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbbeffff640 10 client.18042 handle_command_reply: tid=0 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 client.18042 shutdown 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 msgr2=0x563064aa9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064aa9050 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbc08010ae0 tx=0x7fbc080124d0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 shutdown_connections 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x563064aaae80 0x563064aad2a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563064a33a80 0x563064aa9050 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563064a2aea0 0x563064a2e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbbeffff640 0 client.18042 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 --2- 172.21.15.5:0/3553356672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563064a2a580 0x563064a2db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 >> 172.21.15.5:0/3553356672 conn(0x563064a24cb0 msgr2=0x563064a234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 shutdown_connections 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 1 -- 172.21.15.5:0/3553356672 wait complete. 2024-04-22T13:33:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.458+0000 7fbc19c2c740 20 client.18042 trim_cache size 0 max 16384 2024-04-22T13:33:48.461 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:33:48.461 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:33:48.461 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:33:48.461 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbbfc0a3d20 msgr2=0x7fbbfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbbfc0a3d20 0x7fbbfc0a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc08009540 comp rx=0 tx=0).stop 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 msgr2=0x7fbc10198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc10198730 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005940 tx=0x7fbc0002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 shutdown_connections 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbbfc0a3d20 0x7fbbfc0a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc1019cff0 0x7fbc1019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc10106190 0x7fbc10198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 --2- 172.21.15.5:0/603159392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc101010c0 0x7fbc10198730 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 >> 172.21.15.5:0/603159392 conn(0x7fbc10077fa0 msgr2=0x7fbc100ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 shutdown_connections 2024-04-22T13:33:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.459+0000 7fbc16f59640 1 -- 172.21.15.5:0/603159392 wait complete. 2024-04-22T13:33:48.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:33:48.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:33:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 -- 172.21.15.5:0/1559790821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 msgr2=0x7f5f1010ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 --2- 172.21.15.5:0/1559790821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f1010ee10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009ca0 tx=0x7f5f0802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 -- 172.21.15.5:0/1559790821 shutdown_connections 2024-04-22T13:33:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 --2- 172.21.15.5:0/1559790821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f1010ee10 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 --2- 172.21.15.5:0/1559790821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f10102b20 0x7f5f10106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 --2- 172.21.15.5:0/1559790821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 0x7f5f10102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 -- 172.21.15.5:0/1559790821 >> 172.21.15.5:0/1559790821 conn(0x7f5f100fd9d0 msgr2=0x7f5f100ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 -- 172.21.15.5:0/1559790821 shutdown_connections 2024-04-22T13:33:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.572+0000 7f5f18232640 1 -- 172.21.15.5:0/1559790821 wait complete. 2024-04-22T13:33:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 Processor -- start 2024-04-22T13:33:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 -- start start 2024-04-22T13:33:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 0x7f5f101986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f10198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f1019cf90 0x7f5f1019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10112190 con 0x7f5f10107240 2024-04-22T13:33:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10112010 con 0x7f5f10102170 2024-04-22T13:33:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f18232640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10112310 con 0x7f5f1019cf90 2024-04-22T13:33:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f157cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f10198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f157cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f10198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58740/0 (socket says 172.21.15.5:58740) 2024-04-22T13:33:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f157cd640 1 -- 172.21.15.5:0/1126313445 learned_addr learned my addr 172.21.15.5:0/1126313445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.573+0000 7f5f167cf640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f1019cf90 0x7f5f1019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f15fce640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 0x7f5f101986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f157cd640 1 -- 172.21.15.5:0/1126313445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f1019cf90 msgr2=0x7f5f1019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f157cd640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f1019cf90 0x7f5f1019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f157cd640 1 -- 172.21.15.5:0/1126313445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 msgr2=0x7f5f101986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f157cd640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 0x7f5f101986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f157cd640 1 -- 172.21.15.5:0/1126313445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1019da80 con 0x7f5f10107240 2024-04-22T13:33:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f15fce640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 0x7f5f101986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f157cd640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f10198c10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000e9c0 tx=0x7f5f0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000cd80 con 0x7f5f10107240 2024-04-22T13:33:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1019dd70 con 0x7f5f10107240 2024-04-22T13:33:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f000054d0 con 0x7f5f10107240 2024-04-22T13:33:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00010430 con 0x7f5f10107240 2024-04-22T13:33:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.574+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f101a5560 con 0x7f5f10107240 2024-04-22T13:33:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.575+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f10069250 con 0x7f5f10107240 2024-04-22T13:33:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.576+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5f00004fd0 con 0x7f5f10107240 2024-04-22T13:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.577+0000 7f5f06ffd640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5efc0a3d20 0x7f5efc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.577+0000 7f5f15fce640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5efc0a3d20 0x7f5efc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.577+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f5f000d0ce0 con 0x7f5f10107240 2024-04-22T13:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.577+0000 7f5f15fce640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5efc0a3d20 0x7f5efc0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009b40 tx=0x7f5f0c005e50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.581+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f0009d350 con 0x7f5f10107240 2024-04-22T13:33:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.807+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f10102b20 con 0x7f5f10107240 2024-04-22T13:33:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.808+0000 7f5f06ffd640 1 -- 172.21.15.5:0/1126313445 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f5f0009cc50 con 0x7f5f10107240 2024-04-22T13:33:48.804 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:33:48.804 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:33:48.804 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:33:48.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5efc0a3d20 msgr2=0x7f5efc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5efc0a3d20 0x7f5efc0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009b40 tx=0x7f5f0c005e50 comp rx=0 tx=0).stop 2024-04-22T13:33:48.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 msgr2=0x7f5f10198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f10198c10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000e9c0 tx=0x7f5f0000ee80 comp rx=0 tx=0).stop 2024-04-22T13:33:48.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 shutdown_connections 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f10107240 0x7f5f10198c10 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5efc0a3d20 0x7f5efc0a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f1019cf90 0x7f5f1019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 --2- 172.21.15.5:0/1126313445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f10102170 0x7f5f101986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 >> 172.21.15.5:0/1126313445 conn(0x7f5f100fd9d0 msgr2=0x7f5f100ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 shutdown_connections 2024-04-22T13:33:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.810+0000 7f5f18232640 1 -- 172.21.15.5:0/1126313445 wait complete. 2024-04-22T13:33:48.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:33:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/1158629704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 msgr2=0x7f8bb8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/1158629704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb8101470 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009900 tx=0x7f8ba802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/1158629704 shutdown_connections 2024-04-22T13:33:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/1158629704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb8106160 0x7f8bb810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/1158629704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8101a40 0x7f8bb8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/1158629704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb8101470 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/1158629704 >> 172.21.15.5:0/1158629704 conn(0x7f8bb8077f40 msgr2=0x7f8bb8078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/1158629704 shutdown_connections 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.926+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/1158629704 wait complete. 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 Processor -- start 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 -- start start 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb81995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8106160 0x7f8bb8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb819a0c0 0x7f8bb81a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81110d0 con 0x7f8bb8101090 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8110f50 con 0x7f8bb8106160 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bbd2d2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8111250 con 0x7f8bb819a0c0 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb81995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb81995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40572/0 (socket says 172.21.15.5:40572) 2024-04-22T13:33:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.927+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2662026651 learned_addr learned my addr 172.21.15.5:0/2662026651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8106160 0x7f8bb8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb819a0c0 0x7f8bb81a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2662026651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb819a0c0 msgr2=0x7f8bb81a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb819a0c0 0x7f8bb81a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2662026651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8106160 msgr2=0x7f8bb8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8106160 0x7f8bb8199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2662026651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81a4e40 con 0x7f8bb8101090 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb819a0c0 0x7f8bb81a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8106160 0x7f8bb8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb81995f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005940 tx=0x7f8ba802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8b9ffff640 1 -- 172.21.15.5:0/2662026651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8043020 con 0x7f8bb8101090 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81a50d0 con 0x7f8bb8101090 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8b9ffff640 1 -- 172.21.15.5:0/2662026651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba802cbe0 con 0x7f8bb8101090 2024-04-22T13:33:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8b9ffff640 1 -- 172.21.15.5:0/2662026651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8003bf0 con 0x7f8bb8101090 2024-04-22T13:33:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.928+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81a5670 con 0x7f8bb8101090 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.930+0000 7f8b9ffff640 1 -- 172.21.15.5:0/2662026651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8ba8046020 con 0x7f8bb8101090 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 Processor -- start 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 -- start start 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfd68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd6e80 0x55f50bfdf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfdfa00 0x55f50bfe1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f50bf3b1f0 con 0x55f50bfd64d0 2024-04-22T13:33:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f50bed84b0 con 0x55f50bfdfa00 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f50bf371b0 con 0x55f50bfd6e80 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfd68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfd68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40578/0 (socket says 172.21.15.5:40578) 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bb77fe640 1 -- 172.21.15.5:0/1198800395 learned_addr learned my addr 172.21.15.5:0/1198800395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8b9ffff640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8ba00a3d20 0x7f8ba00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd6e80 0x55f50bfdf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.931+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8ba00a3d20 0x7f8ba00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8b9ffff640 1 -- 172.21.15.5:0/2662026651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8ba80f2a50 con 0x7f8bb8101090 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb77fe640 1 -- 172.21.15.5:0/1198800395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd6e80 msgr2=0x55f50bfdf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfdfa00 0x55f50bfe1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd6e80 0x55f50bfdf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb77fe640 1 -- 172.21.15.5:0/1198800395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfdfa00 msgr2=0x55f50bfe1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfdfa00 0x55f50bfe1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb77fe640 1 -- 172.21.15.5:0/1198800395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50beca1b0 con 0x55f50bfd64d0 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfdfa00 0x55f50bfe1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfd68b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0013910 tx=0x7f8bb0013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb4556a73efe23 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8b9d7fa640 1 -- 172.21.15.5:0/1198800395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0004b20 con 0x55f50bfd64d0 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8b9d7fa640 1 -- 172.21.15.5:0/1198800395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0004cc0 con 0x55f50bfd64d0 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.932+0000 7f8b9d7fa640 1 -- 172.21.15.5:0/1198800395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0005750 con 0x55f50bfd64d0 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- 172.21.15.5:0/1198800395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 msgr2=0x55f50bfd68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8ba00a3d20 0x7f8ba00a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0020000 tx=0x7f8bb001f000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfd68b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0013910 tx=0x7f8bb0013dd0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- 172.21.15.5:0/1198800395 shutdown_connections 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfdfa00 0x55f50bfe1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd6e80 0x55f50bfdf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/1198800395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfd68b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- 172.21.15.5:0/1198800395 >> 172.21.15.5:0/1198800395 conn(0x55f50bfd0ba0 msgr2=0x55f50bfd0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- 172.21.15.5:0/1198800395 shutdown_connections 2024-04-22T13:33:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- 172.21.15.5:0/1198800395 wait complete. 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- start start 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 Processor -- start 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfdf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfdcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f50bfe2970 con 0x55f50bfd64d0 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f50bef4500 con 0x55f50bfd6e80 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f50bfe2af0 con 0x55f50bfdfa00 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bbffa2740 10 client.0 _ll_register_callbacks cb 0x55f50bec2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfdf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfdf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40588/0 (socket says 172.21.15.5:40588) 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.933+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2577619164 learned_addr learned my addr 172.21.15.5:0/2577619164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 msgr2=0x55f50bfdcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfdcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2577619164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 msgr2=0x55f50bfdc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2577619164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50bc4d3f0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfdf310 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80baf30 tx=0x7f8ba808e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19106d5926b4c3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8b9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2577619164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80bf1d0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2577619164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8ba80bcdd0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb0004cc0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2577619164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ba803c350 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bbffa2740 10 client.18060 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f50bd177d0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2577619164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0004cc0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.934+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2577619164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8ba80f03f0 con 0x55f50bfd64d0 2024-04-22T13:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bbffa2740 10 client.18060 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bbffa2740 10 client.18060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bbffa2740 20 client.18060 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50bfefb60 0x55f50bff1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bbffa2740 4 client.18060 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50beda930 con 0x55f50bfefb60 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50bfefb60 0x55f50bff1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50bfefb60 0x55f50bff1ff0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc42756ead513182 server_cookie=8c884af8ddc572ac in_seq=0 out_seq=0 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.935+0000 7f8b9aff5640 10 client.18060 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2577619164 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f50beda930 con 0x55f50bfefb60 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8b9aff5640 10 client.18060 handle_command_reply: tid=0 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 client.18060 shutdown 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 msgr2=0x55f50bfdf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfdf310 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80baf30 tx=0x7f8ba808e3b0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 shutdown_connections 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfdcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50bfefb60 0x55f50bff1ff0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2577619164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfdf310 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8b9aff5640 0 client.18060 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 >> 172.21.15.5:0/2577619164 conn(0x55f50bfd0ba0 msgr2=0x55f50bfd28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 shutdown_connections 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2577619164 wait complete. 2024-04-22T13:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.936+0000 7f8bbffa2740 20 client.18060 trim_cache size 0 max 16384 2024-04-22T13:33:48.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.949+0000 7f8bbffa2740 1 Processor -- start 2024-04-22T13:33:48.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 -- start start 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd64d0 0x55f50bfeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 0x55f50bfdf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81adb50 con 0x55f50bfdfa00 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81af690 con 0x55f50bfd6e80 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ae220 con 0x55f50bfd64d0 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 0x55f50bfdf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 0x55f50bfdf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40594/0 (socket says 172.21.15.5:40594) 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 -- 172.21.15.5:0/3481231933 learned_addr learned my addr 172.21.15.5:0/3481231933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd64d0 0x55f50bfeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 -- 172.21.15.5:0/3481231933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd64d0 msgr2=0x55f50bfeb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd64d0 0x55f50bfeb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 -- 172.21.15.5:0/3481231933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 msgr2=0x55f50bfdedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 -- 172.21.15.5:0/3481231933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50bff2670 con 0x55f50bfdfa00 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 0x55f50bfdf310 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009950 tx=0x7f8bb00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a03d47ddada3f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8b9cff9640 1 -- 172.21.15.5:0/3481231933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0005a20 con 0x55f50bfdfa00 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.950+0000 7f8b9cff9640 1 -- 172.21.15.5:0/3481231933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb00134e0 con 0x55f50bfdfa00 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8b9cff9640 1 -- 172.21.15.5:0/3481231933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0026ae0 con 0x55f50bfdfa00 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 -- 172.21.15.5:0/3481231933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 msgr2=0x55f50bfdf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 0x55f50bfdf310 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009950 tx=0x7f8bb00096c0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 -- 172.21.15.5:0/3481231933 shutdown_connections 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfdfa00 0x55f50bfdf310 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfdedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/3481231933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfd64d0 0x55f50bfeb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 -- 172.21.15.5:0/3481231933 >> 172.21.15.5:0/3481231933 conn(0x55f50bfd0ba0 msgr2=0x55f50bfcfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 -- 172.21.15.5:0/3481231933 shutdown_connections 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 -- 172.21.15.5:0/3481231933 wait complete. 2024-04-22T13:33:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 -- start start 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.951+0000 7f8bbffa2740 1 Processor -- start 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfe38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfe3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfe4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81aefc0 con 0x55f50bfd64d0 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ae8f0 con 0x55f50bfd6e80 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81abb30 con 0x55f50bfdfa00 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 10 client.0 _ll_register_callbacks cb 0x7f8ba8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfe38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfe38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40600/0 (socket says 172.21.15.5:40600) 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2749677090 learned_addr learned my addr 172.21.15.5:0/2749677090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfe3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfe4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 msgr2=0x55f50bfe4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfe4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2749677090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 msgr2=0x55f50bfe3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfe3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2749677090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50c059680 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfe3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfe38a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80df2c0 tx=0x7f8ba8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8d01d206b41006 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8b9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2749677090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80be430 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2749677090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f8ba8091140 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb00134e0 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.952+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2749677090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ba80bf1d0 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 10 client.18066 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f50c05a960 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2749677090 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb00134e0 con 0x55f50bfd64d0 2024-04-22T13:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2749677090 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8ba80ba820 con 0x55f50bfd64d0 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 10 client.18066 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 10 client.18066 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 20 client.18066 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50c05af20 0x55f50c05d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 4 client.18066 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f50bf48c10 con 0x55f50c05af20 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50c05af20 0x55f50c05d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8b9aff5640 10 client.18066 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.953+0000 7f8bb67fc640 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50c05af20 0x55f50c05d340 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42da1fc57150491d server_cookie=c204112fa9fb9dbb in_seq=0 out_seq=0 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8b9aff5640 1 -- 172.21.15.5:0/2749677090 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f50bf48c10 con 0x55f50c05af20 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8b9aff5640 10 client.18066 handle_command_reply: tid=0 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 client.18066 shutdown 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 msgr2=0x55f50bfe38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfe38a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80df2c0 tx=0x7f8ba8000bc0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 shutdown_connections 2024-04-22T13:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f50bfdfa00 0x55f50bfe4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f50bfd6e80 0x55f50bfe3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55f50c05af20 0x55f50c05d340 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 --2- 172.21.15.5:0/2749677090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f50bfd64d0 0x55f50bfe38a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8b9aff5640 0 client.18066 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 >> 172.21.15.5:0/2749677090 conn(0x55f50bfd0ba0 msgr2=0x55f50bfd1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 shutdown_connections 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 1 -- 172.21.15.5:0/2749677090 wait complete. 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbffa2740 20 client.18066 trim_cache size 0 max 16384 2024-04-22T13:33:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8ba00a3d20 msgr2=0x7f8ba00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8ba00a3d20 0x7f8ba00a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0020000 tx=0x7f8bb001f000 comp rx=0 tx=0).stop 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 msgr2=0x7f8bb81995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb81995f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005940 tx=0x7f8ba802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 shutdown_connections 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8ba00a3d20 0x7f8ba00a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb819a0c0 0x7f8bb81a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bb8106160 0x7f8bb8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 --2- 172.21.15.5:0/2662026651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb8101090 0x7f8bb81995f0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.954+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 >> 172.21.15.5:0/2662026651 conn(0x7f8bb8077f40 msgr2=0x7f8bb80ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.955+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 shutdown_connections 2024-04-22T13:33:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:48.955+0000 7f8bbd2d2640 1 -- 172.21.15.5:0/2662026651 wait complete. 2024-04-22T13:33:48.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:33:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 -- 172.21.15.5:0/2016556610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c10c6d0 msgr2=0x7fc20c10eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 --2- 172.21.15.5:0/2016556610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c10c6d0 0x7fc20c10eb60 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc20402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 -- 172.21.15.5:0/2016556610 shutdown_connections 2024-04-22T13:33:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 --2- 172.21.15.5:0/2016556610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c10c6d0 0x7fc20c10eb60 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 --2- 172.21.15.5:0/2016556610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c103b70 0x7fc20c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 --2- 172.21.15.5:0/2016556610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc20c1031c0 0x7fc20c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 -- 172.21.15.5:0/2016556610 >> 172.21.15.5:0/2016556610 conn(0x7fc20c077f70 msgr2=0x7fc20c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 -- 172.21.15.5:0/2016556610 shutdown_connections 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.069+0000 7fc2126df640 1 -- 172.21.15.5:0/2016556610 wait complete. 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 Processor -- start 2024-04-22T13:33:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 -- start start 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c1031c0 0x7fc20c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc20c10c6d0 0x7fc20c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 0x7fc20c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112380 con 0x7fc20c198740 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112200 con 0x7fc20c10c6d0 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc2126df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112500 con 0x7fc20c1031c0 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.070+0000 7fc210c7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 0x7fc20c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 0x7fc20c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40606/0 (socket says 172.21.15.5:40606) 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 -- 172.21.15.5:0/3724459730 learned_addr learned my addr 172.21.15.5:0/3724459730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc20c10c6d0 0x7fc20c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc20bfff640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c1031c0 0x7fc20c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 -- 172.21.15.5:0/3724459730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c1031c0 msgr2=0x7fc20c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c1031c0 0x7fc20c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 -- 172.21.15.5:0/3724459730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc20c10c6d0 msgr2=0x7fc20c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc20c10c6d0 0x7fc20c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 -- 172.21.15.5:0/3724459730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c199340 con 0x7fc20c198740 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc20bfff640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c1031c0 0x7fc20c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc210c7c640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 0x7fc20c198bc0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc204005e30 tx=0x7fc204034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc2097fa640 1 -- 172.21.15.5:0/3724459730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204039040 con 0x7fc20c198740 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc2097fa640 1 -- 172.21.15.5:0/3724459730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc204044070 con 0x7fc20c198740 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c1995d0 con 0x7fc20c198740 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc2097fa640 1 -- 172.21.15.5:0/3724459730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204003e60 con 0x7fc20c198740 2024-04-22T13:33:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.071+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c1a56c0 con 0x7fc20c198740 2024-04-22T13:33:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2097fa640 1 -- 172.21.15.5:0/3724459730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc2040324c0 con 0x7fc20c198740 2024-04-22T13:33:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 Processor -- start 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 -- start start 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56414e5501f0 con 0x56414e5f4a00 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56414e4ed4b0 con 0x56414e5ebe80 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2153a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56414e54c1b0 con 0x56414e5eb4d0 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc2097fa640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1f80a3d20 0x7fc1f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40614/0 (socket says 172.21.15.5:40614) 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 -- 172.21.15.5:0/458013097 learned_addr learned my addr 172.21.15.5:0/458013097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20bfff640 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc210c7c640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1f80a3d20 0x7fc1f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 -- 172.21.15.5:0/458013097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 msgr2=0x56414e5eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 -- 172.21.15.5:0/458013097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 msgr2=0x56414e5f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.074+0000 7fc20b7fe640 1 -- 172.21.15.5:0/458013097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414e4df1b0 con 0x56414e5f4a00 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2097fa640 1 -- 172.21.15.5:0/3724459730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc2040f2a50 con 0x7fc20c198740 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f6e50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0113b0 tx=0x7fc1fc00d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f435d624a42860af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2027fc640 1 -- 172.21.15.5:0/458013097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc019020 con 0x56414e5f4a00 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2027fc640 1 -- 172.21.15.5:0/458013097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc0035f0 con 0x56414e5f4a00 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc210c7c640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1f80a3d20 0x7fc1f80a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc2040bb500 tx=0x7fc2040c1040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2027fc640 1 -- 172.21.15.5:0/458013097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00f630 con 0x56414e5f4a00 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2153a8740 1 -- 172.21.15.5:0/458013097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 msgr2=0x56414e5f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2153a8740 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f6e50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0113b0 tx=0x7fc1fc00d590 comp rx=0 tx=0).stop 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.075+0000 7fc2153a8740 1 -- 172.21.15.5:0/458013097 shutdown_connections 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f6e50 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 --2- 172.21.15.5:0/458013097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- 172.21.15.5:0/458013097 >> 172.21.15.5:0/458013097 conn(0x56414e5e5ba0 msgr2=0x56414e5e5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- 172.21.15.5:0/458013097 shutdown_connections 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- 172.21.15.5:0/458013097 wait complete. 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- start start 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 Processor -- start 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56414e5f7970 con 0x56414e5f4a00 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56414e509500 con 0x56414e5ebe80 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56414e5f7af0 con 0x56414e5eb4d0 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc2153a8740 10 client.0 _ll_register_callbacks cb 0x56414e4d7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc20b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc20b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40628/0 (socket says 172.21.15.5:40628) 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.076+0000 7fc20b7fe640 1 -- 172.21.15.5:0/2517230479 learned_addr learned my addr 172.21.15.5:0/2517230479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20bfff640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 -- 172.21.15.5:0/2517230479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 msgr2=0x56414e5f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 -- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 msgr2=0x56414e5f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 -- 172.21.15.5:0/2517230479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414e2623f0 con 0x56414e5f4a00 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f1ce0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc001eb0 tx=0x7fc1fc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e3ef916d7e81e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc1eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/2517230479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc019030 con 0x56414e5f4a00 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/2517230479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc1fc0041a0 con 0x56414e5f4a00 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1fc0035f0 con 0x56414e5f4a00 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/2517230479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1fc0043a0 con 0x56414e5f4a00 2024-04-22T13:33:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc2153a8740 10 client.18078 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56414e32c7d0 con 0x56414e5f4a00 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.077+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/2517230479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc0035f0 con 0x56414e5f4a00 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/2517230479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc1fc004990 con 0x56414e5f4a00 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc2153a8740 10 client.18078 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc2153a8740 10 client.18078 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc2153a8740 20 client.18078 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc2153a8740 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e604b70 0x56414e606f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc2153a8740 4 client.18078 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56414e4ef930 con 0x56414e604b70 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.078+0000 7fc20bfff640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e604b70 0x56414e606f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.079+0000 7fc20bfff640 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e604b70 0x56414e606f60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97551a3ecb0a39fc server_cookie=adf52bba3e419164 in_seq=0 out_seq=0 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.079+0000 7fc1eb7fe640 10 client.18078 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.079+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/2517230479 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56414e4ef930 con 0x56414e604b70 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.079+0000 7fc1eb7fe640 10 client.18078 handle_command_reply: tid=0 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.079+0000 7fc2153a8740 1 client.18078 shutdown 2024-04-22T13:33:49.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 msgr2=0x56414e5f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f1ce0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc001eb0 tx=0x7fc1fc011f70 comp rx=0 tx=0).stop 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 shutdown_connections 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e604b70 0x56414e606f60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f1ce0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc1eb7fe640 0 client.18078 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 --2- 172.21.15.5:0/2517230479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 >> 172.21.15.5:0/2517230479 conn(0x56414e5e5ba0 msgr2=0x56414e5e78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 shutdown_connections 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 1 -- 172.21.15.5:0/2517230479 wait complete. 2024-04-22T13:33:49.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.080+0000 7fc2153a8740 20 client.18078 trim_cache size 0 max 16384 2024-04-22T13:33:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 Processor -- start 2024-04-22T13:33:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 -- start start 2024-04-22T13:33:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5eb4d0 0x56414e600830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 0x56414e5f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5f4a00 0x56414e5f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1ab9e0 con 0x56414e5ebe80 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1ad520 con 0x56414e5eb4d0 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc2153a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1ac0b0 con 0x56414e5f4a00 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 0x56414e5f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 0x56414e5f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40642/0 (socket says 172.21.15.5:40642) 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 -- 172.21.15.5:0/3988171030 learned_addr learned my addr 172.21.15.5:0/3988171030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5f4a00 0x56414e5f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 -- 172.21.15.5:0/3988171030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5f4a00 msgr2=0x56414e5f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5f4a00 0x56414e5f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 -- 172.21.15.5:0/3988171030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5eb4d0 msgr2=0x56414e600830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5eb4d0 0x56414e600830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.093+0000 7fc20bfff640 1 -- 172.21.15.5:0/3988171030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414e5f71c0 con 0x56414e5ebe80 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc20bfff640 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 0x56414e5f3e00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400ee70 tx=0x7fc1f401ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78add521d68bfc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc201ffb640 1 -- 172.21.15.5:0/3988171030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004220 con 0x56414e5ebe80 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc201ffb640 1 -- 172.21.15.5:0/3988171030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40043c0 con 0x56414e5ebe80 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc201ffb640 1 -- 172.21.15.5:0/3988171030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f401f220 con 0x56414e5ebe80 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc2153a8740 1 -- 172.21.15.5:0/3988171030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 msgr2=0x56414e5f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc2153a8740 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 0x56414e5f3e00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400ee70 tx=0x7fc1f401ba20 comp rx=0 tx=0).stop 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc2153a8740 1 -- 172.21.15.5:0/3988171030 shutdown_connections 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc2153a8740 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5ebe80 0x56414e5f3e00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc2153a8740 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5f4a00 0x56414e5f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.094+0000 7fc2153a8740 1 --2- 172.21.15.5:0/3988171030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5eb4d0 0x56414e600830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- 172.21.15.5:0/3988171030 >> 172.21.15.5:0/3988171030 conn(0x56414e5e5ba0 msgr2=0x56414e5e7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- 172.21.15.5:0/3988171030 shutdown_connections 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- 172.21.15.5:0/3988171030 wait complete. 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- start start 2024-04-22T13:33:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 Processor -- start 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1ace50 con 0x56414e5f4a00 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1ac780 con 0x56414e5ebe80 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1a9a00 con 0x56414e5eb4d0 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.095+0000 7fc2153a8740 10 client.0 _ll_register_callbacks cb 0x7fc1fc00dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40646/0 (socket says 172.21.15.5:40646) 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 -- 172.21.15.5:0/1772171291 learned_addr learned my addr 172.21.15.5:0/1772171291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20bfff640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 -- 172.21.15.5:0/1772171291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 msgr2=0x56414e5fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 -- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 msgr2=0x56414e5fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 -- 172.21.15.5:0/1772171291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414e5f8b60 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc20b7fe640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f8330 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e010 tx=0x7fc1fc003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd53d59ca08eec5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc1eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1772171291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc019040 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1772171291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fc1fc004680 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f40043c0 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1772171291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1fc004880 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc2153a8740 10 client.18084 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56414e66ab80 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.096+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1772171291 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40043c0 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1772171291 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc1fc005000 con 0x56414e5f4a00 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc2153a8740 10 client.18084 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc2153a8740 10 client.18084 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:33:49.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc2153a8740 20 client.18084 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc2153a8740 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e66b190 0x56414e66d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc2153a8740 4 client.18084 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56414e55dc10 con 0x56414e66b190 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc20bfff640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e66b190 0x56414e66d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc20bfff640 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e66b190 0x56414e66d580 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d3a116229afd433 server_cookie=76793c46c8e5834b in_seq=0 out_seq=0 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc1eb7fe640 10 client.18084 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1772171291 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56414e55dc10 con 0x56414e66b190 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.097+0000 7fc1eb7fe640 10 client.18084 handle_command_reply: tid=0 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 client.18084 shutdown 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 msgr2=0x56414e5f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f8330 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e010 tx=0x7fc1fc003870 comp rx=0 tx=0).stop 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 shutdown_connections 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x56414e66b190 0x56414e66d580 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56414e5f4a00 0x56414e5f8330 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc1eb7fe640 0 client.18084 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56414e5ebe80 0x56414e5fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 --2- 172.21.15.5:0/1772171291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e5eb4d0 0x56414e5fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 >> 172.21.15.5:0/1772171291 conn(0x56414e5e5ba0 msgr2=0x56414e5ec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 shutdown_connections 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 1 -- 172.21.15.5:0/1772171291 wait complete. 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2153a8740 20 client.18084 trim_cache size 0 max 16384 2024-04-22T13:33:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1f80a3d20 msgr2=0x7fc1f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1f80a3d20 0x7fc1f80a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc2040bb500 tx=0x7fc2040c1040 comp rx=0 tx=0).stop 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 msgr2=0x7fc20c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 0x7fc20c198bc0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc204005e30 tx=0x7fc204034760 comp rx=0 tx=0).stop 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 shutdown_connections 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20c198740 0x7fc20c198bc0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc1f80a3d20 0x7fc1f80a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc20c10c6d0 0x7fc20c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 --2- 172.21.15.5:0/3724459730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc20c1031c0 0x7fc20c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.098+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 >> 172.21.15.5:0/3724459730 conn(0x7fc20c077f70 msgr2=0x7fc20c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.099+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 shutdown_connections 2024-04-22T13:33:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.099+0000 7fc2126df640 1 -- 172.21.15.5:0/3724459730 wait complete. 2024-04-22T13:33:49.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 -- 172.21.15.5:0/2375663298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 msgr2=0x7fa1981035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 --2- 172.21.15.5:0/2375663298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa1981035a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009900 tx=0x7fa18c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 -- 172.21.15.5:0/2375663298 shutdown_connections 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 --2- 172.21.15.5:0/2375663298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa19810c500 0x7fa19810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 --2- 172.21.15.5:0/2375663298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa198103ae0 0x7fa19810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 --2- 172.21.15.5:0/2375663298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa1981035a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 -- 172.21.15.5:0/2375663298 >> 172.21.15.5:0/2375663298 conn(0x7fa198078000 msgr2=0x7fa198078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 -- 172.21.15.5:0/2375663298 shutdown_connections 2024-04-22T13:33:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.213+0000 7fa19e474640 1 -- 172.21.15.5:0/2375663298 wait complete. 2024-04-22T13:33:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 Processor -- start 2024-04-22T13:33:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 -- start start 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa198100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198103ae0 0x7fa198101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19810c500 0x7fa198101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981121c0 con 0x7fa1981031c0 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa198112040 con 0x7fa19810c500 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa19e474640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa198112340 con 0x7fa198103ae0 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa197fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa198100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa197fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa198100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40672/0 (socket says 172.21.15.5:40672) 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.214+0000 7fa197fff640 1 -- 172.21.15.5:0/2530966905 learned_addr learned my addr 172.21.15.5:0/2530966905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa19ca11640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19810c500 0x7fa198101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa1977fe640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198103ae0 0x7fa198101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa197fff640 1 -- 172.21.15.5:0/2530966905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198103ae0 msgr2=0x7fa198101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa197fff640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198103ae0 0x7fa198101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa197fff640 1 -- 172.21.15.5:0/2530966905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19810c500 msgr2=0x7fa198101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa197fff640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19810c500 0x7fa198101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa197fff640 1 -- 172.21.15.5:0/2530966905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1981a6ba0 con 0x7fa1981031c0 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa1977fe640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198103ae0 0x7fa198101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa19ca11640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19810c500 0x7fa198101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa197fff640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa198100e10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa18c005ab0 tx=0x7fa18c02db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa1957fa640 1 -- 172.21.15.5:0/2530966905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c043020 con 0x7fa1981031c0 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa1957fa640 1 -- 172.21.15.5:0/2530966905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c0052b0 con 0x7fa1981031c0 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1981a6e30 con 0x7fa1981031c0 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa1957fa640 1 -- 172.21.15.5:0/2530966905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c003c70 con 0x7fa1981031c0 2024-04-22T13:33:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.215+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a72f0 con 0x7fa1981031c0 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.217+0000 7fa1957fa640 1 -- 172.21.15.5:0/2530966905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa18c046020 con 0x7fa1981031c0 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.217+0000 7fa1a106f740 1 Processor -- start 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 -- start start 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56276704c1f0 con 0x5627670e7e80 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562766fe94b0 con 0x5627670f0a00 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1a106f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5627670481b0 con 0x5627670e74d0 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40678/0 (socket says 172.21.15.5:40678) 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 -- 172.21.15.5:0/1118729236 learned_addr learned my addr 172.21.15.5:0/1118729236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1957fa640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1800a3d20 0x7fa1800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa19ca11640 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1977fe640 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa19ca11640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1800a3d20 0x7fa1800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 -- 172.21.15.5:0/1118729236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 msgr2=0x5627670e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 -- 172.21.15.5:0/1118729236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 msgr2=0x5627670f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa197fff640 1 -- 172.21.15.5:0/1118729236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562766fdb1b0 con 0x5627670e7e80 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.218+0000 7fa1957fa640 1 -- 172.21.15.5:0/2530966905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa18c0f2a30 con 0x7fa1981031c0 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa197fff640 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f0430 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0bb480 tx=0x7fa18c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121f7371552ac78d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa19ca11640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1800a3d20 0x7fa1800a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa190007fd0 tx=0x7fa190004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa17e7fc640 1 -- 172.21.15.5:0/1118729236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0be410 con 0x5627670e7e80 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa17e7fc640 1 -- 172.21.15.5:0/1118729236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c0035f0 con 0x5627670e7e80 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa17e7fc640 1 -- 172.21.15.5:0/1118729236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0df0f0 con 0x5627670e7e80 2024-04-22T13:33:49.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 -- 172.21.15.5:0/1118729236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 msgr2=0x5627670f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f0430 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0bb480 tx=0x7fa18c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 -- 172.21.15.5:0/1118729236 shutdown_connections 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f0430 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1118729236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 -- 172.21.15.5:0/1118729236 >> 172.21.15.5:0/1118729236 conn(0x5627670e1ba0 msgr2=0x5627670e1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 -- 172.21.15.5:0/1118729236 shutdown_connections 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.219+0000 7fa1a106f740 1 -- 172.21.15.5:0/1118729236 wait complete. 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 -- start start 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 Processor -- start 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670f0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e7e80 0x5627670ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 0x5627670edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56276704c1f0 con 0x5627670f0a00 2024-04-22T13:33:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562767005500 con 0x5627670e74d0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5627670f3730 con 0x5627670e7e80 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1a106f740 10 client.0 _ll_register_callbacks cb 0x562766fd3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa19ca11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 0x5627670edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa19ca11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 0x5627670edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40694/0 (socket says 172.21.15.5:40694) 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa19ca11640 1 -- 172.21.15.5:0/608063079 learned_addr learned my addr 172.21.15.5:0/608063079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa197fff640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.220+0000 7fa1977fe640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e7e80 0x5627670ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa19ca11640 1 -- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e7e80 msgr2=0x5627670ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa19ca11640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e7e80 0x5627670ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa19ca11640 1 -- 172.21.15.5:0/608063079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 msgr2=0x5627670f0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa19ca11640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670f0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa19ca11640 1 -- 172.21.15.5:0/608063079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562766d5e3f0 con 0x5627670f0a00 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa19ca11640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 0x5627670edb70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa190014fd0 tx=0x7fa190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532f08d79d2030b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1777fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1777fe640 1 -- 172.21.15.5:0/608063079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190010ea0 con 0x5627670f0a00 2024-04-22T13:33:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1777fe640 1 -- 172.21.15.5:0/608063079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa190004d20 con 0x5627670f0a00 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562766feb930 con 0x5627670f0a00 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1777fe640 1 -- 172.21.15.5:0/608063079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa19000ee80 con 0x5627670f0a00 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1a106f740 10 client.18096 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562766e287d0 con 0x5627670f0a00 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1777fe640 1 -- 172.21.15.5:0/608063079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562766feb930 con 0x5627670f0a00 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.221+0000 7fa1777fe640 1 -- 172.21.15.5:0/608063079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa19001a040 con 0x5627670f0a00 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1a106f740 10 client.18096 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1a106f740 10 client.18096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1a106f740 20 client.18096 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1a106f740 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5627671008f0 0x562767102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1a106f740 4 client.18096 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562767059c10 con 0x5627671008f0 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa197fff640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5627671008f0 0x562767102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa1777fe640 10 client.18096 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.222+0000 7fa197fff640 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5627671008f0 0x562767102d30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52690ba5135427d3 server_cookie=49505baa5023cef0 in_seq=0 out_seq=0 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1777fe640 1 -- 172.21.15.5:0/608063079 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562767059c10 con 0x5627671008f0 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1777fe640 10 client.18096 handle_command_reply: tid=0 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 client.18096 shutdown 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 msgr2=0x5627670edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 0x5627670edb70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa190014fd0 tx=0x7fa190004a40 comp rx=0 tx=0).stop 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 shutdown_connections 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x5627671008f0 0x562767102d30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670f0a00 0x5627670edb70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e7e80 0x5627670ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 --2- 172.21.15.5:0/608063079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670f0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 >> 172.21.15.5:0/608063079 conn(0x5627670e1ba0 msgr2=0x5627670e38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1777fe640 0 client.18096 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 shutdown_connections 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 1 -- 172.21.15.5:0/608063079 wait complete. 2024-04-22T13:33:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.223+0000 7fa1a106f740 20 client.18096 trim_cache size 0 max 16384 2024-04-22T13:33:49.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 Processor -- start 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 -- start start 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670f0a00 0x562767108600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981afb80 con 0x5627670e7e80 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981b16c0 con 0x5627670e74d0 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1a106f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981b0250 con 0x5627670f0a00 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40704/0 (socket says 172.21.15.5:40704) 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 -- 172.21.15.5:0/1980519043 learned_addr learned my addr 172.21.15.5:0/1980519043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa197fff640 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 -- 172.21.15.5:0/1980519043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670f0a00 msgr2=0x562767108600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670f0a00 0x562767108600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 -- 172.21.15.5:0/1980519043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 msgr2=0x5627670fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 -- 172.21.15.5:0/1980519043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627670ff450 con 0x5627670e7e80 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa197fff640 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa1977fe640 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670e8ae0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa18800fc20 tx=0x7fa188011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acb43a2225fccbd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.237+0000 7fa17dffb640 1 -- 172.21.15.5:0/1980519043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18801a640 con 0x5627670e7e80 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa17dffb640 1 -- 172.21.15.5:0/1980519043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa188019890 con 0x5627670e7e80 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa17dffb640 1 -- 172.21.15.5:0/1980519043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18800e9e0 con 0x5627670e7e80 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 -- 172.21.15.5:0/1980519043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 msgr2=0x5627670e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670e8ae0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa18800fc20 tx=0x7fa188011810 comp rx=0 tx=0).stop 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 -- 172.21.15.5:0/1980519043 shutdown_connections 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670e8ae0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670f0a00 0x562767108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 --2- 172.21.15.5:0/1980519043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670e74d0 0x5627670fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 -- 172.21.15.5:0/1980519043 >> 172.21.15.5:0/1980519043 conn(0x5627670e1ba0 msgr2=0x5627670e83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 -- 172.21.15.5:0/1980519043 shutdown_connections 2024-04-22T13:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 -- 172.21.15.5:0/1980519043 wait complete. 2024-04-22T13:33:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.238+0000 7fa1a106f740 1 -- start start 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 Processor -- start 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981b0ff0 con 0x5627670e7e80 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981b0920 con 0x5627670f0a00 2024-04-22T13:33:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981adb60 con 0x5627670e74d0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1a106f740 10 client.0 _ll_register_callbacks cb 0x7fa190010940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40708/0 (socket says 172.21.15.5:40708) 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 -- 172.21.15.5:0/2240599925 learned_addr learned my addr 172.21.15.5:0/2240599925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa197fff640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 -- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 msgr2=0x5627670f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 -- 172.21.15.5:0/2240599925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 msgr2=0x5627670f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa19ca11640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa1977fe640 1 -- 172.21.15.5:0/2240599925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627670f4620 con 0x5627670e7e80 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.239+0000 7fa19ca11640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1977fe640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f8e50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa188019830 tx=0x7fa188005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55525ba2ceb4443c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1777fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1777fe640 1 -- 172.21.15.5:0/2240599925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188028020 con 0x5627670e7e80 2024-04-22T13:33:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1777fe640 1 -- 172.21.15.5:0/2240599925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa188019c50 con 0x5627670e7e80 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa188019890 con 0x5627670e7e80 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1777fe640 1 -- 172.21.15.5:0/2240599925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa188033050 con 0x5627670e7e80 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 10 client.18102 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562767166a10 con 0x5627670e7e80 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1777fe640 1 -- 172.21.15.5:0/2240599925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa188019890 con 0x5627670e7e80 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1777fe640 1 -- 172.21.15.5:0/2240599925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa188018070 con 0x5627670e7e80 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 10 client.18102 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 10 client.18102 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 20 client.18102 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562767167100 0x562767169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 4 client.18102 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.240+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562767053a80 con 0x562767167100 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa197fff640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562767167100 0x562767169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1777fe640 10 client.18102 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa197fff640 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562767167100 0x562767169520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=994e290bcf9b6a44 server_cookie=a20ebd6be33376fe in_seq=0 out_seq=0 2024-04-22T13:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1777fe640 1 -- 172.21.15.5:0/2240599925 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562767053a80 con 0x562767167100 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1777fe640 10 client.18102 handle_command_reply: tid=0 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 client.18102 shutdown 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 msgr2=0x5627670f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f8e50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa188019830 tx=0x7fa188005e50 comp rx=0 tx=0).stop 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 shutdown_connections 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x562767167100 0x562767169520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627670e7e80 0x5627670f8e50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5627670f0a00 0x5627670f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 --2- 172.21.15.5:0/2240599925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627670e74d0 0x5627670f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1777fe640 0 client.18102 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 >> 172.21.15.5:0/2240599925 conn(0x5627670e1ba0 msgr2=0x5627670e3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 shutdown_connections 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.241+0000 7fa1a106f740 1 -- 172.21.15.5:0/2240599925 wait complete. 2024-04-22T13:33:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa1a106f740 20 client.18102 trim_cache size 0 max 16384 2024-04-22T13:33:49.243 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1800a3d20 msgr2=0x7fa1800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1800a3d20 0x7fa1800a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa190007fd0 tx=0x7fa190004ab0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 msgr2=0x7fa198100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa198100e10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa18c005ab0 tx=0x7fa18c02db40 comp rx=0 tx=0).stop 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 shutdown_connections 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1800a3d20 0x7fa1800a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19810c500 0x7fa198101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198103ae0 0x7fa198101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 --2- 172.21.15.5:0/2530966905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1981031c0 0x7fa198100e10 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 >> 172.21.15.5:0/2530966905 conn(0x7fa198078000 msgr2=0x7fa1980fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 shutdown_connections 2024-04-22T13:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.242+0000 7fa19e474640 1 -- 172.21.15.5:0/2530966905 wait complete. 2024-04-22T13:33:49.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:33:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.356+0000 7f2b11e32640 1 -- 172.21.15.5:0/1285375869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c106160 msgr2=0x7f2b0c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.356+0000 7f2b11e32640 1 --2- 172.21.15.5:0/1285375869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c106160 0x7f2b0c10dd30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009ca0 tx=0x7f2b0402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 -- 172.21.15.5:0/1285375869 shutdown_connections 2024-04-22T13:33:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 --2- 172.21.15.5:0/1285375869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c106160 0x7f2b0c10dd30 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 --2- 172.21.15.5:0/1285375869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c101a40 0x7f2b0c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 --2- 172.21.15.5:0/1285375869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0c101090 0x7f2b0c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 -- 172.21.15.5:0/1285375869 >> 172.21.15.5:0/1285375869 conn(0x7f2b0c078000 msgr2=0x7f2b0c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 -- 172.21.15.5:0/1285375869 shutdown_connections 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 -- 172.21.15.5:0/1285375869 wait complete. 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 Processor -- start 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.357+0000 7f2b11e32640 1 -- start start 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 0x7f2b0c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0c106160 0x7f2b0c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c19cfa0 0x7f2b0c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1121e0 con 0x7f2b0c101090 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c112060 con 0x7f2b0c106160 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c112360 con 0x7f2b0c19cfa0 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 0x7f2b0c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 0x7f2b0c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40720/0 (socket says 172.21.15.5:40720) 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/795964505 learned_addr learned my addr 172.21.15.5:0/795964505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c19cfa0 0x7f2b0c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/795964505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c19cfa0 msgr2=0x7f2b0c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0affd640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0c106160 0x7f2b0c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c19cfa0 0x7f2b0c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/795964505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0c106160 msgr2=0x7f2b0c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0c106160 0x7f2b0c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/795964505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c19da90 con 0x7f2b0c101090 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c19cfa0 0x7f2b0c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 0x7f2b0c1986e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d950 tx=0x7f2afc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b08ff9640 1 -- 172.21.15.5:0/795964505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc017070 con 0x7f2b0c101090 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c19dd80 con 0x7f2b0c101090 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b08ff9640 1 -- 172.21.15.5:0/795964505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc009c30 con 0x7f2b0c101090 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b08ff9640 1 -- 172.21.15.5:0/795964505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00f620 con 0x7f2b0c101090 2024-04-22T13:33:49.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.358+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a5570 con 0x7f2b0c101090 2024-04-22T13:33:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b08ff9640 1 -- 172.21.15.5:0/795964505 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2afc010460 con 0x7f2b0c101090 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 Processor -- start 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 -- start start 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eaa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c84e0f1f0 con 0x563c84eb3a00 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563c84dac4b0 con 0x563c84eaa4d0 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563c84e0b1b0 con 0x563c84eaae80 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b0affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b0affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40730/0 (socket says 172.21.15.5:40730) 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b0affd640 1 -- 172.21.15.5:0/2410224297 learned_addr learned my addr 172.21.15.5:0/2410224297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.361+0000 7f2b08ff9640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2af40a3d20 0x7f2af40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eaa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2af40a3d20 0x7f2af40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0affd640 1 -- 172.21.15.5:0/2410224297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 msgr2=0x563c84eb3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b08ff9640 1 -- 172.21.15.5:0/795964505 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2afc0d04c0 con 0x7f2b0c101090 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0affd640 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0affd640 1 -- 172.21.15.5:0/2410224297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 msgr2=0x563c84eaa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0affd640 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eaa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0affd640 1 -- 172.21.15.5:0/2410224297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c84d9e1b0 con 0x563c84eb3a00 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eaa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0affd640 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb5e50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009a90 tx=0x7f2b00009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bd56c70023a6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2af1ffb640 1 -- 172.21.15.5:0/2410224297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00016070 con 0x563c84eb3a00 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2af40a3d20 0x7f2af40a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b04005e30 tx=0x7f2b040095c0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2af1ffb640 1 -- 172.21.15.5:0/2410224297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0000ecb0 con 0x563c84eb3a00 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.362+0000 7f2af1ffb640 1 -- 172.21.15.5:0/2410224297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00003db0 con 0x563c84eb3a00 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2410224297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 msgr2=0x563c84eb5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb5e50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009a90 tx=0x7f2b00009f50 comp rx=0 tx=0).stop 2024-04-22T13:33:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2410224297 shutdown_connections 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb5e50 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2410224297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eaa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2410224297 >> 172.21.15.5:0/2410224297 conn(0x563c84ea4ba0 msgr2=0x563c84ea4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2410224297 shutdown_connections 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2410224297 wait complete. 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 -- start start 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.363+0000 7f2b14a1a740 1 Processor -- start 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eb32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c84eb6970 con 0x563c84eb3a00 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563c84dc8500 con 0x563c84eaa4d0 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563c84eb6af0 con 0x563c84eaae80 2024-04-22T13:33:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b14a1a740 10 client.0 _ll_register_callbacks cb 0x563c84d96b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40732/0 (socket says 172.21.15.5:40732) 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 -- 172.21.15.5:0/922813651 learned_addr learned my addr 172.21.15.5:0/922813651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eb32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0affd640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 -- 172.21.15.5:0/922813651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 msgr2=0x563c84eb0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 -- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 msgr2=0x563c84eb32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eb32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 -- 172.21.15.5:0/922813651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c84b213f0 con 0x563c84eb3a00 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb0b80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2b04032ac0 tx=0x7f2b0402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5b233a4b5fde98 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.364+0000 7f2aeaffd640 1 -- 172.21.15.5:0/922813651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04051060 con 0x563c84eb3a00 2024-04-22T13:33:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2aeaffd640 1 -- 172.21.15.5:0/922813651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2b04004040 con 0x563c84eb3a00 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b0000ecb0 con 0x563c84eb3a00 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2aeaffd640 1 -- 172.21.15.5:0/922813651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b04039040 con 0x563c84eb3a00 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 10 client.18114 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c84beb7d0 con 0x563c84eb3a00 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2aeaffd640 1 -- 172.21.15.5:0/922813651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0000ecb0 con 0x563c84eb3a00 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2aeaffd640 1 -- 172.21.15.5:0/922813651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2b04041980 con 0x563c84eb3a00 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 10 client.18114 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 10 client.18114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 20 client.18114 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84ec3bc0 0x563c84ec6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 4 client.18114 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.365+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c84dae930 con 0x563c84ec3bc0 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.366+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84ec3bc0 0x563c84ec6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.366+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84ec3bc0 0x563c84ec6030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c0925e9090cf7d3 server_cookie=b6d0f74846f45d2c in_seq=0 out_seq=0 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.366+0000 7f2aeaffd640 10 client.18114 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2aeaffd640 1 -- 172.21.15.5:0/922813651 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c84dae930 con 0x563c84ec3bc0 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2aeaffd640 10 client.18114 handle_command_reply: tid=0 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 client.18114 shutdown 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 msgr2=0x563c84eb0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb0b80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2b04032ac0 tx=0x7f2b0402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 shutdown_connections 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84ec3bc0 0x563c84ec6030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84eb3a00 0x563c84eb0b80 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaae80 0x563c84eb0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2aeaffd640 0 client.18114 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/922813651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaa4d0 0x563c84eb32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 >> 172.21.15.5:0/922813651 conn(0x563c84ea4ba0 msgr2=0x563c84ea68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 shutdown_connections 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 1 -- 172.21.15.5:0/922813651 wait complete. 2024-04-22T13:33:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.367+0000 7f2b14a1a740 20 client.18114 trim_cache size 0 max 16384 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 Processor -- start 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 -- start start 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84ea3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84eb2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1ada40 con 0x563c84ea4ba0 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1af580 con 0x563c84eaae80 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.380+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1ae110 con 0x563c84eaa4d0 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40742/0 (socket says 172.21.15.5:40742) 2024-04-22T13:33:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2817076217 learned_addr learned my addr 172.21.15.5:0/2817076217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2817076217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 msgr2=0x563c84ea3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84eb2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0affd640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84ea3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84ea3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2817076217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 msgr2=0x563c84eb2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84eb2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2817076217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c84eb6850 con 0x563c84ea4ba0 2024-04-22T13:33:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84eb2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0affd640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84ea3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebf830 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0cdca0 tx=0x7f2afc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721dcfb33de8a1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2af17fa640 1 -- 172.21.15.5:0/2817076217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc020040 con 0x563c84ea4ba0 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2af17fa640 1 -- 172.21.15.5:0/2817076217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0a5100 con 0x563c84ea4ba0 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.381+0000 7f2af17fa640 1 -- 172.21.15.5:0/2817076217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc016170 con 0x563c84ea4ba0 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2817076217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 msgr2=0x563c84ebf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebf830 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0cdca0 tx=0x7f2afc007f90 comp rx=0 tx=0).stop 2024-04-22T13:33:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2817076217 shutdown_connections 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84eb2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84ea3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/2817076217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebf830 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2817076217 >> 172.21.15.5:0/2817076217 conn(0x563c84eb0ce0 msgr2=0x563c84ea3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2817076217 shutdown_connections 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 -- 172.21.15.5:0/2817076217 wait complete. 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 -- start start 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.382+0000 7f2b14a1a740 1 Processor -- start 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84eaceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84ead3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1aeeb0 con 0x563c84ea4ba0 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1ae7e0 con 0x563c84eaae80 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1aba20 con 0x563c84eaa4d0 2024-04-22T13:33:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 10 client.0 _ll_register_callbacks cb 0x7f2b04009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40758/0 (socket says 172.21.15.5:40758) 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/130177042 learned_addr learned my addr 172.21.15.5:0/130177042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84ead3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/130177042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 msgr2=0x563c84eaceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0affd640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84eaceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84eaceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 msgr2=0x563c84ead3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84ead3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/130177042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c84eada60 con 0x563c84ea4ba0 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0bfff640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84ead3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0affd640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84eaceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b0b7fe640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebbfa0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2afc098e60 tx=0x7f2afc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696be40faee58f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2aeaffd640 1 -- 172.21.15.5:0/130177042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc020040 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2aeaffd640 1 -- 172.21.15.5:0/130177042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2afc016170 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2afc0a5100 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2aeaffd640 1 -- 172.21.15.5:0/130177042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2afc0d0030 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 10 client.18120 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.383+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c84f29f90 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2aeaffd640 1 -- 172.21.15.5:0/130177042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0a5100 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2aeaffd640 1 -- 172.21.15.5:0/130177042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2afc098270 con 0x563c84ea4ba0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b14a1a740 10 client.18120 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b14a1a740 10 client.18120 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b14a1a740 20 client.18120 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84eb3a00 0x563c84f2c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b14a1a740 4 client.18120 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c84e1cc10 con 0x563c84eb3a00 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b0affd640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84eb3a00 0x563c84f2c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2aeaffd640 10 client.18120 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.384+0000 7f2b0affd640 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84eb3a00 0x563c84f2c6c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cb6835ff1fcdb53 server_cookie=1f1b94f21b318729 in_seq=0 out_seq=0 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2aeaffd640 1 -- 172.21.15.5:0/130177042 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c84e1cc10 con 0x563c84eb3a00 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2aeaffd640 10 client.18120 handle_command_reply: tid=0 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 client.18120 shutdown 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 msgr2=0x563c84ebbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebbfa0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2afc098e60 tx=0x7f2afc009ed0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 shutdown_connections 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x563c84eb3a00 0x563c84f2c6c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563c84eaae80 0x563c84ead3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c84eaa4d0 0x563c84eaceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2aeaffd640 0 client.18120 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 --2- 172.21.15.5:0/130177042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c84ea4ba0 0x563c84ebbfa0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 >> 172.21.15.5:0/130177042 conn(0x563c84eb0ce0 msgr2=0x563c84eabaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 shutdown_connections 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 1 -- 172.21.15.5:0/130177042 wait complete. 2024-04-22T13:33:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b14a1a740 20 client.18120 trim_cache size 0 max 16384 2024-04-22T13:33:49.389 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2af40a3d20 msgr2=0x7f2af40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b11e32640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2af40a3d20 0x7f2af40a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b04005e30 tx=0x7f2b040095c0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 msgr2=0x7f2b0c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.385+0000 7f2b11e32640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 0x7f2b0c1986e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d950 tx=0x7f2afc00de10 comp rx=0 tx=0).stop 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 shutdown_connections 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2af40a3d20 0x7f2af40a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b0c19cfa0 0x7f2b0c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0c106160 0x7f2b0c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 --2- 172.21.15.5:0/795964505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b0c101090 0x7f2b0c1986e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 >> 172.21.15.5:0/795964505 conn(0x7f2b0c078000 msgr2=0x7f2b0c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 shutdown_connections 2024-04-22T13:33:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.386+0000 7f2b11e32640 1 -- 172.21.15.5:0/795964505 wait complete. 2024-04-22T13:33:49.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:49 smithi005 ceph-mon[28006]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:49.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:49 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1126313445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:49.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:33:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 -- 172.21.15.5:0/2351490015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220103b70 msgr2=0x7f222010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 --2- 172.21.15.5:0/2351490015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220103b70 0x7f222010bfc0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2208009900 tx=0x7f220802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 -- 172.21.15.5:0/2351490015 shutdown_connections 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 --2- 172.21.15.5:0/2351490015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220103b70 0x7f222010bfc0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 --2- 172.21.15.5:0/2351490015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f222010c6d0 0x7f222010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 --2- 172.21.15.5:0/2351490015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22201031c0 0x7f22201035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 -- 172.21.15.5:0/2351490015 >> 172.21.15.5:0/2351490015 conn(0x7f2220077f70 msgr2=0x7f2220078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 -- 172.21.15.5:0/2351490015 shutdown_connections 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 -- 172.21.15.5:0/2351490015 wait complete. 2024-04-22T13:33:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.501+0000 7f2224aba640 1 Processor -- start 2024-04-22T13:33:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 -- start start 2024-04-22T13:33:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22201031c0 0x7f2220100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f222010c6d0 0x7f2220101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 0x7f22201a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2220112330 con 0x7f2220101890 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22201121b0 con 0x7f222010c6d0 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2224aba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22201124b0 con 0x7f22201031c0 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 0x7f22201a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 0x7f22201a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40774/0 (socket says 172.21.15.5:40774) 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 -- 172.21.15.5:0/3959968454 learned_addr learned my addr 172.21.15.5:0/3959968454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221e59c640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22201031c0 0x7f2220100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f222010c6d0 0x7f2220101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 -- 172.21.15.5:0/3959968454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22201031c0 msgr2=0x7f2220100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22201031c0 0x7f2220100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 -- 172.21.15.5:0/3959968454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f222010c6d0 msgr2=0x7f2220101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f222010c6d0 0x7f2220101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 -- 172.21.15.5:0/3959968454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22201a7140 con 0x7f2220101890 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.502+0000 7f221ed9d640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 0x7f22201a6a40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f221800eaa0 tx=0x7f221800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.503+0000 7f2217fff640 1 -- 172.21.15.5:0/3959968454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218004d20 con 0x7f2220101890 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.503+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22201a7430 con 0x7f2220101890 2024-04-22T13:33:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.503+0000 7f2217fff640 1 -- 172.21.15.5:0/3959968454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2218005020 con 0x7f2220101890 2024-04-22T13:33:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.503+0000 7f2217fff640 1 -- 172.21.15.5:0/3959968454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180116b0 con 0x7f2220101890 2024-04-22T13:33:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.503+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201a7900 con 0x7f2220101890 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.505+0000 7f2217fff640 1 -- 172.21.15.5:0/3959968454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f22180051d0 con 0x7f2220101890 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.505+0000 7f2227785740 1 Processor -- start 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.505+0000 7f2227785740 1 -- start start 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.505+0000 7f2227785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a0484d0 0x55652a0488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2227785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a051430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2227785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a051a00 0x55652a053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2227785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556529fad1f0 con 0x55652a048e80 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2227785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556529f4a4b0 con 0x55652a0484d0 2024-04-22T13:33:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2227785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556529fa91b0 con 0x55652a051a00 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2217fff640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f220c0a3d50 0x7f220c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a051430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a051430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40780/0 (socket says 172.21.15.5:40780) 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 -- 172.21.15.5:0/3610093313 learned_addr learned my addr 172.21.15.5:0/3610093313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221ed9d640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f220c0a3d50 0x7f220c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a0484d0 0x55652a0488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a051a00 0x55652a053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2217fff640 1 -- 172.21.15.5:0/3959968454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f22180d08e0 con 0x7f2220101890 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 -- 172.21.15.5:0/3610093313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a051a00 msgr2=0x55652a053e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a051a00 0x55652a053e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 -- 172.21.15.5:0/3610093313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a0484d0 msgr2=0x55652a0488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a0484d0 0x55652a0488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 -- 172.21.15.5:0/3610093313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556529f3c1b0 con 0x55652a048e80 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a051a00 0x55652a053e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221e59c640 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a051430 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f221000ad00 tx=0x7f221000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93fa12efbe25f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f221ed9d640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f220c0a3d50 0x7f220c0a6210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2218099280 tx=0x7f2218098510 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2214d99640 1 -- 172.21.15.5:0/3610093313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221001d070 con 0x55652a048e80 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2214d99640 1 -- 172.21.15.5:0/3610093313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2210018af0 con 0x55652a048e80 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.506+0000 7f2214d99640 1 -- 172.21.15.5:0/3610093313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210010b30 con 0x55652a048e80 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 -- 172.21.15.5:0/3610093313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 msgr2=0x55652a051430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a051430 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f221000ad00 tx=0x7f221000f790 comp rx=0 tx=0).stop 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 -- 172.21.15.5:0/3610093313 shutdown_connections 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a051430 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a051a00 0x55652a053e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 --2- 172.21.15.5:0/3610093313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a0484d0 0x55652a0488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 -- 172.21.15.5:0/3610093313 >> 172.21.15.5:0/3610093313 conn(0x55652a042ba0 msgr2=0x55652a042f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 -- 172.21.15.5:0/3610093313 shutdown_connections 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.507+0000 7f2227785740 1 -- 172.21.15.5:0/3610093313 wait complete. 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 -- start start 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 Processor -- start 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a0512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a04e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a051a00 0x55652a04ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55652a054970 con 0x55652a048e80 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556529f66500 con 0x55652a051a00 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55652a054af0 con 0x55652a0484d0 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2227785740 10 client.0 _ll_register_callbacks cb 0x556529f34b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a04e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a04e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40794/0 (socket says 172.21.15.5:40794) 2024-04-22T13:33:49.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 -- 172.21.15.5:0/3557701738 learned_addr learned my addr 172.21.15.5:0/3557701738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a0512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a051a00 0x55652a04ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 -- 172.21.15.5:0/3557701738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 msgr2=0x55652a0512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a0512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 -- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a051a00 msgr2=0x55652a04ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a051a00 0x55652a04ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f221e59c640 1 -- 172.21.15.5:0/3557701738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556529cbf3f0 con 0x55652a048e80 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.508+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a051a00 0x55652a04ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f221e59c640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a04e680 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2210000c00 tx=0x7f22100186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93c5958cd2dde75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2205ffb640 1 -- 172.21.15.5:0/3557701738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221001d040 con 0x55652a048e80 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2205ffb640 1 -- 172.21.15.5:0/3557701738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f2210010870 con 0x55652a048e80 2024-04-22T13:33:49.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2210018af0 con 0x55652a048e80 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2205ffb640 1 -- 172.21.15.5:0/3557701738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2210031050 con 0x55652a048e80 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 10 client.18132 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556529d897d0 con 0x55652a048e80 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2205ffb640 1 -- 172.21.15.5:0/3557701738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2210018af0 con 0x55652a048e80 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2205ffb640 1 -- 172.21.15.5:0/3557701738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2210020030 con 0x55652a048e80 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 10 client.18132 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 20 client.18132 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a061aa0 0x55652a063e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 4 client.18132 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.509+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556529f4c930 con 0x55652a061aa0 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.510+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a061aa0 0x55652a063e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.510+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a061aa0 0x55652a063e90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23e1648ab5c3e11 server_cookie=57fd4266883105fb in_seq=0 out_seq=0 2024-04-22T13:33:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.510+0000 7f2205ffb640 10 client.18132 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2205ffb640 1 -- 172.21.15.5:0/3557701738 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556529f4c930 con 0x55652a061aa0 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2205ffb640 10 client.18132 handle_command_reply: tid=0 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 client.18132 shutdown 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 msgr2=0x55652a04e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a04e680 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2210000c00 tx=0x7f22100186f0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 shutdown_connections 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a061aa0 0x55652a063e90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a048e80 0x55652a04e680 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a051a00 0x55652a04ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2205ffb640 0 client.18132 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 --2- 172.21.15.5:0/3557701738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a0512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 >> 172.21.15.5:0/3557701738 conn(0x55652a042ba0 msgr2=0x55652a0448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 shutdown_connections 2024-04-22T13:33:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 1 -- 172.21.15.5:0/3557701738 wait complete. 2024-04-22T13:33:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.511+0000 7f2227785740 20 client.18132 trim_cache size 0 max 16384 2024-04-22T13:33:49.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:49 smithi049 ceph-mon[31379]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:49.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:49 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1126313445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 Processor -- start 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 -- start start 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a05d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a04d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a050db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22201afd40 con 0x55652a051a00 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b1880 con 0x55652a048e80 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2227785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b0410 con 0x55652a0484d0 2024-04-22T13:33:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a050db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a050db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40796/0 (socket says 172.21.15.5:40796) 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 -- 172.21.15.5:0/3706733808 learned_addr learned my addr 172.21.15.5:0/3706733808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f221e59c640 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a04d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a05d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 -- 172.21.15.5:0/3706733808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 msgr2=0x55652a05d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a05d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 -- 172.21.15.5:0/3706733808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 msgr2=0x55652a04d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a04d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 -- 172.21.15.5:0/3706733808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55652a04cbd0 con 0x55652a051a00 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.525+0000 7f2215d9b640 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a050db0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f222006a2d0 tx=0x7f2208005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cfd5d6128410c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2207fff640 1 -- 172.21.15.5:0/3706733808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220804cc90 con 0x55652a051a00 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2207fff640 1 -- 172.21.15.5:0/3706733808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220804ce30 con 0x55652a051a00 2024-04-22T13:33:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2207fff640 1 -- 172.21.15.5:0/3706733808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220804c2d0 con 0x55652a051a00 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 -- 172.21.15.5:0/3706733808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 msgr2=0x55652a050db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a050db0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f222006a2d0 tx=0x7f2208005e40 comp rx=0 tx=0).stop 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 -- 172.21.15.5:0/3706733808 shutdown_connections 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a050db0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a04d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 --2- 172.21.15.5:0/3706733808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a05d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 -- 172.21.15.5:0/3706733808 >> 172.21.15.5:0/3706733808 conn(0x55652a042ba0 msgr2=0x55652a041c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 -- 172.21.15.5:0/3706733808 shutdown_connections 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 -- 172.21.15.5:0/3706733808 wait complete. 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.526+0000 7f2227785740 1 -- start start 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 Processor -- start 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a059d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a05a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a04aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b11b0 con 0x55652a051a00 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b0ae0 con 0x55652a048e80 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22201add40 con 0x55652a0484d0 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2227785740 10 client.0 _ll_register_callbacks cb 0x7f22100106f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a04aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a04aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40802/0 (socket says 172.21.15.5:40802) 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 -- 172.21.15.5:0/2615202521 learned_addr learned my addr 172.21.15.5:0/2615202521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f221e59c640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a05a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a059d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 -- 172.21.15.5:0/2615202521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 msgr2=0x55652a059d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a059d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 -- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 msgr2=0x55652a05a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a05a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 -- 172.21.15.5:0/2615202521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55652a04b280 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2215d9b640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a04aab0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2208005890 tx=0x7f220804f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9ed9e0836a33a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.527+0000 7f2205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2205ffb640 1 -- 172.21.15.5:0/2615202521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220804c2d0 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2205ffb640 1 -- 172.21.15.5:0/2615202521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7f220804c470 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220804ce30 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2205ffb640 1 -- 172.21.15.5:0/2615202521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f220804c670 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 10 client.18138 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55652a0c77c0 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2205ffb640 1 -- 172.21.15.5:0/2615202521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220804ce30 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2205ffb640 1 -- 172.21.15.5:0/2615202521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2208064450 con 0x55652a051a00 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 10 client.18138 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 20 client.18138 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a0c7e60 0x55652a0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 4 client.18138 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556529fbac10 con 0x55652a0c7e60 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.528+0000 7f221e59c640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a0c7e60 0x55652a0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f221e59c640 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a0c7e60 0x55652a0ca280 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5dd504f321561c1 server_cookie=697b354689e22e52 in_seq=0 out_seq=0 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2205ffb640 10 client.18138 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2205ffb640 1 -- 172.21.15.5:0/2615202521 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556529fbac10 con 0x55652a0c7e60 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2205ffb640 10 client.18138 handle_command_reply: tid=0 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 client.18138 shutdown 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 msgr2=0x55652a04aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a04aab0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2208005890 tx=0x7f220804f040 comp rx=0 tx=0).stop 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 shutdown_connections 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55652a0c7e60 0x55652a0ca280 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55652a051a00 0x55652a04aab0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2205ffb640 0 client.18138 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55652a048e80 0x55652a05a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 --2- 172.21.15.5:0/2615202521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55652a0484d0 0x55652a059d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 >> 172.21.15.5:0/2615202521 conn(0x55652a042ba0 msgr2=0x55652a04d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 shutdown_connections 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 1 -- 172.21.15.5:0/2615202521 wait complete. 2024-04-22T13:33:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.529+0000 7f2227785740 20 client.18138 trim_cache size 0 max 16384 2024-04-22T13:33:49.534 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f220c0a3d50 msgr2=0x7f220c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f220c0a3d50 0x7f220c0a6210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2218099280 tx=0x7f2218098510 comp rx=0 tx=0).stop 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 msgr2=0x7f22201a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 0x7f22201a6a40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f221800eaa0 tx=0x7f221800eea0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 shutdown_connections 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101890 0x7f22201a6a40 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f220c0a3d50 0x7f220c0a6210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f222010c6d0 0x7f2220101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 --2- 172.21.15.5:0/3959968454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22201031c0 0x7f2220100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 >> 172.21.15.5:0/3959968454 conn(0x7f2220077f70 msgr2=0x7f22200fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 shutdown_connections 2024-04-22T13:33:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:49.530+0000 7f2224aba640 1 -- 172.21.15.5:0/3959968454 wait complete. 2024-04-22T13:33:49.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:49 smithi107 ceph-mon[31859]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:49.537 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:49 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1126313445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:50.538 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:33:50.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:33:50.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:33:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 -- 172.21.15.5:0/3531001604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb720103b50 msgr2=0x7fb720103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 --2- 172.21.15.5:0/3531001604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb720103b50 0x7fb720103fb0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009930 tx=0x7fb70c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 -- 172.21.15.5:0/3531001604 shutdown_connections 2024-04-22T13:33:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 --2- 172.21.15.5:0/3531001604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb720103b50 0x7fb720103fb0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 --2- 172.21.15.5:0/3531001604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7201044f0 0x7fb72010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 --2- 172.21.15.5:0/3531001604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb720102270 0x7fb720102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 -- 172.21.15.5:0/3531001604 >> 172.21.15.5:0/3531001604 conn(0x7fb7200fda10 msgr2=0x7fb7200ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 -- 172.21.15.5:0/3531001604 shutdown_connections 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.662+0000 7fb7271fb640 1 -- 172.21.15.5:0/3531001604 wait complete. 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 Processor -- start 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 -- start start 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb720102270 0x7fb7201a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb720103b50 0x7fb7201a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 0x7fb72019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb720112290 con 0x7fb7201044f0 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb720112110 con 0x7fb720102270 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb7271fb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb720112410 con 0x7fb720103b50 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.663+0000 7fb725798640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 0x7fb72019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 0x7fb72019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40822/0 (socket says 172.21.15.5:40822) 2024-04-22T13:33:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 -- 172.21.15.5:0/2430194870 learned_addr learned my addr 172.21.15.5:0/2430194870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb724f97640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb720102270 0x7fb7201a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb71ffff640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb720103b50 0x7fb7201a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 -- 172.21.15.5:0/2430194870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb720103b50 msgr2=0x7fb7201a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb720103b50 0x7fb7201a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 -- 172.21.15.5:0/2430194870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb720102270 msgr2=0x7fb7201a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb720102270 0x7fb7201a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 -- 172.21.15.5:0/2430194870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72019afd0 con 0x7fb7201044f0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb725798640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 0x7fb72019a800 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb71800eaa0 tx=0x7fb71800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb71dffb640 1 -- 172.21.15.5:0/2430194870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71800cdb0 con 0x7fb7201044f0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb71dffb640 1 -- 172.21.15.5:0/2430194870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb718005020 con 0x7fb7201044f0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb71dffb640 1 -- 172.21.15.5:0/2430194870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180116d0 con 0x7fb7201044f0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72019b2c0 con 0x7fb7201044f0 2024-04-22T13:33:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.664+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7201a74e0 con 0x7fb7201044f0 2024-04-22T13:33:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb71dffb640 1 -- 172.21.15.5:0/2430194870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb7180051d0 con 0x7fb7201044f0 2024-04-22T13:33:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 Processor -- start 2024-04-22T13:33:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 -- start start 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55899e90bf00 con 0x55899e9a8290 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55899e8a9590 con 0x55899e9a78e0 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb729de1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55899e907ec0 con 0x55899e9b0e10 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb71dffb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7100a3d20 0x7fb7100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb724f97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb724f97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40832/0 (socket says 172.21.15.5:40832) 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb724f97640 1 -- 172.21.15.5:0/2666104811 learned_addr learned my addr 172.21.15.5:0/2666104811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.667+0000 7fb725798640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7100a3d20 0x7fb7100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb71ffff640 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb71dffb640 1 -- 172.21.15.5:0/2430194870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb7180d09e0 con 0x7fb7201044f0 2024-04-22T13:33:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb71ffff640 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb724f97640 1 -- 172.21.15.5:0/2666104811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 msgr2=0x55899e9b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb724f97640 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb724f97640 1 -- 172.21.15.5:0/2666104811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 msgr2=0x55899e9a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb724f97640 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb724f97640 1 -- 172.21.15.5:0/2666104811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899e6e67e0 con 0x55899e9a8290 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb71ffff640 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb724f97640 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9b0840 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb714009d20 tx=0x7fb714011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d644a48b135f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb725798640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7100a3d20 0x7fb7100a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7180ce210 tx=0x7fb7180bd040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb70affd640 1 -- 172.21.15.5:0/2666104811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71401e070 con 0x55899e9a8290 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb70affd640 1 -- 172.21.15.5:0/2666104811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714019ae0 con 0x55899e9a8290 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.668+0000 7fb70affd640 1 -- 172.21.15.5:0/2666104811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400ea60 con 0x55899e9a8290 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 -- 172.21.15.5:0/2666104811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 msgr2=0x55899e9b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9b0840 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb714009d20 tx=0x7fb714011810 comp rx=0 tx=0).stop 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 -- 172.21.15.5:0/2666104811 shutdown_connections 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9b0840 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 --2- 172.21.15.5:0/2666104811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 -- 172.21.15.5:0/2666104811 >> 172.21.15.5:0/2666104811 conn(0x55899e9a1fb0 msgr2=0x55899e9a2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 -- 172.21.15.5:0/2666104811 shutdown_connections 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 -- 172.21.15.5:0/2666104811 wait complete. 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 -- start start 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.669+0000 7fb729de1740 1 Processor -- start 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 0x55899e9addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9a8290 0x55899e9ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9b0e10 0x55899e9ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55899e9b3d80 con 0x55899e9a78e0 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55899e8c5580 con 0x55899e9b0e10 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55899e9b3f00 con 0x55899e9a8290 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb729de1740 10 client.0 _ll_register_callbacks cb 0x55899e8cb650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 0x55899e9addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 0x55899e9addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40840/0 (socket says 172.21.15.5:40840) 2024-04-22T13:33:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 -- 172.21.15.5:0/306894004 learned_addr learned my addr 172.21.15.5:0/306894004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb724f97640 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9a8290 0x55899e9ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 -- 172.21.15.5:0/306894004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9a8290 msgr2=0x55899e9ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9a8290 0x55899e9ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 -- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9b0e10 msgr2=0x55899e9ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9b0e10 0x55899e9ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 -- 172.21.15.5:0/306894004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899e746f70 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb71ffff640 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 0x55899e9addd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb70c005940 tx=0x7fb70c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f533e84e70fec4b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.670+0000 7fb6fffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb6fffff640 1 -- 172.21.15.5:0/306894004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c047020 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb6fffff640 1 -- 172.21.15.5:0/306894004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb70c02c9f0 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb714019ae0 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb6fffff640 1 -- 172.21.15.5:0/306894004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb70c052050 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 10 client.18150 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55899e5de170 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb6fffff640 1 -- 172.21.15.5:0/306894004 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714019ae0 con 0x55899e9a78e0 2024-04-22T13:33:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb6fffff640 1 -- 172.21.15.5:0/306894004 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb70c040040 con 0x55899e9a78e0 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 10 client.18150 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 20 client.18150 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899e9c1360 0x55899e9c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 4 client.18150 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.671+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899e919920 con 0x55899e9c1360 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.672+0000 7fb724f97640 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899e9c1360 0x55899e9c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.672+0000 7fb724f97640 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899e9c1360 0x55899e9c3770 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3741c2831f8bfed8 server_cookie=b81ffb6996e8730d in_seq=0 out_seq=0 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.672+0000 7fb6fffff640 10 client.18150 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb6fffff640 1 -- 172.21.15.5:0/306894004 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55899e919920 con 0x55899e9c1360 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb6fffff640 10 client.18150 handle_command_reply: tid=0 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 client.18150 shutdown 2024-04-22T13:33:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 msgr2=0x55899e9addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 0x55899e9addd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb70c005940 tx=0x7fb70c0045f0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 shutdown_connections 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899e9c1360 0x55899e9c3770 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9b0e10 0x55899e9ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9a8290 0x55899e9ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb6fffff640 0 client.18150 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 --2- 172.21.15.5:0/306894004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a78e0 0x55899e9addd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 >> 172.21.15.5:0/306894004 conn(0x55899e9a1fb0 msgr2=0x55899e9a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 shutdown_connections 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 1 -- 172.21.15.5:0/306894004 wait complete. 2024-04-22T13:33:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.673+0000 7fb729de1740 20 client.18150 trim_cache size 0 max 16384 2024-04-22T13:33:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 Processor -- start 2024-04-22T13:33:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 -- start start 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201afca0 con 0x55899e9a8290 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b17e0 con 0x55899e9a78e0 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb729de1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b0370 con 0x55899e9b0e10 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9bcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40848/0 (socket says 172.21.15.5:40848) 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 -- 172.21.15.5:0/35516159 learned_addr learned my addr 172.21.15.5:0/35516159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb71ffff640 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb71ffff640 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 -- 172.21.15.5:0/35516159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 msgr2=0x55899e9a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 -- 172.21.15.5:0/35516159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 msgr2=0x55899e9b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.692+0000 7fb724f97640 1 -- 172.21.15.5:0/35516159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899e99d880 con 0x55899e9a8290 2024-04-22T13:33:50.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.693+0000 7fb724f97640 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9bcc70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb714009d20 tx=0x7fb7140096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20acb19e50cfd67c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.693+0000 7fb70a7fc640 1 -- 172.21.15.5:0/35516159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71401e040 con 0x55899e9a8290 2024-04-22T13:33:50.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.693+0000 7fb70a7fc640 1 -- 172.21.15.5:0/35516159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714009780 con 0x55899e9a8290 2024-04-22T13:33:50.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.693+0000 7fb70a7fc640 1 -- 172.21.15.5:0/35516159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400e8c0 con 0x55899e9a8290 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.693+0000 7fb729de1740 1 -- 172.21.15.5:0/35516159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 msgr2=0x55899e9bcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.693+0000 7fb729de1740 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9bcc70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb714009d20 tx=0x7fb7140096d0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- 172.21.15.5:0/35516159 shutdown_connections 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9bcc70 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899e9a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 --2- 172.21.15.5:0/35516159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- 172.21.15.5:0/35516159 >> 172.21.15.5:0/35516159 conn(0x55899e9a1fb0 msgr2=0x55899e9a3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- 172.21.15.5:0/35516159 shutdown_connections 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- 172.21.15.5:0/35516159 wait complete. 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- start start 2024-04-22T13:33:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 Processor -- start 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899ea268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b1110 con 0x55899e9a8290 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b0a40 con 0x55899e9a78e0 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201adc80 con 0x55899e9b0e10 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb729de1740 10 client.0 _ll_register_callbacks cb 0x7fb714005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb724f97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb724f97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40850/0 (socket says 172.21.15.5:40850) 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.694+0000 7fb724f97640 1 -- 172.21.15.5:0/977491666 learned_addr learned my addr 172.21.15.5:0/977491666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb71ffff640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb71ffff640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899ea268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb724f97640 1 -- 172.21.15.5:0/977491666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 msgr2=0x55899ea268b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb724f97640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899ea268b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb724f97640 1 -- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 msgr2=0x55899e9aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb724f97640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb724f97640 1 -- 172.21.15.5:0/977491666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899e9abbc0 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb724f97640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9ab2f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb71400a010 tx=0x7fb714019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0db100f38a6358 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb6fffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb6fffff640 1 -- 172.21.15.5:0/977491666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71402d040 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb6fffff640 1 -- 172.21.15.5:0/977491666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fb7140050d0 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb714009780 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb6fffff640 1 -- 172.21.15.5:0/977491666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb71401e040 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb729de1740 10 client.18156 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55899ea27f00 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.695+0000 7fb6fffff640 1 -- 172.21.15.5:0/977491666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714009780 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb6fffff640 1 -- 172.21.15.5:0/977491666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb71400e4d0 con 0x55899e9a8290 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb729de1740 10 client.18156 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb729de1740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb729de1740 20 client.18156 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb729de1740 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899ea283a0 0x55899ea2a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb729de1740 4 client.18156 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55899e913790 con 0x55899ea283a0 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb71ffff640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899ea283a0 0x55899ea2a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb71ffff640 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899ea283a0 0x55899ea2a7c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8156440d07c4a2f8 server_cookie=e67064ba200c0a46 in_seq=0 out_seq=0 2024-04-22T13:33:50.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.696+0000 7fb6fffff640 10 client.18156 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb6fffff640 1 -- 172.21.15.5:0/977491666 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55899e913790 con 0x55899ea283a0 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb6fffff640 10 client.18156 handle_command_reply: tid=0 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 client.18156 shutdown 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 msgr2=0x55899e9ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9ab2f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb71400a010 tx=0x7fb714019530 comp rx=0 tx=0).stop 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 shutdown_connections 2024-04-22T13:33:50.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55899ea283a0 0x55899ea2a7c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55899e9a8290 0x55899e9ab2f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55899e9b0e10 0x55899ea268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 --2- 172.21.15.5:0/977491666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55899e9a78e0 0x55899e9aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb6fffff640 0 client.18156 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 >> 172.21.15.5:0/977491666 conn(0x55899e9a1fb0 msgr2=0x55899e9a3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 shutdown_connections 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 1 -- 172.21.15.5:0/977491666 wait complete. 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.698+0000 7fb729de1740 20 client.18156 trim_cache size 0 max 16384 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:33:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:33:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8a5a21a6-7d90-49df-9841-c5ae63ad2770", 2024-04-22T13:33:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:33:50.699 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:33:50.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7100a3d20 msgr2=0x7fb7100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7100a3d20 0x7fb7100a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7180ce210 tx=0x7fb7180bd040 comp rx=0 tx=0).stop 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 msgr2=0x7fb72019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 0x7fb72019a800 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb71800eaa0 tx=0x7fb71800ef60 comp rx=0 tx=0).stop 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 shutdown_connections 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7201044f0 0x7fb72019a800 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb7100a3d20 0x7fb7100a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb720103b50 0x7fb7201a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 --2- 172.21.15.5:0/2430194870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb720102270 0x7fb7201a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 >> 172.21.15.5:0/2430194870 conn(0x7fb7200fda10 msgr2=0x7fb72010d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 shutdown_connections 2024-04-22T13:33:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.699+0000 7fb7271fb640 1 -- 172.21.15.5:0/2430194870 wait complete. 2024-04-22T13:33:50.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:50.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 -- 172.21.15.5:0/1133413822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 msgr2=0x7fa15810dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 --2- 172.21.15.5:0/1133413822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa15810dcf0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ca0 tx=0x7fa15002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 -- 172.21.15.5:0/1133413822 shutdown_connections 2024-04-22T13:33:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 --2- 172.21.15.5:0/1133413822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa15810dcf0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 --2- 172.21.15.5:0/1133413822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa158101a00 0x7fa158105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 --2- 172.21.15.5:0/1133413822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa158101050 0x7fa158101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 -- 172.21.15.5:0/1133413822 >> 172.21.15.5:0/1133413822 conn(0x7fa158077fa0 msgr2=0x7fa1580783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 -- 172.21.15.5:0/1133413822 shutdown_connections 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 -- 172.21.15.5:0/1133413822 wait complete. 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 Processor -- start 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.815+0000 7fa15f826640 1 -- start start 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa158101050 0x7fa1581986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa158198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa15819cf50 0x7fa15819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581121a0 con 0x7fa158106120 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa158112020 con 0x7fa158101050 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa158112320 con 0x7fa15819cf50 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa158198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa158198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40862/0 (socket says 172.21.15.5:40862) 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 -- 172.21.15.5:0/1854087535 learned_addr learned my addr 172.21.15.5:0/1854087535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa158101050 0x7fa1581986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa15819cf50 0x7fa15819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 -- 172.21.15.5:0/1854087535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa15819cf50 msgr2=0x7fa15819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa15819cf50 0x7fa15819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 -- 172.21.15.5:0/1854087535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa158101050 msgr2=0x7fa1581986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa158101050 0x7fa1581986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 -- 172.21.15.5:0/1854087535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15819da40 con 0x7fa158106120 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa158198c20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d770 tx=0x7fa14800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa14e7fc640 1 -- 172.21.15.5:0/1854087535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148017070 con 0x7fa158106120 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa14e7fc640 1 -- 172.21.15.5:0/1854087535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa148009c30 con 0x7fa158106120 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15819dd30 con 0x7fa158106120 2024-04-22T13:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa14e7fc640 1 -- 172.21.15.5:0/1854087535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14800f620 con 0x7fa158106120 2024-04-22T13:33:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.816+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1581a5520 con 0x7fa158106120 2024-04-22T13:33:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa14e7fc640 1 -- 172.21.15.5:0/1854087535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa14801a070 con 0x7fa158106120 2024-04-22T13:33:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 Processor -- start 2024-04-22T13:33:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 -- start start 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a276d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a276271e0 con 0x559a276c2bd0 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a275c4390 con 0x559a276c22b0 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa14e7fc640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1440a3d20 0x7fa1440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.819+0000 7fa162410740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a276231a0 con 0x559a276cfe90 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40876/0 (socket says 172.21.15.5:40876) 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15ddc3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 -- 172.21.15.5:0/1206371110 learned_addr learned my addr 172.21.15.5:0/1206371110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a276d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1440a3d20 0x7fa1440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa14e7fc640 1 -- 172.21.15.5:0/1854087535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa1480d02a0 con 0x7fa158106120 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 -- 172.21.15.5:0/1206371110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 msgr2=0x559a276d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a276d2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 -- 172.21.15.5:0/1206371110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 msgr2=0x559a276c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 -- 172.21.15.5:0/1206371110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a274037d0 con 0x559a276c2bd0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cf950 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa154009d20 tx=0x7fa154011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13725b872421a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1440a3d20 0x7fa1440a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa150004af0 tx=0x7fa15002c0f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa1437fe640 1 -- 172.21.15.5:0/1206371110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15401e070 con 0x559a276c2bd0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa1437fe640 1 -- 172.21.15.5:0/1206371110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa154019ae0 con 0x559a276c2bd0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.820+0000 7fa1437fe640 1 -- 172.21.15.5:0/1206371110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15400eac0 con 0x559a276c2bd0 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 -- 172.21.15.5:0/1206371110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 msgr2=0x559a276cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cf950 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa154009d20 tx=0x7fa154011810 comp rx=0 tx=0).stop 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 -- 172.21.15.5:0/1206371110 shutdown_connections 2024-04-22T13:33:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cf950 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a276d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 --2- 172.21.15.5:0/1206371110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 -- 172.21.15.5:0/1206371110 >> 172.21.15.5:0/1206371110 conn(0x559a276bccb0 msgr2=0x559a276bd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 -- 172.21.15.5:0/1206371110 shutdown_connections 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 -- 172.21.15.5:0/1206371110 wait complete. 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 -- start start 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.821+0000 7fa162410740 1 Processor -- start 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c2bd0 0x559a276cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 0x559a276cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a276d2c50 con 0x559a276cfe90 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a275e02f0 con 0x559a276c22b0 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a276d2dd0 con 0x559a276c2bd0 2024-04-22T13:33:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa162410740 10 client.0 _ll_register_callbacks cb 0x559a275ed580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 0x559a276cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 0x559a276cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40882/0 (socket says 172.21.15.5:40882) 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 -- 172.21.15.5:0/2032961207 learned_addr learned my addr 172.21.15.5:0/2032961207 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c2bd0 0x559a276cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 -- 172.21.15.5:0/2032961207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c2bd0 msgr2=0x559a276cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c2bd0 0x559a276cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 -- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 msgr2=0x559a276cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 -- 172.21.15.5:0/2032961207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a273393f0 con 0x559a276cfe90 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 0x559a276cd930 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa150036000 tx=0x7fa1500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510543e49d44e741 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.822+0000 7fa140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:50.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa140ff9640 1 -- 172.21.15.5:0/2032961207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150054060 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa154019ae0 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa140ff9640 1 -- 172.21.15.5:0/2032961207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa150003b80 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa140ff9640 1 -- 172.21.15.5:0/2032961207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa150039070 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 10 client.18168 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a272f90f0 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa140ff9640 1 -- 172.21.15.5:0/2032961207 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa154019ae0 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa140ff9640 1 -- 172.21.15.5:0/2032961207 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa15004f470 con 0x559a276cfe90 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 10 client.18168 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 20 client.18168 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a276dfec0 0x559a276e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 4 client.18168 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a27634c00 con 0x559a276dfec0 2024-04-22T13:33:50.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.823+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a276dfec0 0x559a276e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.824+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a276dfec0 0x559a276e2330 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63024eb4cd1f6a5 server_cookie=be84c4a5830458f4 in_seq=0 out_seq=0 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.824+0000 7fa140ff9640 10 client.18168 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.824+0000 7fa140ff9640 1 -- 172.21.15.5:0/2032961207 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a27634c00 con 0x559a276dfec0 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.824+0000 7fa140ff9640 10 client.18168 handle_command_reply: tid=0 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 client.18168 shutdown 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 msgr2=0x559a276cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 0x559a276cd930 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa150036000 tx=0x7fa1500095c0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 shutdown_connections 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a276dfec0 0x559a276e2330 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276cfe90 0x559a276cd930 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c2bd0 0x559a276cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 --2- 172.21.15.5:0/2032961207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c22b0 0x559a276cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 >> 172.21.15.5:0/2032961207 conn(0x559a276bccb0 msgr2=0x559a276bea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa140ff9640 0 client.18168 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 shutdown_connections 2024-04-22T13:33:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 1 -- 172.21.15.5:0/2032961207 wait complete. 2024-04-22T13:33:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.825+0000 7fa162410740 20 client.18168 trim_cache size 0 max 16384 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.842+0000 7fa162410740 1 Processor -- start 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 -- start start 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c22b0 0x559a276cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276cfe90 0x559a276dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581ab8e0 con 0x559a276c2bd0 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581ad420 con 0x559a276cfe90 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa162410740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581abfb0 con 0x559a276c22b0 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40892/0 (socket says 172.21.15.5:40892) 2024-04-22T13:33:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 -- 172.21.15.5:0/2677405593 learned_addr learned my addr 172.21.15.5:0/2677405593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c22b0 0x559a276cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 -- 172.21.15.5:0/2677405593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c22b0 msgr2=0x559a276cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c22b0 0x559a276cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 -- 172.21.15.5:0/2677405593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276cfe90 msgr2=0x559a276dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276cfe90 0x559a276dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 -- 172.21.15.5:0/2677405593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a276c7c60 con 0x559a276c2bd0 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c22b0 0x559a276cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cbdf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa148000c00 tx=0x7fa148016b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8585829fea9c46c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa142ffd640 1 -- 172.21.15.5:0/2677405593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1480cf040 con 0x559a276c2bd0 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa142ffd640 1 -- 172.21.15.5:0/2677405593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa148009de0 con 0x559a276c2bd0 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.843+0000 7fa142ffd640 1 -- 172.21.15.5:0/2677405593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148021040 con 0x559a276c2bd0 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 -- 172.21.15.5:0/2677405593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 msgr2=0x559a276cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cbdf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa148000c00 tx=0x7fa148016b40 comp rx=0 tx=0).stop 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 -- 172.21.15.5:0/2677405593 shutdown_connections 2024-04-22T13:33:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c2bd0 0x559a276cbdf0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276cfe90 0x559a276dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 --2- 172.21.15.5:0/2677405593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276c22b0 0x559a276cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 -- 172.21.15.5:0/2677405593 >> 172.21.15.5:0/2677405593 conn(0x559a276bccb0 msgr2=0x559a276be970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 -- 172.21.15.5:0/2677405593 shutdown_connections 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 -- 172.21.15.5:0/2677405593 wait complete. 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 -- start start 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.844+0000 7fa162410740 1 Processor -- start 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 0x559a276d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c2bd0 0x559a276d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a27745470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581acd50 con 0x559a276c22b0 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581ac680 con 0x559a276c2bd0 2024-04-22T13:33:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581a98c0 con 0x559a276cfe90 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa162410740 10 client.0 _ll_register_callbacks cb 0x7fa154005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 0x559a276d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 0x559a276d40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40898/0 (socket says 172.21.15.5:40898) 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 -- 172.21.15.5:0/3555940538 learned_addr learned my addr 172.21.15.5:0/3555940538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c2bd0 0x559a276d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a27745470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 -- 172.21.15.5:0/3555940538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 msgr2=0x559a27745470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a27745470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 -- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c2bd0 msgr2=0x559a276d4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c2bd0 0x559a276d4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 -- 172.21.15.5:0/3555940538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a277459b0 con 0x559a276c22b0 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15ddc3640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a27745470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c2bd0 0x559a276d4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa15d5c2640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 0x559a276d40f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa15401c040 tx=0x7fa154005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffcea4c4b019423 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.845+0000 7fa140ff9640 1 -- 172.21.15.5:0/3555940538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15401e040 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa140ff9640 1 -- 172.21.15.5:0/3555940538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8469+0+0 (secure 0 0 0) 0x7fa15401ad20 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa148009de0 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa140ff9640 1 -- 172.21.15.5:0/3555940538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1540050b0 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 10 client.18174 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a27746b90 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa140ff9640 1 -- 172.21.15.5:0/3555940538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa148009de0 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa140ff9640 1 -- 172.21.15.5:0/3555940538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa15400ec60 con 0x559a276c22b0 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 10 client.18174 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 20 client.18174 populate_metadata read hostname 'smithi005' 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a27747150 0x559a27749590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 4 client.18174 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a2762ea70 con 0x559a27747150 2024-04-22T13:33:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.846+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a27747150 0x559a27749590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa15cdc1640 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a27747150 0x559a27749590 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2106101da6006633 server_cookie=27cc0c42eb42807d in_seq=0 out_seq=0 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa140ff9640 10 client.18174 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa140ff9640 1 -- 172.21.15.5:0/3555940538 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559a2762ea70 con 0x559a27747150 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa140ff9640 10 client.18174 handle_command_reply: tid=0 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 client.18174 shutdown 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 msgr2=0x559a276d40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 0x559a276d40f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa15401c040 tx=0x7fa154005f10 comp rx=0 tx=0).stop 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 shutdown_connections 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a27747150 0x559a27749590 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a276cfe90 0x559a27745470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa140ff9640 0 client.18174 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a276c2bd0 0x559a276d4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 --2- 172.21.15.5:0/3555940538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a276c22b0 0x559a276d40f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 >> 172.21.15.5:0/3555940538 conn(0x559a276bccb0 msgr2=0x559a276bb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 shutdown_connections 2024-04-22T13:33:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 1 -- 172.21.15.5:0/3555940538 wait complete. 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.847+0000 7fa162410740 20 client.18174 trim_cache size 0 max 16384 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "8a5a21a6-7d90-49df-9841-c5ae63ad2770": { 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8a5a21a6-7d90-49df-9841-c5ae63ad2770", 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:33:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:33:50.851 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:33:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1440a3d20 msgr2=0x7fa1440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1440a3d20 0x7fa1440a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa150004af0 tx=0x7fa15002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 msgr2=0x7fa158198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa158198c20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d770 tx=0x7fa14800dc30 comp rx=0 tx=0).stop 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 shutdown_connections 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa158106120 0x7fa158198c20 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa1440a3d20 0x7fa1440a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa15819cf50 0x7fa15819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 --2- 172.21.15.5:0/1854087535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa158101050 0x7fa1581986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 >> 172.21.15.5:0/1854087535 conn(0x7fa158077fa0 msgr2=0x7fa1580feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 shutdown_connections 2024-04-22T13:33:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:33:50.848+0000 7fa15f826640 1 -- 172.21.15.5:0/1854087535 wait complete. 2024-04-22T13:33:50.854 INFO:tasks.cephfs.filesystem:scrub status for tag:8a5a21a6-7d90-49df-9841-c5ae63ad2770 - {'path': '/', 'tag': '8a5a21a6-7d90-49df-9841-c5ae63ad2770', 'options': 'recursive,force'} 2024-04-22T13:33:51.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:51 smithi049 ceph-mon[31379]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:51.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:51 smithi107 ceph-mon[31859]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:51.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:51 smithi005 ceph-mon[28006]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:52.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:52 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:33:52.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:52 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:52.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:52 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:33:52.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:52 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:52.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:52 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:33:52.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:52 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:52.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:52 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:33:52.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:52 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:52.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:52 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:33:52.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:52 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:52.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:52 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:33:52.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:52 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:53.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:53 smithi049 ceph-mon[31379]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:53.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:53 smithi107 ceph-mon[31859]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:53.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:53 smithi005 ceph-mon[28006]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:55.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:55 smithi049 ceph-mon[31379]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:55.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:55 smithi107 ceph-mon[31859]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:55.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:55 smithi005 ceph-mon[28006]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:56.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:56 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:33:56.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:56 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:33:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:56 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:33:57.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:57 smithi049 ceph-mon[31379]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:57.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:57 smithi107 ceph-mon[31859]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:57 smithi005 ceph-mon[28006]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:59.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:33:59 smithi049 ceph-mon[31379]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:59.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:33:59 smithi107 ceph-mon[31859]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:59.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:33:59 smithi005 ceph-mon[28006]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:01.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:01 smithi049 ceph-mon[31379]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:01.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:01 smithi107 ceph-mon[31859]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:01 smithi005 ceph-mon[28006]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:03.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:03 smithi049 ceph-mon[31379]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:34:03.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:03 smithi107 ceph-mon[31859]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:34:03.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:03 smithi005 ceph-mon[28006]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:34:05.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:05 smithi049 ceph-mon[31379]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:05.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:05 smithi107 ceph-mon[31859]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:05 smithi005 ceph-mon[28006]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:07.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:07 smithi049 ceph-mon[31379]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:34:07.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:07 smithi107 ceph-mon[31859]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:34:07.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:07 smithi005 ceph-mon[28006]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:34:09.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:09 smithi049 ceph-mon[31379]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:09.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:09 smithi107 ceph-mon[31859]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:09.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:09 smithi005 ceph-mon[28006]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:11.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:11 smithi049 ceph-mon[31379]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:11.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:11 smithi107 ceph-mon[31859]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:11.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:11 smithi005 ceph-mon[28006]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:12.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:12 smithi049 ceph-mon[31379]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T13:34:12.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:12 smithi107 ceph-mon[31859]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T13:34:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:12 smithi005 ceph-mon[28006]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T13:34:13.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:13 smithi049 ceph-mon[31379]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:34:13.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:13 smithi107 ceph-mon[31859]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:34:13.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:13 smithi005 ceph-mon[28006]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:34:15.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:15 smithi049 ceph-mon[31379]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:34:15.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:15 smithi107 ceph-mon[31859]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:34:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:15 smithi005 ceph-mon[28006]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:34:16.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:16 smithi049 ceph-mon[31379]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:16.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:16 smithi107 ceph-mon[31859]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:16.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:16 smithi005 ceph-mon[28006]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:18.770 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:18 smithi049 ceph-mon[31379]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T13:34:18.770 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:18 smithi049 ceph-mon[31379]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:34:18.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:18 smithi107 ceph-mon[31859]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T13:34:18.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:18 smithi107 ceph-mon[31859]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:34:18.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:18 smithi005 ceph-mon[28006]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T13:34:18.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:18 smithi005 ceph-mon[28006]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:34:20.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:20.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:20.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:20 smithi005 ceph-mon[28006]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 121 B/s wr, 6 op/s 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 -- 172.21.15.5:0/1980086620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 msgr2=0x7f883c10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 --2- 172.21.15.5:0/1980086620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c10e3b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8830009860 tx=0x7f883002bc00 comp rx=0 tx=0).stop 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 -- 172.21.15.5:0/1980086620 shutdown_connections 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 --2- 172.21.15.5:0/1980086620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f883c10e8f0 0x7f883c110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 --2- 172.21.15.5:0/1980086620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c10e3b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 --2- 172.21.15.5:0/1980086620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c0692f0 0x7f883c0696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 -- 172.21.15.5:0/1980086620 >> 172.21.15.5:0/1980086620 conn(0x7f883c077fa0 msgr2=0x7f883c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 -- 172.21.15.5:0/1980086620 shutdown_connections 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.966+0000 7f8842715640 1 -- 172.21.15.5:0/1980086620 wait complete. 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 Processor -- start 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 -- start start 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f883c0692f0 0x7f883c1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 0x7f883c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f883c114330 con 0x7f883c10e8f0 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1141b0 con 0x7f883c0692f0 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8842715640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1144b0 con 0x7f883c069c10 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8840cb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 0x7f883c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8840cb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 0x7f883c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36738/0 (socket says 172.21.15.5:36738) 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f8840cb2640 1 -- 172.21.15.5:0/1934238134 learned_addr learned my addr 172.21.15.5:0/1934238134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883bfff640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f883c0692f0 0x7f883c1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883b7fe640 1 -- 172.21.15.5:0/1934238134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f883c0692f0 msgr2=0x7f883c1a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883b7fe640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f883c0692f0 0x7f883c1a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883b7fe640 1 -- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 msgr2=0x7f883c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883b7fe640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 0x7f883c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.967+0000 7f883b7fe640 1 -- 172.21.15.5:0/1934238134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c19af30 con 0x7f883c069c10 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f8840cb2640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 0x7f883c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f883b7fe640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c1a0af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8830037040 tx=0x7f8830004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f88397fa640 1 -- 172.21.15.5:0/1934238134 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830041020 con 0x7f883c069c10 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f88397fa640 1 -- 172.21.15.5:0/1934238134 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883002ca50 con 0x7f883c069c10 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c19b1c0 con 0x7f883c069c10 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f88397fa640 1 -- 172.21.15.5:0/1934238134 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830039040 con 0x7f883c069c10 2024-04-22T13:34:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.968+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1a7530 con 0x7f883c069c10 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.970+0000 7f88397fa640 1 -- 172.21.15.5:0/1934238134 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8830044020 con 0x7f883c069c10 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.970+0000 7f88453e1740 1 Processor -- start 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 -- start start 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ea4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409ead540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eafe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564409e091e0 con 0x564409eada80 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564409da6390 con 0x564409ea4ea0 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88453e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564409e051a0 con 0x564409ea4580 2024-04-22T13:34:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88397fa640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88240a3da0 0x7f88240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409ead540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eafe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409ead540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:42914/0 (socket says 172.21.15.5:42914) 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f8840cb2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ea4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 -- 172.21.15.5:0/2928707356 learned_addr learned my addr 172.21.15.5:0/2928707356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f8840cb2640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88240a3da0 0x7f88240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f88397fa640 1 -- 172.21.15.5:0/1934238134 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f88300f1af0 con 0x7f883c069c10 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 -- 172.21.15.5:0/2928707356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 msgr2=0x564409ea4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ea4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 -- 172.21.15.5:0/2928707356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 msgr2=0x564409eafe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eafe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f883bfff640 1 -- 172.21.15.5:0/2928707356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564409be57d0 con 0x564409ea4ea0 2024-04-22T13:34:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.971+0000 7f8840cb2640 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ea4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f883bfff640 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409ead540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f882c009cd0 tx=0x7f882c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce6cf20deb5e8647 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f8840cb2640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88240a3da0 0x7f88240a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88340049d0 tx=0x7f8834009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88227fc640 1 -- 172.21.15.5:0/2928707356 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c01e070 con 0x564409ea4ea0 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88227fc640 1 -- 172.21.15.5:0/2928707356 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882c019ae0 con 0x564409ea4ea0 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88227fc640 1 -- 172.21.15.5:0/2928707356 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c00e920 con 0x564409ea4ea0 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 -- 172.21.15.5:0/2928707356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 msgr2=0x564409ead540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409ead540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f882c009cd0 tx=0x7f882c011810 comp rx=0 tx=0).stop 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 -- 172.21.15.5:0/2928707356 shutdown_connections 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eafe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409ead540 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 --2- 172.21.15.5:0/2928707356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ea4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.972+0000 7f88453e1740 1 -- 172.21.15.5:0/2928707356 >> 172.21.15.5:0/2928707356 conn(0x564409e9ecb0 msgr2=0x564409e9f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 -- 172.21.15.5:0/2928707356 shutdown_connections 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 -- 172.21.15.5:0/2928707356 wait complete. 2024-04-22T13:34:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 -- start start 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 Processor -- start 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409eaaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409eab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564409eb0870 con 0x564409eada80 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564409dc22f0 con 0x564409ea4ea0 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564409eb09f0 con 0x564409ea4580 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f88453e1740 10 client.0 _ll_register_callbacks cb 0x564409dcf580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f8840cb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.973+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409eab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409eab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:42928/0 (socket says 172.21.15.5:42928) 2024-04-22T13:34:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f8840cb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36754/0 (socket says 172.21.15.5:36754) 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409eaaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883b7fe640 1 -- 172.21.15.5:0/2219822360 learned_addr learned my addr 172.21.15.5:0/2219822360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 -- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 msgr2=0x564409eab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409eab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 -- 172.21.15.5:0/2219822360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 msgr2=0x564409eab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 -- 172.21.15.5:0/2219822360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564409b1b3f0 con 0x564409ea4580 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f883bfff640 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409eaaad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f882c009230 tx=0x7f882c009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e62fba3cf6f913 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f881b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f881b7fe640 1 -- 172.21.15.5:0/2219822360 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c028020 con 0x564409ea4580 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f881b7fe640 1 -- 172.21.15.5:0/2219822360 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f882c005240 con 0x564409ea4580 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c019ae0 con 0x564409ea4580 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f881b7fe640 1 -- 172.21.15.5:0/2219822360 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f882c033050 con 0x564409ea4580 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f88453e1740 10 client.27442 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.974+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564409adb0f0 con 0x564409ea4580 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f881b7fe640 1 -- 172.21.15.5:0/2219822360 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882c019ae0 con 0x564409ea4580 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f881b7fe640 1 -- 172.21.15.5:0/2219822360 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f882c021030 con 0x564409ea4580 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f88453e1740 10 client.27442 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f88453e1740 10 client.27442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f88453e1740 20 client.27442 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f88453e1740 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409ebfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f88453e1740 4 client.27442 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564409e16c00 con 0x564409ebd9d0 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f883b7fe640 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409ebfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f883b7fe640 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409ebfdc0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9305ffb591982d51 server_cookie=9a2b5c5040c84a74 in_seq=0 out_seq=0 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.975+0000 7f881b7fe640 10 client.27442 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f881b7fe640 1 -- 172.21.15.5:0/2219822360 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564409e16c00 con 0x564409ebd9d0 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f881b7fe640 10 client.27442 handle_command_reply: tid=0 2024-04-22T13:34:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 client.27442 shutdown 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 msgr2=0x564409eaaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409eaaad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f882c009230 tx=0x7f882c009460 comp rx=0 tx=0).stop 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 shutdown_connections 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409ebfdc0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409eab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4ea0 0x564409eab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f881b7fe640 0 client.27442 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 --2- 172.21.15.5:0/2219822360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409eaaad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 >> 172.21.15.5:0/2219822360 conn(0x564409e9ecb0 msgr2=0x564409ea0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 shutdown_connections 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.976+0000 7f88453e1740 1 -- 172.21.15.5:0/2219822360 wait complete. 2024-04-22T13:34:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.977+0000 7f88453e1740 20 client.27442 trim_cache size 0 max 16384 2024-04-22T13:34:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 Processor -- start 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 -- start start 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ead200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409ea4ea0 0x564409eb9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409eada80 0x564409ec5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1afd50 con 0x564409ea4ea0 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1b1890 con 0x564409eada80 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f88453e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1b0420 con 0x564409ea4580 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f883bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ead200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409ea4ea0 0x564409eb9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f883bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ead200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51624/0 (socket says 172.21.15.5:51624) 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f883bfff640 1 -- 172.21.15.5:0/2693592574 learned_addr learned my addr 172.21.15.5:0/2693592574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.994+0000 7f8840cb2640 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409eada80 0x564409ec5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f883bfff640 1 -- 172.21.15.5:0/2693592574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409eada80 msgr2=0x564409ec5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f883bfff640 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409eada80 0x564409ec5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f883bfff640 1 -- 172.21.15.5:0/2693592574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409ea4ea0 msgr2=0x564409eb9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f883bfff640 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409ea4ea0 0x564409eb9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f883bfff640 1 -- 172.21.15.5:0/2693592574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564409e9fcb0 con 0x564409ea4580 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f883bfff640 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ead200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f882c0059f0 tx=0x7f882c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25555bf6f8bca3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f8821ffb640 1 -- 172.21.15.5:0/2693592574 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c028040 con 0x564409ea4580 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f8821ffb640 1 -- 172.21.15.5:0/2693592574 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882c0097e0 con 0x564409ea4580 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.995+0000 7f8821ffb640 1 -- 172.21.15.5:0/2693592574 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c01e040 con 0x564409ea4580 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 -- 172.21.15.5:0/2693592574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 msgr2=0x564409ead200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ead200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f882c0059f0 tx=0x7f882c005850 comp rx=0 tx=0).stop 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 -- 172.21.15.5:0/2693592574 shutdown_connections 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409ea4ea0 0x564409eb9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409eada80 0x564409ec5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 --2- 172.21.15.5:0/2693592574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4580 0x564409ead200 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 -- 172.21.15.5:0/2693592574 >> 172.21.15.5:0/2693592574 conn(0x564409e9ecb0 msgr2=0x564409ea0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 -- 172.21.15.5:0/2693592574 shutdown_connections 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 -- 172.21.15.5:0/2693592574 wait complete. 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 -- start start 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.996+0000 7f88453e1740 1 Processor -- start 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4580 0x564409eb5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 0x564409ea7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409ea7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1b11c0 con 0x564409eada80 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1b0af0 con 0x564409ea4580 2024-04-22T13:34:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1add50 con 0x564409ea4ea0 2024-04-22T13:34:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 10 client.0 _ll_register_callbacks cb 0x7f882c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4580 0x564409eb5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 0x564409ea7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4580 0x564409eb5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:42940/0 (socket says 172.21.15.5:42940) 2024-04-22T13:34:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883bfff640 1 -- 172.21.15.5:0/359296708 learned_addr learned my addr 172.21.15.5:0/359296708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 0x564409ea7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51636/0 (socket says 172.21.15.5:51636) 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f8840cb2640 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409ea7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 -- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4580 msgr2=0x564409eb5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4580 0x564409eb5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 -- 172.21.15.5:0/359296708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 msgr2=0x564409ea7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409ea7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 -- 172.21.15.5:0/359296708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564409ea7df0 con 0x564409ea4ea0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f883b7fe640 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 0x564409ea7340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f88300ba570 tx=0x7f883008d480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aff3fe3eb7c8e70 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f881b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f881b7fe640 1 -- 172.21.15.5:0/359296708 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300886b0 con 0x564409ea4ea0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f881b7fe640 1 -- 172.21.15.5:0/359296708 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f88300de0b0 con 0x564409ea4ea0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.997+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c0097e0 con 0x564409ea4ea0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f881b7fe640 1 -- 172.21.15.5:0/359296708 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88300be2a0 con 0x564409ea4ea0 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 10 client.27448 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564409f23b90 con 0x564409ea4ea0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f881b7fe640 1 -- 172.21.15.5:0/359296708 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882c0097e0 con 0x564409ea4ea0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f881b7fe640 1 -- 172.21.15.5:0/359296708 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f883003c030 con 0x564409ea4ea0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 10 client.27448 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 10 client.27448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 20 client.27448 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409f26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 4 client.27448 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564409e10a70 con 0x564409ebd9d0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.998+0000 7f883bfff640 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409f26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f881b7fe640 10 client.27448 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f883bfff640 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409f26300 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afdb5ad414eba67 server_cookie=d73bd94e57836975 in_seq=0 out_seq=0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f881b7fe640 1 -- 172.21.15.5:0/359296708 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564409e10a70 con 0x564409ebd9d0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f881b7fe640 10 client.27448 handle_command_reply: tid=0 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 client.27448 shutdown 2024-04-22T13:34:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 msgr2=0x564409ea7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 0x564409ea7340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f88300ba570 tx=0x7f883008d480 comp rx=0 tx=0).stop 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 shutdown_connections 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x564409ebd9d0 0x564409f26300 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564409eada80 0x564409ea7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564409ea4ea0 0x564409ea7340 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 --2- 172.21.15.5:0/359296708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564409ea4580 0x564409eb5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f881b7fe640 0 client.27448 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 >> 172.21.15.5:0/359296708 conn(0x564409e9ecb0 msgr2=0x564409ea9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 shutdown_connections 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 1 -- 172.21.15.5:0/359296708 wait complete. 2024-04-22T13:34:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:20.999+0000 7f88453e1740 20 client.27448 trim_cache size 0 max 16384 2024-04-22T13:34:21.006 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:34:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:34:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:34:21.006 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88240a3da0 msgr2=0x7f88240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88240a3da0 0x7f88240a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88340049d0 tx=0x7f8834009500 comp rx=0 tx=0).stop 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 msgr2=0x7f883c1a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c1a0af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8830037040 tx=0x7f8830004040 comp rx=0 tx=0).stop 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 shutdown_connections 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10e8f0 0x7f883c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88240a3da0 0x7f88240a6260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f883c069c10 0x7f883c1a0af0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 --2- 172.21.15.5:0/1934238134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f883c0692f0 0x7f883c1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 >> 172.21.15.5:0/1934238134 conn(0x7f883c077fa0 msgr2=0x7f883c10fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 shutdown_connections 2024-04-22T13:34:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.000+0000 7f8842715640 1 -- 172.21.15.5:0/1934238134 wait complete. 2024-04-22T13:34:21.009 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:34:21.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:34:21.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:20 smithi049 ceph-mon[31379]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 121 B/s wr, 6 op/s 2024-04-22T13:34:21.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:20 smithi107 ceph-mon[31859]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 121 B/s wr, 6 op/s 2024-04-22T13:34:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2924191947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea181072c0 msgr2=0x7fea1810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2924191947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea181072c0 0x7fea1810ee70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ca0 tx=0x7fea1002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2924191947 shutdown_connections 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2924191947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea181072c0 0x7fea1810ee70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2924191947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea18069f10 0x7fea18106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2924191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea18069560 0x7fea18069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2924191947 >> 172.21.15.5:0/2924191947 conn(0x7fea180771d0 msgr2=0x7fea180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2924191947 shutdown_connections 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2924191947 wait complete. 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 Processor -- start 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.157+0000 7fea1dfb2640 1 -- start start 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea1dfb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 0x7fea18198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea1dfb2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea181072c0 0x7fea18198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea1dfb2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea1819cf30 0x7fea1819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea1dfb2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea181121b0 con 0x7fea18069560 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea1dfb2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112030 con 0x7fea1819cf30 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea1dfb2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112330 con 0x7fea181072c0 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 0x7fea18198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 0x7fea18198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36788/0 (socket says 172.21.15.5:36788) 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 -- 172.21.15.5:0/2064586451 learned_addr learned my addr 172.21.15.5:0/2064586451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea17fff640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea1819cf30 0x7fea1819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea181072c0 0x7fea18198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 -- 172.21.15.5:0/2064586451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea181072c0 msgr2=0x7fea18198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea181072c0 0x7fea18198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 -- 172.21.15.5:0/2064586451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea1819cf30 msgr2=0x7fea1819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea1819cf30 0x7fea1819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 -- 172.21.15.5:0/2064586451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1819da20 con 0x7fea18069560 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea177fe640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 0x7fea18198670 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00e9f0 tx=0x7fea0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.158+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00cd80 con 0x7fea18069560 2024-04-22T13:34:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.159+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0c0108e0 con 0x7fea18069560 2024-04-22T13:34:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.159+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1819dd10 con 0x7fea18069560 2024-04-22T13:34:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.159+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c018980 con 0x7fea18069560 2024-04-22T13:34:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.159+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181a5500 con 0x7fea18069560 2024-04-22T13:34:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.159+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea18068e00 con 0x7fea18069560 2024-04-22T13:34:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.161+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fea0c010430 con 0x7fea18069560 2024-04-22T13:34:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.162+0000 7fea14ff9640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fea000a3d20 0x7fea000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.162+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fea0c0d0fe0 con 0x7fea18069560 2024-04-22T13:34:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.162+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fea000a3d20 0x7fea000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.162+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fea000a3d20 0x7fea000a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea18100690 tx=0x7fea08005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.166+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea0c09d030 con 0x7fea18069560 2024-04-22T13:34:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.396+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea181999c0 con 0x7fea18069560 2024-04-22T13:34:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.397+0000 7fea14ff9640 1 -- 172.21.15.5:0/2064586451 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7fea181999c0 con 0x7fea18069560 2024-04-22T13:34:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:34:21.392 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:34:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:34:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fea000a3d20 msgr2=0x7fea000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fea000a3d20 0x7fea000a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea18100690 tx=0x7fea08005f90 comp rx=0 tx=0).stop 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 msgr2=0x7fea18198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 0x7fea18198670 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00e9f0 tx=0x7fea0c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 shutdown_connections 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fea000a3d20 0x7fea000a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea1819cf30 0x7fea1819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea181072c0 0x7fea18198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 --2- 172.21.15.5:0/2064586451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18069560 0x7fea18198670 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 >> 172.21.15.5:0/2064586451 conn(0x7fea180771d0 msgr2=0x7fea180fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 shutdown_connections 2024-04-22T13:34:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.399+0000 7fea1dfb2640 1 -- 172.21.15.5:0/2064586451 wait complete. 2024-04-22T13:34:21.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:34:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 -- 172.21.15.5:0/1925935115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac107260 msgr2=0x7f6cac10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1925935115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac107260 0x7f6cac10ee30 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009ca0 tx=0x7f6ca402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 -- 172.21.15.5:0/1925935115 shutdown_connections 2024-04-22T13:34:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1925935115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac107260 0x7f6cac10ee30 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1925935115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac102b40 0x7f6cac106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1925935115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac102190 0x7f6cac102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 -- 172.21.15.5:0/1925935115 >> 172.21.15.5:0/1925935115 conn(0x7f6cac0fda10 msgr2=0x7f6cac0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 -- 172.21.15.5:0/1925935115 shutdown_connections 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.515+0000 7f6cb361f640 1 -- 172.21.15.5:0/1925935115 wait complete. 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 Processor -- start 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 -- start start 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac102190 0x7f6cac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac107260 0x7f6cac198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 0x7f6cac19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1121d0 con 0x7f6cac19cfe0 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac112050 con 0x7f6cac102190 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb361f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac112350 con 0x7f6cac107260 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 0x7f6cac19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 0x7f6cac19d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36798/0 (socket says 172.21.15.5:36798) 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/1208327725 learned_addr learned my addr 172.21.15.5:0/1208327725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.516+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac102190 0x7f6cac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac107260 0x7f6cac198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/1208327725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac107260 msgr2=0x7f6cac198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac107260 0x7f6cac198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/1208327725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac102190 msgr2=0x7f6cac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac102190 0x7f6cac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/1208327725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac19dad0 con 0x7f6cac19cfe0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 0x7f6cac19d490 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4005e30 tx=0x7f6ca4032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6ca27fc640 1 -- 172.21.15.5:0/1208327725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4041020 con 0x7f6cac19cfe0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6ca27fc640 1 -- 172.21.15.5:0/1208327725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca4032970 con 0x7f6cac19cfe0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac19dd60 con 0x7f6cac19cfe0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6ca27fc640 1 -- 172.21.15.5:0/1208327725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4039040 con 0x7f6cac19cfe0 2024-04-22T13:34:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.517+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac1a5520 con 0x7f6cac19cfe0 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.519+0000 7f6ca27fc640 1 -- 172.21.15.5:0/1208327725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6ca4044020 con 0x7f6cac19cfe0 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 Processor -- start 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 -- start start 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6ca27fc640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6c980a3d20 0x7f6c980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee8a9f71f0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee8a9944b0 con 0x55ee8aa924d0 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb620d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ee8a9f31b0 con 0x55ee8aa92e80 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6c980a3d20 0x7f6c980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51668/0 (socket says 172.21.15.5:51668) 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36806/0 (socket says 172.21.15.5:36806) 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 -- 172.21.15.5:0/3159171881 learned_addr learned my addr 172.21.15.5:0/3159171881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6ca27fc640 1 -- 172.21.15.5:0/1208327725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f6ca40f2470 con 0x7f6cac19cfe0 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 -- 172.21.15.5:0/3159171881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 msgr2=0x55ee8aa928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 -- 172.21.15.5:0/3159171881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 msgr2=0x55ee8aa9de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.520+0000 7f6cb13bb640 1 -- 172.21.15.5:0/3159171881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8a9861b0 con 0x55ee8aa92e80 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6c980a3d20 0x7f6c980a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0fdec0 tx=0x7f6c9c00b820 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9b430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009d20 tx=0x7f6ca8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459dc9d9dde7f360 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6c977fe640 1 -- 172.21.15.5:0/3159171881 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca801e070 con 0x55ee8aa92e80 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6c977fe640 1 -- 172.21.15.5:0/3159171881 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca8009e10 con 0x55ee8aa92e80 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6c977fe640 1 -- 172.21.15.5:0/3159171881 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800e9a0 con 0x55ee8aa92e80 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6cb620d740 1 -- 172.21.15.5:0/3159171881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 msgr2=0x55ee8aa9b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.521+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9b430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009d20 tx=0x7f6ca8011810 comp rx=0 tx=0).stop 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- 172.21.15.5:0/3159171881 shutdown_connections 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9b430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3159171881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- 172.21.15.5:0/3159171881 >> 172.21.15.5:0/3159171881 conn(0x55ee8aa8cba0 msgr2=0x55ee8aa8cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- 172.21.15.5:0/3159171881 shutdown_connections 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- 172.21.15.5:0/3159171881 wait complete. 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- start start 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 Processor -- start 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa9b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa98690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa98c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee8aa9e970 con 0x55ee8aa9ba00 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee8a9b0500 con 0x55ee8aa924d0 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ee8aa9eaf0 con 0x55ee8aa92e80 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb620d740 10 client.0 _ll_register_callbacks cb 0x55ee8a97eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.522+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa98c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb0bba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa98690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb0bba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa98690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51672/0 (socket says 172.21.15.5:51672) 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa98c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36820/0 (socket says 172.21.15.5:36820) 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb0bba640 1 -- 172.21.15.5:0/2078015972 learned_addr learned my addr 172.21.15.5:0/2078015972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa9b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 msgr2=0x55ee8aa98690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa98690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/2078015972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 msgr2=0x55ee8aa9b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa9b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/2078015972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8a7093f0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa9b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa98690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa98c10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40efc70 tx=0x7f6ca40ba830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399c348c160b92dc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6c94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6c94ff9640 1 -- 172.21.15.5:0/2078015972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40c4000 con 0x55ee8aa9ba00 2024-04-22T13:34:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6c94ff9640 1 -- 172.21.15.5:0/2078015972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f6ca4034ca0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca8009e10 con 0x55ee8aa9ba00 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6c94ff9640 1 -- 172.21.15.5:0/2078015972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ca40bebc0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb620d740 10 client.18210 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.523+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee8a7d37d0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6c94ff9640 1 -- 172.21.15.5:0/2078015972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca8009e10 con 0x55ee8aa9ba00 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6c94ff9640 1 -- 172.21.15.5:0/2078015972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6ca403c030 con 0x55ee8aa9ba00 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb620d740 10 client.18210 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb620d740 10 client.18210 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb620d740 20 client.18210 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb620d740 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8aaabc60 0x55ee8aaae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb620d740 4 client.18210 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8a996930 con 0x55ee8aaabc60 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8aaabc60 0x55ee8aaae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8aaabc60 0x55ee8aaae090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c43149e7ef37d25 server_cookie=4022d035da9cd859 in_seq=0 out_seq=0 2024-04-22T13:34:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.524+0000 7f6c94ff9640 10 client.18210 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6c94ff9640 1 -- 172.21.15.5:0/2078015972 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee8a996930 con 0x55ee8aaabc60 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6c94ff9640 10 client.18210 handle_command_reply: tid=0 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 client.18210 shutdown 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 msgr2=0x55ee8aa98c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa98c10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40efc70 tx=0x7f6ca40ba830 comp rx=0 tx=0).stop 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 shutdown_connections 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8aaabc60 0x55ee8aaae090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa98c10 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa92e80 0x55ee8aa98690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 --2- 172.21.15.5:0/2078015972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aa9b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6c94ff9640 0 client.18210 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 >> 172.21.15.5:0/2078015972 conn(0x55ee8aa8cba0 msgr2=0x55ee8aa8e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 shutdown_connections 2024-04-22T13:34:21.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.525+0000 7f6cb620d740 1 -- 172.21.15.5:0/2078015972 wait complete. 2024-04-22T13:34:21.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.526+0000 7f6cb620d740 20 client.18210 trim_cache size 0 max 16384 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 Processor -- start 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 -- start start 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1adaf0 con 0x55ee8aa92e80 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1af630 con 0x55ee8aa924d0 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb620d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1ae1c0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb0bba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb0bba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36830/0 (socket says 172.21.15.5:36830) 2024-04-22T13:34:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb0bba640 1 -- 172.21.15.5:0/1680769272 learned_addr learned my addr 172.21.15.5:0/1680769272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.539+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb0bba640 1 -- 172.21.15.5:0/1680769272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa9ba00 msgr2=0x55ee8aa9b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb0bba640 1 -- 172.21.15.5:0/1680769272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 msgr2=0x55ee8aaa7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb0bba640 1 -- 172.21.15.5:0/1680769272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8aa92c20 con 0x55ee8aa92e80 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9adc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00bf30 tx=0x7f6c9c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c62b365566bcc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6c96ffd640 1 -- 172.21.15.5:0/1680769272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c02d070 con 0x55ee8aa92e80 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6c96ffd640 1 -- 172.21.15.5:0/1680769272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c01ea60 con 0x55ee8aa92e80 2024-04-22T13:34:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.540+0000 7f6c96ffd640 1 -- 172.21.15.5:0/1680769272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0288f0 con 0x55ee8aa92e80 2024-04-22T13:34:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 -- 172.21.15.5:0/1680769272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 msgr2=0x55ee8aa9adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9adc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00bf30 tx=0x7f6c9c00b410 comp rx=0 tx=0).stop 2024-04-22T13:34:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 -- 172.21.15.5:0/1680769272 shutdown_connections 2024-04-22T13:34:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa92e80 0x55ee8aa9adc0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa9b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 --2- 172.21.15.5:0/1680769272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 -- 172.21.15.5:0/1680769272 >> 172.21.15.5:0/1680769272 conn(0x55ee8aa8cba0 msgr2=0x55ee8aa98650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 -- 172.21.15.5:0/1680769272 shutdown_connections 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 -- 172.21.15.5:0/1680769272 wait complete. 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 -- start start 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.541+0000 7f6cb620d740 1 Processor -- start 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa92e80 0x55ee8aaa42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa94c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1aef60 con 0x55ee8aa9ba00 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1ae890 con 0x55ee8aa92e80 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1abaf0 con 0x55ee8aa924d0 2024-04-22T13:34:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb620d740 10 client.0 _ll_register_callbacks cb 0x7f6ca40ba760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa94c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb13bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb0bba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa92e80 0x55ee8aaa42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb13bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51696/0 (socket says 172.21.15.5:51696) 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb13bb640 1 -- 172.21.15.5:0/3755988023 learned_addr learned my addr 172.21.15.5:0/3755988023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa924d0 msgr2=0x55ee8aaa3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/3755988023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa92e80 msgr2=0x55ee8aaa42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa92e80 0x55ee8aaa42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 -- 172.21.15.5:0/3755988023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8aa95450 con 0x55ee8aa9ba00 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb0bba640 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa92e80 0x55ee8aaa42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6cb1bbc640 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa94c50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40bc770 tx=0x7f6ca4003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8810b003df1eea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6c94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.542+0000 7f6c94ff9640 1 -- 172.21.15.5:0/3755988023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40be000 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6c94ff9640 1 -- 172.21.15.5:0/3755988023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f6ca4034ca0 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c01ea60 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6c94ff9640 1 -- 172.21.15.5:0/3755988023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ca40c2870 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 10 client.18216 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee8ab11d10 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6c94ff9640 1 -- 172.21.15.5:0/3755988023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c01ea60 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6c94ff9640 1 -- 172.21.15.5:0/3755988023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6ca4004b40 con 0x55ee8aa9ba00 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 10 client.18216 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 10 client.18216 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 20 client.18216 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8ab12300 0x55ee8ab14720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 4 client.18216 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee8aa04c10 con 0x55ee8ab12300 2024-04-22T13:34:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.543+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8ab12300 0x55ee8ab14720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb13bb640 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8ab12300 0x55ee8ab14720 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89fd1d4a6929f01f server_cookie=598a6d93b817cb3a in_seq=0 out_seq=0 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6c94ff9640 10 client.18216 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6c94ff9640 1 -- 172.21.15.5:0/3755988023 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee8aa04c10 con 0x55ee8ab12300 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6c94ff9640 10 client.18216 handle_command_reply: tid=0 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 client.18216 shutdown 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 msgr2=0x55ee8aa94c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa94c50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40bc770 tx=0x7f6ca4003c30 comp rx=0 tx=0).stop 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 shutdown_connections 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55ee8ab12300 0x55ee8ab14720 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8aa9ba00 0x55ee8aa94c50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee8aa92e80 0x55ee8aaa42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6c94ff9640 0 client.18216 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 --2- 172.21.15.5:0/3755988023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee8aa924d0 0x55ee8aaa3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 >> 172.21.15.5:0/3755988023 conn(0x55ee8aa8cba0 msgr2=0x55ee8aa8b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 shutdown_connections 2024-04-22T13:34:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 1 -- 172.21.15.5:0/3755988023 wait complete. 2024-04-22T13:34:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.544+0000 7f6cb620d740 20 client.18216 trim_cache size 0 max 16384 2024-04-22T13:34:21.548 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6c980a3d20 msgr2=0x7f6c980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6c980a3d20 0x7f6c980a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0fdec0 tx=0x7f6c9c00b820 comp rx=0 tx=0).stop 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 msgr2=0x7f6cac19d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 0x7f6cac19d490 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4005e30 tx=0x7f6ca4032420 comp rx=0 tx=0).stop 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 shutdown_connections 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac19cfe0 0x7f6cac19d490 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6c980a3d20 0x7f6c980a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cac107260 0x7f6cac198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 --2- 172.21.15.5:0/1208327725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cac102190 0x7f6cac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 >> 172.21.15.5:0/1208327725 conn(0x7f6cac0fda10 msgr2=0x7f6cac0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 shutdown_connections 2024-04-22T13:34:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.545+0000 7f6cb361f640 1 -- 172.21.15.5:0/1208327725 wait complete. 2024-04-22T13:34:21.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 -- 172.21.15.5:0/1930484309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 msgr2=0x7efda4103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1930484309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda4103fb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7efd90009930 tx=0x7efd9002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 -- 172.21.15.5:0/1930484309 shutdown_connections 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1930484309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda4103fb0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1930484309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda41044f0 0x7efda410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1930484309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4102270 0x7efda4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 -- 172.21.15.5:0/1930484309 >> 172.21.15.5:0/1930484309 conn(0x7efda40fda10 msgr2=0x7efda40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 -- 172.21.15.5:0/1930484309 shutdown_connections 2024-04-22T13:34:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 -- 172.21.15.5:0/1930484309 wait complete. 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 Processor -- start 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.663+0000 7efdaaed2640 1 -- start start 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda4102270 0x7efda419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda41044f0 0x7efda41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efda4112290 con 0x7efda4103b50 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efda4112110 con 0x7efda41044f0 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efda4112410 con 0x7efda4102270 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36870/0 (socket says 172.21.15.5:36870) 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 -- 172.21.15.5:0/1681888670 learned_addr learned my addr 172.21.15.5:0/1681888670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda8c6e640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda4102270 0x7efda419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 -- 172.21.15.5:0/1681888670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda4102270 msgr2=0x7efda419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda4102270 0x7efda419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda946f640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda41044f0 0x7efda41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 -- 172.21.15.5:0/1681888670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda41044f0 msgr2=0x7efda41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda41044f0 0x7efda41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 -- 172.21.15.5:0/1681888670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4198d70 con 0x7efda4103b50 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda3fff640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda419e980 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7efd90009230 tx=0x7efd9002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda1ffb640 1 -- 172.21.15.5:0/1681888670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90043020 con 0x7efda4103b50 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda1ffb640 1 -- 172.21.15.5:0/1681888670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9002caf0 con 0x7efda4103b50 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda4199000 con 0x7efda4103b50 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.664+0000 7efda1ffb640 1 -- 172.21.15.5:0/1681888670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90003af0 con 0x7efda4103b50 2024-04-22T13:34:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.665+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a5300 con 0x7efda4103b50 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efda1ffb640 1 -- 172.21.15.5:0/1681888670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efd90046020 con 0x7efda4103b50 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 Processor -- start 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 -- start start 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1a040c1f0 con 0x55b1a04b0a00 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1a03a94b0 con 0x55b1a04a74d0 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.667+0000 7efdadba3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b1a04081b0 con 0x55b1a04a7e80 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36882/0 (socket says 172.21.15.5:36882) 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda3fff640 1 -- 172.21.15.5:0/2602989548 learned_addr learned my addr 172.21.15.5:0/2602989548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda946f640 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda1ffb640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda946f640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 -- 172.21.15.5:0/2602989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 msgr2=0x55b1a04a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 -- 172.21.15.5:0/2602989548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 msgr2=0x55b1a04b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 -- 172.21.15.5:0/2602989548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a039b1b0 con 0x55b1a04a7e80 2024-04-22T13:34:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda3fff640 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda1ffb640 1 -- 172.21.15.5:0/1681888670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7efd900f2840 con 0x7efda4103b50 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda8c6e640 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04b0430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd98009c20 tx=0x7efd98011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564b5f308380b4a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efda946f640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005fd0 tx=0x7efd9c004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efd8effd640 1 -- 172.21.15.5:0/2602989548 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98004050 con 0x55b1a04a7e80 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efd8effd640 1 -- 172.21.15.5:0/2602989548 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd980054d0 con 0x55b1a04a7e80 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.668+0000 7efd8effd640 1 -- 172.21.15.5:0/2602989548 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98004d00 con 0x55b1a04a7e80 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 -- 172.21.15.5:0/2602989548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 msgr2=0x55b1a04b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04b0430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd98009c20 tx=0x7efd98011810 comp rx=0 tx=0).stop 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 -- 172.21.15.5:0/2602989548 shutdown_connections 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04b0430 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 --2- 172.21.15.5:0/2602989548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 -- 172.21.15.5:0/2602989548 >> 172.21.15.5:0/2602989548 conn(0x55b1a04a1ba0 msgr2=0x55b1a04a1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 -- 172.21.15.5:0/2602989548 shutdown_connections 2024-04-22T13:34:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 -- 172.21.15.5:0/2602989548 wait complete. 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 -- start start 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.669+0000 7efdadba3740 1 Processor -- start 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 0x55b1a04b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04b0a00 0x55b1a04adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1a04b3970 con 0x55b1a04a74d0 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b1a03c5500 con 0x55b1a04b0a00 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b1a04b3af0 con 0x55b1a04a7e80 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efdadba3740 10 client.0 _ll_register_callbacks cb 0x55b1a0393b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 0x55b1a04b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 0x55b1a04b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36886/0 (socket says 172.21.15.5:36886) 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 -- 172.21.15.5:0/3860330902 learned_addr learned my addr 172.21.15.5:0/3860330902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 -- 172.21.15.5:0/3860330902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 msgr2=0x55b1a04ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 -- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04b0a00 msgr2=0x55b1a04adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04b0a00 0x55b1a04adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 -- 172.21.15.5:0/3860330902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a011e3f0 con 0x55b1a04a74d0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda3fff640 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 0x55b1a04b0300 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efd980049e0 tx=0x7efd980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e077c4da2c6496a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.670+0000 7efd83fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efd83fff640 1 -- 172.21.15.5:0/3860330902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98029070 con 0x55b1a04a74d0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efd83fff640 1 -- 172.21.15.5:0/3860330902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7efd98003a00 con 0x55b1a04a74d0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd980054d0 con 0x55b1a04a74d0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efd83fff640 1 -- 172.21.15.5:0/3860330902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd98035050 con 0x55b1a04a74d0 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 10 client.18228 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1a01e87d0 con 0x55b1a04a74d0 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efd83fff640 1 -- 172.21.15.5:0/3860330902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd980054d0 con 0x55b1a04a74d0 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efd83fff640 1 -- 172.21.15.5:0/3860330902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efd98005070 con 0x55b1a04a74d0 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 10 client.18228 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 10 client.18228 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 20 client.18228 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a04c0b10 0x55b1a04c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 4 client.18228 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.671+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a03ab930 con 0x55b1a04c0b10 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.672+0000 7efda3fff640 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a04c0b10 0x55b1a04c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.672+0000 7efda3fff640 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a04c0b10 0x55b1a04c2f00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17f02e94d3441047 server_cookie=6354ead22ab704d6 in_seq=0 out_seq=0 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.672+0000 7efd83fff640 10 client.18228 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efd83fff640 1 -- 172.21.15.5:0/3860330902 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1a03ab930 con 0x55b1a04c0b10 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efd83fff640 10 client.18228 handle_command_reply: tid=0 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 client.18228 shutdown 2024-04-22T13:34:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 msgr2=0x55b1a04b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 0x55b1a04b0300 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efd980049e0 tx=0x7efd980046c0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 shutdown_connections 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a04c0b10 0x55b1a04c2f00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04b0a00 0x55b1a04adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 --2- 172.21.15.5:0/3860330902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04a74d0 0x55b1a04b0300 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efd83fff640 0 client.18228 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 >> 172.21.15.5:0/3860330902 conn(0x55b1a04a1ba0 msgr2=0x55b1a04a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 shutdown_connections 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 1 -- 172.21.15.5:0/3860330902 wait complete. 2024-04-22T13:34:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.673+0000 7efdadba3740 20 client.18228 trim_cache size 0 max 16384 2024-04-22T13:34:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 Processor -- start 2024-04-22T13:34:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 -- start start 2024-04-22T13:34:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efda41adab0 con 0x55b1a04b0a00 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efda41af5f0 con 0x55b1a04a74d0 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efdadba3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efda41ae180 con 0x55b1a04a7e80 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36898/0 (socket says 172.21.15.5:36898) 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 -- 172.21.15.5:0/4118636066 learned_addr learned my addr 172.21.15.5:0/4118636066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda8c6e640 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda3fff640 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 -- 172.21.15.5:0/4118636066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 msgr2=0x55b1a04afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 -- 172.21.15.5:0/4118636066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 msgr2=0x55b1a04bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 -- 172.21.15.5:0/4118636066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a04a7c20 con 0x55b1a04b0a00 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.687+0000 7efda946f640 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b0300 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0133b0 tx=0x7efd9c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c2b8d4f0e03b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efd8e7fc640 1 -- 172.21.15.5:0/4118636066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c019070 con 0x55b1a04b0a00 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efd8e7fc640 1 -- 172.21.15.5:0/4118636066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c005020 con 0x55b1a04b0a00 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efd8e7fc640 1 -- 172.21.15.5:0/4118636066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c020690 con 0x55b1a04b0a00 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 -- 172.21.15.5:0/4118636066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 msgr2=0x55b1a04b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b0300 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0133b0 tx=0x7efd9c009510 comp rx=0 tx=0).stop 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 -- 172.21.15.5:0/4118636066 shutdown_connections 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04b0300 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a7e80 0x55b1a04afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 --2- 172.21.15.5:0/4118636066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a74d0 0x55b1a04bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 -- 172.21.15.5:0/4118636066 >> 172.21.15.5:0/4118636066 conn(0x55b1a04a1ba0 msgr2=0x55b1a04a0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 -- 172.21.15.5:0/4118636066 shutdown_connections 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.688+0000 7efdadba3740 1 -- 172.21.15.5:0/4118636066 wait complete. 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 -- start start 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 Processor -- start 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a74d0 0x55b1a04b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a7e80 0x55b1a04aae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efda41aef20 con 0x55b1a04b0a00 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efda41ae850 con 0x55b1a04a7e80 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efda41abab0 con 0x55b1a04a74d0 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efdadba3740 10 client.0 _ll_register_callbacks cb 0x7efd98009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04a9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36908/0 (socket says 172.21.15.5:36908) 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 -- 172.21.15.5:0/3503818972 learned_addr learned my addr 172.21.15.5:0/3503818972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda3fff640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a7e80 0x55b1a04aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a74d0 0x55b1a04b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 -- 172.21.15.5:0/3503818972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a74d0 msgr2=0x55b1a04b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a74d0 0x55b1a04b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 -- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a7e80 msgr2=0x55b1a04aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.689+0000 7efda946f640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a7e80 0x55b1a04aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efda946f640 1 -- 172.21.15.5:0/3503818972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a04a9e40 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a74d0 0x55b1a04b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efda946f640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04a9900 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7efd9c004a80 tx=0x7efd9c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd36841e1785890 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efd83fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efd83fff640 1 -- 172.21.15.5:0/3503818972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c02e020 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efd83fff640 1 -- 172.21.15.5:0/3503818972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7efd9c0053e0 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c005020 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efd83fff640 1 -- 172.21.15.5:0/3503818972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd9c039050 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 10 client.18234 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1a0526620 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efd83fff640 1 -- 172.21.15.5:0/3503818972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c005020 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efd83fff640 1 -- 172.21.15.5:0/3503818972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efd9c01f070 con 0x55b1a04b0a00 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 10 client.18234 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 10 client.18234 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 20 client.18234 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a0526d60 0x55b1a0529180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 4 client.18234 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.690+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1a0419c10 con 0x55b1a0526d60 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a0526d60 0x55b1a0529180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efda8c6e640 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a0526d60 0x55b1a0529180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d26685a7480c8cd server_cookie=99b78dc5dc9db674 in_seq=0 out_seq=0 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efd83fff640 10 client.18234 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efd83fff640 1 -- 172.21.15.5:0/3503818972 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1a0419c10 con 0x55b1a0526d60 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efd83fff640 10 client.18234 handle_command_reply: tid=0 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 client.18234 shutdown 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 msgr2=0x55b1a04a9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04a9900 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7efd9c004a80 tx=0x7efd9c004db0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 shutdown_connections 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55b1a0526d60 0x55b1a0529180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1a04b0a00 0x55b1a04a9900 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efd83fff640 0 client.18234 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1a04a7e80 0x55b1a04aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 --2- 172.21.15.5:0/3503818972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1a04a74d0 0x55b1a04b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 >> 172.21.15.5:0/3503818972 conn(0x55b1a04a1ba0 msgr2=0x55b1a04a8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 shutdown_connections 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.691+0000 7efdadba3740 1 -- 172.21.15.5:0/3503818972 wait complete. 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdadba3740 20 client.18234 trim_cache size 0 max 16384 2024-04-22T13:34:21.693 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 msgr2=0x7efd940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005fd0 tx=0x7efd9c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 msgr2=0x7efda419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda419e980 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7efd90009230 tx=0x7efd9002c710 comp rx=0 tx=0).stop 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 shutdown_connections 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda4103b50 0x7efda419e980 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7efd940a3d20 0x7efd940a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda41044f0 0x7efda41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 --2- 172.21.15.5:0/1681888670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda4102270 0x7efda419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 >> 172.21.15.5:0/1681888670 conn(0x7efda40fda10 msgr2=0x7efda40fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 shutdown_connections 2024-04-22T13:34:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.692+0000 7efdaaed2640 1 -- 172.21.15.5:0/1681888670 wait complete. 2024-04-22T13:34:21.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:34:21.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:34:21.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:34:21.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:34:21.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:21 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2064586451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:21.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 -- 172.21.15.5:0/357750260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b8103ae0 msgr2=0x7f47b810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 --2- 172.21.15.5:0/357750260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b8103ae0 0x7f47b810bfc0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009900 tx=0x7f47ac02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 -- 172.21.15.5:0/357750260 shutdown_connections 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 --2- 172.21.15.5:0/357750260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b8103ae0 0x7f47b810bfc0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 --2- 172.21.15.5:0/357750260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47b810c500 0x7f47b810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 --2- 172.21.15.5:0/357750260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 -- 172.21.15.5:0/357750260 >> 172.21.15.5:0/357750260 conn(0x7f47b8078000 msgr2=0x7f47b8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 -- 172.21.15.5:0/357750260 shutdown_connections 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 -- 172.21.15.5:0/357750260 wait complete. 2024-04-22T13:34:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.807+0000 7f47be031640 1 Processor -- start 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 -- start start 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47b8103ae0 0x7f47b8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 0x7f47b819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8112190 con 0x7f47b810c500 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8112010 con 0x7f47b8103ae0 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47be031640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8112310 con 0x7f47b81031c0 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 0x7f47b819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 0x7f47b819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36932/0 (socket says 172.21.15.5:36932) 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b8198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51756/0 (socket says 172.21.15.5:51756) 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b7fff640 1 -- 172.21.15.5:0/3542892164 learned_addr learned my addr 172.21.15.5:0/3542892164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47b8103ae0 0x7f47b8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 -- 172.21.15.5:0/3542892164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47b8103ae0 msgr2=0x7f47b8198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47b8103ae0 0x7f47b8198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 -- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 msgr2=0x7f47b819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 0x7f47b819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 -- 172.21.15.5:0/3542892164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b819d710 con 0x7f47b81031c0 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b7fff640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 0x7f47b819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.808+0000 7f47b77fe640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b8198770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9f0 tx=0x7f47a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.809+0000 7f47b4ff9640 1 -- 172.21.15.5:0/3542892164 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a800ccc0 con 0x7f47b81031c0 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.809+0000 7f47b4ff9640 1 -- 172.21.15.5:0/3542892164 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a8005020 con 0x7f47b81031c0 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.809+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b819da00 con 0x7f47b81031c0 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.809+0000 7f47b4ff9640 1 -- 172.21.15.5:0/3542892164 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8004a00 con 0x7f47b81031c0 2024-04-22T13:34:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.809+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b81a5290 con 0x7f47b81031c0 2024-04-22T13:34:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b4ff9640 1 -- 172.21.15.5:0/3542892164 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f47a8004ba0 con 0x7f47b81031c0 2024-04-22T13:34:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 Processor -- start 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 -- start start 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bede4d0 0x55f17bede8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bee7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bee9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f17be431f0 con 0x55f17bedee80 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f17bde04b0 con 0x55f17bee7a00 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47c0c24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f17be3f1b0 con 0x55f17bede4d0 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bee7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bee7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36940/0 (socket says 172.21.15.5:36940) 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 -- 172.21.15.5:0/3795720006 learned_addr learned my addr 172.21.15.5:0/3795720006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b7fff640 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bede4d0 0x55f17bede8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bee9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b4ff9640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47a00a3da0 0x7f47a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 -- 172.21.15.5:0/3795720006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bede4d0 msgr2=0x55f17bede8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bede4d0 0x55f17bede8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 -- 172.21.15.5:0/3795720006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 msgr2=0x55f17bee9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bee9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b77fe640 1 -- 172.21.15.5:0/3795720006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17bdd21b0 con 0x55f17bedee80 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.812+0000 7f47b7fff640 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bede4d0 0x55f17bede8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f47b7fff640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47a00a3da0 0x7f47a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f47b4ff9640 1 -- 172.21.15.5:0/3542892164 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f47a80d06d0 con 0x7f47b81031c0 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f47b77fe640 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bee7430 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f47a80cdd90 tx=0x7f47a8098a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0e92cd07561622 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f479dffb640 1 -- 172.21.15.5:0/3795720006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a809c440 con 0x55f17bedee80 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f479dffb640 1 -- 172.21.15.5:0/3795720006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a809cd70 con 0x55f17bedee80 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f479dffb640 1 -- 172.21.15.5:0/3795720006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80a3fe0 con 0x55f17bedee80 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f47b7fff640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47a00a3da0 0x7f47a00a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009fd0 tx=0x7f47b0004a40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.813+0000 7f47c0c24740 1 -- 172.21.15.5:0/3795720006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 msgr2=0x55f17bee7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bee7430 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f47a80cdd90 tx=0x7f47a8098a40 comp rx=0 tx=0).stop 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- 172.21.15.5:0/3795720006 shutdown_connections 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bee7430 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bee9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- 172.21.15.5:0/3795720006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bede4d0 0x55f17bede8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- 172.21.15.5:0/3795720006 >> 172.21.15.5:0/3795720006 conn(0x55f17bed8ba0 msgr2=0x55f17bed8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- 172.21.15.5:0/3795720006 shutdown_connections 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- 172.21.15.5:0/3795720006 wait complete. 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- start start 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 Processor -- start 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17bee7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bee7a00 0x55f17bee4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f17beea970 con 0x55f17bee7a00 2024-04-22T13:34:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f17bdfc500 con 0x55f17bede4d0 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f17beeaaf0 con 0x55f17bedee80 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.814+0000 7f47c0c24740 10 client.0 _ll_register_callbacks cb 0x55f17bdcab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bee7a00 0x55f17bee4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17bee7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17bee7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:43076/0 (socket says 172.21.15.5:43076) 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b77fe640 1 -- 172.21.15.5:0/1614346168 learned_addr learned my addr 172.21.15.5:0/1614346168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 -- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 msgr2=0x55f17bee45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 -- 172.21.15.5:0/1614346168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 msgr2=0x55f17bee7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17bee7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 -- 172.21.15.5:0/1614346168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17bb553f0 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17bee7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47b7fff640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bee7a00 0x55f17bee4b00 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f47b001c6a0 tx=0x7f47b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15c1870c52354b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f4796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f4796ffd640 1 -- 172.21.15.5:0/1614346168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b002c060 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f4796ffd640 1 -- 172.21.15.5:0/1614346168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f47b000e940 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a809cd70 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f4796ffd640 1 -- 172.21.15.5:0/1614346168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47b000eb40 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47c0c24740 10 client.18246 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.815+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f17bc1f7d0 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f4796ffd640 1 -- 172.21.15.5:0/1614346168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a809cd70 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f4796ffd640 1 -- 172.21.15.5:0/1614346168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47b002f3b0 con 0x55f17bee7a00 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47c0c24740 10 client.18246 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47c0c24740 10 client.18246 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:34:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47c0c24740 20 client.18246 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bef7b90 0x55f17bef9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47c0c24740 4 client.18246 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f17bde2930 con 0x55f17bef7b90 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bef7b90 0x55f17bef9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bef7b90 0x55f17bef9fe0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=522876aa746ce7a8 server_cookie=7aed977d9d350ac3 in_seq=0 out_seq=0 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.816+0000 7f4796ffd640 10 client.18246 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f4796ffd640 1 -- 172.21.15.5:0/1614346168 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f17bde2930 con 0x55f17bef7b90 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f4796ffd640 10 client.18246 handle_command_reply: tid=0 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 client.18246 shutdown 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bee7a00 msgr2=0x55f17bee4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bee7a00 0x55f17bee4b00 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f47b001c6a0 tx=0x7f47b0004ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 shutdown_connections 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bef7b90 0x55f17bef9fe0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bee7a00 0x55f17bee4b00 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f4796ffd640 0 client.18246 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1614346168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17bee7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 >> 172.21.15.5:0/1614346168 conn(0x55f17bed8ba0 msgr2=0x55f17beda8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 shutdown_connections 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 1 -- 172.21.15.5:0/1614346168 wait complete. 2024-04-22T13:34:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.817+0000 7f47c0c24740 20 client.18246 trim_cache size 0 max 16384 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 Processor -- start 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 -- start start 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 0x55f17bef3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bedf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81adb80 con 0x55f17bede4d0 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81af6c0 con 0x55f17bee7a00 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47c0c24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81ae250 con 0x55f17bedee80 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 0x55f17bef3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 0x55f17bef3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36958/0 (socket says 172.21.15.5:36958) 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 -- 172.21.15.5:0/1770834881 learned_addr learned my addr 172.21.15.5:0/1770834881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 -- 172.21.15.5:0/1770834881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 msgr2=0x55f17bee3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 -- 172.21.15.5:0/1770834881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 msgr2=0x55f17bedf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bedf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 -- 172.21.15.5:0/1770834881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17bee53b0 con 0x55f17bede4d0 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 0x55f17bef3830 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f47a80ce420 tx=0x7f47a80d6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb3dc07d5981e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f479d7fa640 1 -- 172.21.15.5:0/1770834881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a801c040 con 0x55f17bede4d0 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f479d7fa640 1 -- 172.21.15.5:0/1770834881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a809cd70 con 0x55f17bede4d0 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.831+0000 7f479d7fa640 1 -- 172.21.15.5:0/1770834881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80a4c40 con 0x55f17bede4d0 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 -- 172.21.15.5:0/1770834881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 msgr2=0x55f17bef3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 0x55f17bef3830 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f47a80ce420 tx=0x7f47a80d6550 comp rx=0 tx=0).stop 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 -- 172.21.15.5:0/1770834881 shutdown_connections 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bee7a00 0x55f17bedf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bedee80 0x55f17bee3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1770834881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bede4d0 0x55f17bef3830 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 -- 172.21.15.5:0/1770834881 >> 172.21.15.5:0/1770834881 conn(0x55f17bed8ba0 msgr2=0x55f17bedf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 -- 172.21.15.5:0/1770834881 shutdown_connections 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 -- 172.21.15.5:0/1770834881 wait complete. 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.832+0000 7f47c0c24740 1 -- start start 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 Processor -- start 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17beefd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bef0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bee7a00 0x55f17bee0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81aeff0 con 0x55f17bedee80 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81ae920 con 0x55f17bede4d0 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81abb80 con 0x55f17bee7a00 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47c0c24740 10 client.0 _ll_register_callbacks cb 0x7f47b000fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bef0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bef0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36974/0 (socket says 172.21.15.5:36974) 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 -- 172.21.15.5:0/1026773125 learned_addr learned my addr 172.21.15.5:0/1026773125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b7fff640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bee7a00 0x55f17bee0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17beefd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 -- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bee7a00 msgr2=0x55f17bee0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bee7a00 0x55f17bee0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 -- 172.21.15.5:0/1026773125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 msgr2=0x55f17beefd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17beefd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 -- 172.21.15.5:0/1026773125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17bee1510 con 0x55f17bedee80 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17beefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f47b6ffd640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bef0280 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009900 tx=0x7f47ac005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b989c591233e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f4796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.833+0000 7f4796ffd640 1 -- 172.21.15.5:0/1026773125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0053e0 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f4796ffd640 1 -- 172.21.15.5:0/1026773125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f47ac005580 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a809cd70 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f4796ffd640 1 -- 172.21.15.5:0/1026773125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47ac02c880 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 10 client.18252 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f17bf5dc30 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f4796ffd640 1 -- 172.21.15.5:0/1026773125 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a809cd70 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f4796ffd640 1 -- 172.21.15.5:0/1026773125 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47ac0035f0 con 0x55f17bedee80 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 10 client.18252 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 10 client.18252 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 20 client.18252 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bf5e3d0 0x55f17bf607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 4 client.18252 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f17be50c10 con 0x55f17bf5e3d0 2024-04-22T13:34:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.834+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bf5e3d0 0x55f17bf607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47b77fe640 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bf5e3d0 0x55f17bf607f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eed818739f08b84 server_cookie=2eedd93a46ea6dd9 in_seq=0 out_seq=0 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f4796ffd640 10 client.18252 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f4796ffd640 1 -- 172.21.15.5:0/1026773125 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f17be50c10 con 0x55f17bf5e3d0 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f4796ffd640 10 client.18252 handle_command_reply: tid=0 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 client.18252 shutdown 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 msgr2=0x55f17bef0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bef0280 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009900 tx=0x7f47ac005270 comp rx=0 tx=0).stop 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 shutdown_connections 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55f17bf5e3d0 0x55f17bf607f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17bedee80 0x55f17bef0280 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17bee7a00 0x55f17bee0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f4796ffd640 0 client.18252 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 --2- 172.21.15.5:0/1026773125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f17bede4d0 0x55f17beefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 >> 172.21.15.5:0/1026773125 conn(0x55f17bed8ba0 msgr2=0x55f17bedfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 shutdown_connections 2024-04-22T13:34:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 1 -- 172.21.15.5:0/1026773125 wait complete. 2024-04-22T13:34:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.835+0000 7f47c0c24740 20 client.18252 trim_cache size 0 max 16384 2024-04-22T13:34:21.838 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47a00a3da0 msgr2=0x7f47a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47a00a3da0 0x7f47a00a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009fd0 tx=0x7f47b0004a40 comp rx=0 tx=0).stop 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 msgr2=0x7f47b8198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b8198770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9f0 tx=0x7f47a800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 shutdown_connections 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47b810c500 0x7f47b819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47a00a3da0 0x7f47a00a6260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47b8103ae0 0x7f47b8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 --2- 172.21.15.5:0/3542892164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b81031c0 0x7f47b8198770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 >> 172.21.15.5:0/3542892164 conn(0x7f47b8078000 msgr2=0x7f47b80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 shutdown_connections 2024-04-22T13:34:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.836+0000 7f47be031640 1 -- 172.21.15.5:0/3542892164 wait complete. 2024-04-22T13:34:21.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:34:21.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 -- 172.21.15.5:0/1482316491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 msgr2=0x7ff96410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/1482316491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96410eb60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009ca0 tx=0x7ff95c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 -- 172.21.15.5:0/1482316491 shutdown_connections 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/1482316491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96410eb60 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/1482316491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff964103b70 0x7ff96410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/1482316491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9641031c0 0x7ff9641035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 -- 172.21.15.5:0/1482316491 >> 172.21.15.5:0/1482316491 conn(0x7ff964077f70 msgr2=0x7ff964078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 -- 172.21.15.5:0/1482316491 shutdown_connections 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 -- 172.21.15.5:0/1482316491 wait complete. 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 Processor -- start 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.952+0000 7ff96a8e0640 1 -- start start 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff96a8e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9641031c0 0x7ff96419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff96a8e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff96a8e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff964198700 0x7ff964198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff96a8e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112380 con 0x7ff96410c6d0 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff96a8e0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112200 con 0x7ff9641031c0 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff96a8e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112500 con 0x7ff964198700 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36992/0 (socket says 172.21.15.5:36992) 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff963fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9641031c0 0x7ff96419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 -- 172.21.15.5:0/2049122212 learned_addr learned my addr 172.21.15.5:0/2049122212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff968e7d640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff964198700 0x7ff964198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 -- 172.21.15.5:0/2049122212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff964198700 msgr2=0x7ff964198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff964198700 0x7ff964198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 -- 172.21.15.5:0/2049122212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9641031c0 msgr2=0x7ff96419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9641031c0 0x7ff96419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 -- 172.21.15.5:0/2049122212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff964199300 con 0x7ff96410c6d0 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff963fff640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9641031c0 0x7ff96419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9637fe640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96419eb80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9f0 tx=0x7ff95800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.953+0000 7ff9617fa640 1 -- 172.21.15.5:0/2049122212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95800ccc0 con 0x7ff96410c6d0 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.954+0000 7ff9617fa640 1 -- 172.21.15.5:0/2049122212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9580108e0 con 0x7ff96410c6d0 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.954+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9641a53e0 con 0x7ff96410c6d0 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.954+0000 7ff9617fa640 1 -- 172.21.15.5:0/2049122212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958018860 con 0x7ff96410c6d0 2024-04-22T13:34:21.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.954+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641a57e0 con 0x7ff96410c6d0 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.956+0000 7ff9617fa640 1 -- 172.21.15.5:0/2049122212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff958010430 con 0x7ff96410c6d0 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 Processor -- start 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 -- start start 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c7d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c86430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c88e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5582d1be21f0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5582d1b7f4b0 con 0x5582d1c86a00 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5582d1bde1b0 con 0x5582d1c7de80 2024-04-22T13:34:21.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c7d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c7d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37000/0 (socket says 172.21.15.5:37000) 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 -- 172.21.15.5:0/806941977 learned_addr learned my addr 172.21.15.5:0/806941977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff963fff640 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c86430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff9617fa640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff94c0a3d20 0x7ff94c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff9637fe640 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c88e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 -- 172.21.15.5:0/806941977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 msgr2=0x5582d1c86430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c86430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 -- 172.21.15.5:0/806941977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 msgr2=0x5582d1c88e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c88e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff968e7d640 1 -- 172.21.15.5:0/806941977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d1b711b0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.957+0000 7ff9617fa640 1 -- 172.21.15.5:0/2049122212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff9580d0aa0 con 0x7ff96410c6d0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff968e7d640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff94c0a3d20 0x7ff94c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff968e7d640 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c7d8b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02c700 tx=0x7ff95c032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab31352eaa72e7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff94a7fc640 1 -- 172.21.15.5:0/806941977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c039040 con 0x5582d1c7d4d0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff94a7fc640 1 -- 172.21.15.5:0/806941977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c032970 con 0x5582d1c7d4d0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff94a7fc640 1 -- 172.21.15.5:0/806941977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004280 con 0x5582d1c7d4d0 2024-04-22T13:34:21.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff96d5ad740 1 -- 172.21.15.5:0/806941977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 msgr2=0x5582d1c7d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff968e7d640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff94c0a3d20 0x7ff94c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff95c004af0 tx=0x7ff95c004620 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.958+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c7d8b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02c700 tx=0x7ff95c032340 comp rx=0 tx=0).stop 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- 172.21.15.5:0/806941977 shutdown_connections 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c88e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c86430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/806941977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c7d8b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- 172.21.15.5:0/806941977 >> 172.21.15.5:0/806941977 conn(0x5582d1c77ba0 msgr2=0x5582d1c77f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- 172.21.15.5:0/806941977 shutdown_connections 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- 172.21.15.5:0/806941977 wait complete. 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- start start 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 Processor -- start 2024-04-22T13:34:21.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7d4d0 0x5582d1c86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c83610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c83b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5582d1c89970 con 0x5582d1c86a00 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5582d1b9b500 con 0x5582d1c7d4d0 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5582d1c89af0 con 0x5582d1c7de80 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.959+0000 7ff96d5ad740 10 client.0 _ll_register_callbacks cb 0x5582d1b69b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c83b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c83b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37014/0 (socket says 172.21.15.5:37014) 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 -- 172.21.15.5:0/92984028 learned_addr learned my addr 172.21.15.5:0/92984028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff963fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7d4d0 0x5582d1c86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff9637fe640 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c83610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 -- 172.21.15.5:0/92984028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 msgr2=0x5582d1c83610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c83610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 -- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7d4d0 msgr2=0x5582d1c86210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7d4d0 0x5582d1c86210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 -- 172.21.15.5:0/92984028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d18f43f0 con 0x5582d1c86a00 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff968e7d640 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c83b50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff95c037040 tx=0x7ff95c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50697efeba7c4a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff9437fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff9437fe640 1 -- 172.21.15.5:0/92984028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c039040 con 0x5582d1c86a00 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff9437fe640 1 -- 172.21.15.5:0/92984028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff95c02cca0 con 0x5582d1c86a00 2024-04-22T13:34:21.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c032970 con 0x5582d1c86a00 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff9437fe640 1 -- 172.21.15.5:0/92984028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff95c009800 con 0x5582d1c86a00 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff96d5ad740 10 client.18264 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.960+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582d19be7d0 con 0x5582d1c86a00 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff9437fe640 1 -- 172.21.15.5:0/92984028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c032970 con 0x5582d1c86a00 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff9437fe640 1 -- 172.21.15.5:0/92984028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff95c03fcb0 con 0x5582d1c86a00 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff96d5ad740 10 client.18264 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff96d5ad740 10 client.18264 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff96d5ad740 20 client.18264 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1c96a40 0x5582d1c98e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff96d5ad740 4 client.18264 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582d1b81930 con 0x5582d1c96a40 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff963fff640 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1c96a40 0x5582d1c98e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff9437fe640 10 client.18264 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.961+0000 7ff963fff640 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1c96a40 0x5582d1c98e30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=788e647eb697cd9e server_cookie=629114ac3b85ab11 in_seq=0 out_seq=0 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff9437fe640 1 -- 172.21.15.5:0/92984028 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582d1b81930 con 0x5582d1c96a40 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff9437fe640 10 client.18264 handle_command_reply: tid=0 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 client.18264 shutdown 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 msgr2=0x5582d1c83b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c83b50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff95c037040 tx=0x7ff95c040000 comp rx=0 tx=0).stop 2024-04-22T13:34:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 shutdown_connections 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1c96a40 0x5582d1c98e30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c83b50 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c83610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/92984028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7d4d0 0x5582d1c86210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff9437fe640 0 client.18264 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 >> 172.21.15.5:0/92984028 conn(0x5582d1c77ba0 msgr2=0x5582d1c798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 shutdown_connections 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.962+0000 7ff96d5ad740 1 -- 172.21.15.5:0/92984028 wait complete. 2024-04-22T13:34:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.963+0000 7ff96d5ad740 20 client.18264 trim_cache size 0 max 16384 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 Processor -- start 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 -- start start 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7d4d0 0x5582d1c92860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7de80 0x5582d1c82dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641adc60 con 0x5582d1c86a00 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641af7a0 con 0x5582d1c7de80 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641ae330 con 0x5582d1c7d4d0 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff968e7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff968e7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c855c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37026/0 (socket says 172.21.15.5:37026) 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff968e7d640 1 -- 172.21.15.5:0/1364691701 learned_addr learned my addr 172.21.15.5:0/1364691701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff9637fe640 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7de80 0x5582d1c82dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.976+0000 7ff963fff640 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7d4d0 0x5582d1c92860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff968e7d640 1 -- 172.21.15.5:0/1364691701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7d4d0 msgr2=0x5582d1c92860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff968e7d640 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7d4d0 0x5582d1c92860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff968e7d640 1 -- 172.21.15.5:0/1364691701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7de80 msgr2=0x5582d1c82dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff968e7d640 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7de80 0x5582d1c82dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff968e7d640 1 -- 172.21.15.5:0/1364691701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d1c88df0 con 0x5582d1c86a00 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff963fff640 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7d4d0 0x5582d1c92860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff968e7d640 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c855c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000f80 tx=0x7ff95c0099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8230078b35c0a043 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff949ffb640 1 -- 172.21.15.5:0/1364691701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c039040 con 0x5582d1c86a00 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff949ffb640 1 -- 172.21.15.5:0/1364691701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c02cca0 con 0x5582d1c86a00 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff949ffb640 1 -- 172.21.15.5:0/1364691701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c03f8c0 con 0x5582d1c86a00 2024-04-22T13:34:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 -- 172.21.15.5:0/1364691701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 msgr2=0x5582d1c855c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c855c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000f80 tx=0x7ff95c0099a0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 -- 172.21.15.5:0/1364691701 shutdown_connections 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c86a00 0x5582d1c855c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c7de80 0x5582d1c82dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/1364691701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7d4d0 0x5582d1c92860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.977+0000 7ff96d5ad740 1 -- 172.21.15.5:0/1364691701 >> 172.21.15.5:0/1364691701 conn(0x5582d1c77ba0 msgr2=0x5582d1c76b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 -- 172.21.15.5:0/1364691701 shutdown_connections 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 -- 172.21.15.5:0/1364691701 wait complete. 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 -- start start 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 Processor -- start 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c80900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c80e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641af0d0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641aea00 con 0x5582d1c86a00 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641abc60 con 0x5582d1c7de80 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff96d5ad740 10 client.0 _ll_register_callbacks cb 0x7ff954005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37030/0 (socket says 172.21.15.5:37030) 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.978+0000 7ff963fff640 1 -- 172.21.15.5:0/304418034 learned_addr learned my addr 172.21.15.5:0/304418034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff968e7d640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c80e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9637fe640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c80900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff963fff640 1 -- 172.21.15.5:0/304418034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 msgr2=0x5582d1c80900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff963fff640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c80900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff963fff640 1 -- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 msgr2=0x5582d1c80e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff963fff640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c80e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff963fff640 1 -- 172.21.15.5:0/304418034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d1cfbd10 con 0x5582d1c7d4d0 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff968e7d640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c80e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9637fe640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c80900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff963fff640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c803c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x5582d1c78c10 tx=0x7ff954005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904366dfb1227aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9437fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9437fe640 1 -- 172.21.15.5:0/304418034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954004230 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9437fe640 1 -- 172.21.15.5:0/304418034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff9540043d0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c02cca0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9437fe640 1 -- 172.21.15.5:0/304418034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff954031050 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff96d5ad740 10 client.18270 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582d1cfcfe0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.979+0000 7ff9437fe640 1 -- 172.21.15.5:0/304418034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c02cca0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff9437fe640 1 -- 172.21.15.5:0/304418034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff9540213f0 con 0x5582d1c7d4d0 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 10 client.18270 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 10 client.18270 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 20 client.18270 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1cfd740 0x5582d1cffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 4 client.18270 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582d1c83ee0 con 0x5582d1cfd740 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff9637fe640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1cfd740 0x5582d1cffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff9637fe640 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1cfd740 0x5582d1cffb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30059182bb128bc7 server_cookie=ca1785ecf4585d4b in_seq=0 out_seq=0 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff9437fe640 10 client.18270 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff9437fe640 1 -- 172.21.15.5:0/304418034 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582d1c83ee0 con 0x5582d1cfd740 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff9437fe640 10 client.18270 handle_command_reply: tid=0 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 1 client.18270 shutdown 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 msgr2=0x5582d1c803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.980+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c803c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x5582d1c78c10 tx=0x7ff954005dc0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 shutdown_connections 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x5582d1cfd740 0x5582d1cffb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5582d1c86a00 0x5582d1c80e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff9437fe640 0 client.18270 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582d1c7de80 0x5582d1c80900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 --2- 172.21.15.5:0/304418034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582d1c7d4d0 0x5582d1c803c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 >> 172.21.15.5:0/304418034 conn(0x5582d1c77ba0 msgr2=0x5582d1c7eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 shutdown_connections 2024-04-22T13:34:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 1 -- 172.21.15.5:0/304418034 wait complete. 2024-04-22T13:34:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96d5ad740 20 client.18270 trim_cache size 0 max 16384 2024-04-22T13:34:21.983 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff94c0a3d20 msgr2=0x7ff94c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff94c0a3d20 0x7ff94c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff95c004af0 tx=0x7ff95c004620 comp rx=0 tx=0).stop 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 msgr2=0x7ff96419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96419eb80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9f0 tx=0x7ff95800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 shutdown_connections 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff96410c6d0 0x7ff96419eb80 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff94c0a3d20 0x7ff94c0a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff964198700 0x7ff964198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 --2- 172.21.15.5:0/2049122212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9641031c0 0x7ff96419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 >> 172.21.15.5:0/2049122212 conn(0x7ff964077f70 msgr2=0x7ff9640fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 shutdown_connections 2024-04-22T13:34:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:21.981+0000 7ff96a8e0640 1 -- 172.21.15.5:0/2049122212 wait complete. 2024-04-22T13:34:21.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:34:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:34:22.022 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:34:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:34:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:34:22.023 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:21 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2064586451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:22.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:34:22.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:34:22.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:34:22.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:34:22.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:21 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2064586451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.096+0000 7fed23843640 1 -- 172.21.15.5:0/8246573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 msgr2=0x7fed1c102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.096+0000 7fed23843640 1 --2- 172.21.15.5:0/8246573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c102560 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fed18009900 tx=0x7fed1802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- 172.21.15.5:0/8246573 shutdown_connections 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 --2- 172.21.15.5:0/8246573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 0x7fed1c10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 --2- 172.21.15.5:0/8246573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed1c102b30 0x7fed1c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 --2- 172.21.15.5:0/8246573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c102560 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- 172.21.15.5:0/8246573 >> 172.21.15.5:0/8246573 conn(0x7fed1c0fda00 msgr2=0x7fed1c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- 172.21.15.5:0/8246573 shutdown_connections 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- 172.21.15.5:0/8246573 wait complete. 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 Processor -- start 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- start start 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 0x7fed1c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed1c19cfc0 0x7fed1c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1121c0 con 0x7fed1c102180 2024-04-22T13:34:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c112040 con 0x7fed1c19cfc0 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.097+0000 7fed23843640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c112340 con 0x7fed1c107250 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37046/0 (socket says 172.21.15.5:37046) 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 -- 172.21.15.5:0/3491380374 learned_addr learned my addr 172.21.15.5:0/3491380374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed20dde640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 0x7fed1c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed21de0640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed1c19cfc0 0x7fed1c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 -- 172.21.15.5:0/3491380374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 msgr2=0x7fed1c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 0x7fed1c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 -- 172.21.15.5:0/3491380374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed1c19cfc0 msgr2=0x7fed1c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed1c19cfc0 0x7fed1c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 -- 172.21.15.5:0/3491380374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c19dab0 con 0x7fed1c102180 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed20dde640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 0x7fed1c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed215df640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c198700 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fed18005940 tx=0x7fed1802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed127fc640 1 -- 172.21.15.5:0/3491380374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18043020 con 0x7fed1c102180 2024-04-22T13:34:22.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c19dd40 con 0x7fed1c102180 2024-04-22T13:34:22.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed127fc640 1 -- 172.21.15.5:0/3491380374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1802cbe0 con 0x7fed1c102180 2024-04-22T13:34:22.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed127fc640 1 -- 172.21.15.5:0/3491380374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18003af0 con 0x7fed1c102180 2024-04-22T13:34:22.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.098+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c1a5590 con 0x7fed1c102180 2024-04-22T13:34:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 Processor -- start 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed127fc640 1 -- 172.21.15.5:0/3491380374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fed18046020 con 0x7fed1c102180 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 -- start start 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bc4d0 0x55a5d85bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 0x55a5d85c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5d85211f0 con 0x55a5d85bce80 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a5d84be4b0 con 0x55a5d85c5a00 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed26438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a5d851d1b0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 0x55a5d85c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 0x55a5d85c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37054/0 (socket says 172.21.15.5:37054) 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed215df640 1 -- 172.21.15.5:0/1258215639 learned_addr learned my addr 172.21.15.5:0/1258215639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed21de0640 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bc4d0 0x55a5d85bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed20dde640 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed215df640 1 -- 172.21.15.5:0/1258215639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bc4d0 msgr2=0x55a5d85bc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed215df640 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bc4d0 0x55a5d85bc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.101+0000 7fed215df640 1 -- 172.21.15.5:0/1258215639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 msgr2=0x55a5d85c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed215df640 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed215df640 1 -- 172.21.15.5:0/1258215639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d84b01b0 con 0x55a5d85bce80 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed20dde640 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed127fc640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed080a3d20 0x7fed080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed215df640 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 0x55a5d85c5430 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fed180f00b0 tx=0x7fed180f8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62664dd9cb6a7a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed21de0640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed080a3d20 0x7fed080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed127fc640 1 -- 172.21.15.5:0/3491380374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fed180f28e0 con 0x7fed1c102180 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed077fe640 1 -- 172.21.15.5:0/1258215639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180bac30 con 0x55a5d85bce80 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed077fe640 1 -- 172.21.15.5:0/1258215639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed180bf090 con 0x55a5d85bce80 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.102+0000 7fed077fe640 1 -- 172.21.15.5:0/1258215639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180be2f0 con 0x55a5d85bce80 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 -- 172.21.15.5:0/1258215639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 msgr2=0x55a5d85c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed21de0640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed080a3d20 0x7fed080a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed14005fd0 tx=0x7fed14009600 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 0x55a5d85c5430 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fed180f00b0 tx=0x7fed180f8a80 comp rx=0 tx=0).stop 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 -- 172.21.15.5:0/1258215639 shutdown_connections 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bce80 0x55a5d85c5430 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 --2- 172.21.15.5:0/1258215639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bc4d0 0x55a5d85bc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 -- 172.21.15.5:0/1258215639 >> 172.21.15.5:0/1258215639 conn(0x55a5d85b6ba0 msgr2=0x55a5d85b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 -- 172.21.15.5:0/1258215639 shutdown_connections 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 -- 172.21.15.5:0/1258215639 wait complete. 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 -- start start 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.103+0000 7fed26438740 1 Processor -- start 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5d85c8730 con 0x55a5d85bc4d0 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a5d84da500 con 0x55a5d85bce80 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a5d85c88b0 con 0x55a5d85c5a00 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 10 client.0 _ll_register_callbacks cb 0x55a5d84a8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37058/0 (socket says 172.21.15.5:37058) 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 -- 172.21.15.5:0/4216250827 learned_addr learned my addr 172.21.15.5:0/4216250827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed20dde640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed21de0640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 -- 172.21.15.5:0/4216250827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 msgr2=0x55a5d85c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 -- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 msgr2=0x55a5d85c2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85c2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 -- 172.21.15.5:0/4216250827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d82333f0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed21de0640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed215df640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85c5290 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fed1802bc70 tx=0x7fed180bb230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6edd1de93264ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed04ff9640 1 -- 172.21.15.5:0/4216250827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180be4d0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed04ff9640 1 -- 172.21.15.5:0/4216250827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fed180053d0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.104+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed180bf090 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed04ff9640 1 -- 172.21.15.5:0/4216250827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed180be760 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 10 client.18282 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5d82fd7d0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed04ff9640 1 -- 172.21.15.5:0/4216250827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed180bf090 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed04ff9640 1 -- 172.21.15.5:0/4216250827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed180bebf0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 10 client.18282 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 20 client.18282 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d85d5790 0x55a5d85d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 4 client.18282 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d84c0930 con 0x55a5d85d5790 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.105+0000 7fed20dde640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d85d5790 0x55a5d85d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed20dde640 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d85d5790 0x55a5d85d7b80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0110c737e86461 server_cookie=f5118ef5d33f02a0 in_seq=0 out_seq=0 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed04ff9640 10 client.18282 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed04ff9640 1 -- 172.21.15.5:0/4216250827 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5d84c0930 con 0x55a5d85d5790 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed04ff9640 10 client.18282 handle_command_reply: tid=0 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 client.18282 shutdown 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 msgr2=0x55a5d85c5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85c5290 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fed1802bc70 tx=0x7fed180bb230 comp rx=0 tx=0).stop 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 shutdown_connections 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d85d5790 0x55a5d85d7b80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed04ff9640 0 client.18282 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85c2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.106+0000 7fed26438740 1 --2- 172.21.15.5:0/4216250827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85c5290 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.107+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 >> 172.21.15.5:0/4216250827 conn(0x55a5d85b6ba0 msgr2=0x55a5d85b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.107+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 shutdown_connections 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.107+0000 7fed26438740 1 -- 172.21.15.5:0/4216250827 wait complete. 2024-04-22T13:34:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.107+0000 7fed26438740 20 client.18282 trim_cache size 0 max 16384 2024-04-22T13:34:22.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.120+0000 7fed26438740 1 Processor -- start 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 -- start start 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bce80 0x55a5d85b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1adb00 con 0x55a5d85bc4d0 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1af640 con 0x55a5d85c5a00 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed26438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1ae1d0 con 0x55a5d85bce80 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37060/0 (socket says 172.21.15.5:37060) 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 -- 172.21.15.5:0/743148015 learned_addr learned my addr 172.21.15.5:0/743148015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed20dde640 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bce80 0x55a5d85b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed21de0640 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 -- 172.21.15.5:0/743148015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bce80 msgr2=0x55a5d85b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bce80 0x55a5d85b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 -- 172.21.15.5:0/743148015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 msgr2=0x55a5d85c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 -- 172.21.15.5:0/743148015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d85be200 con 0x55a5d85bc4d0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed215df640 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85d15f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fed180bcc40 tx=0x7fed180f8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e3abaa48a9f412 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed06ffd640 1 -- 172.21.15.5:0/743148015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180be4d0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed06ffd640 1 -- 172.21.15.5:0/743148015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed180bf090 con 0x55a5d85bc4d0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.121+0000 7fed06ffd640 1 -- 172.21.15.5:0/743148015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180be2f0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 -- 172.21.15.5:0/743148015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 msgr2=0x55a5d85d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85d15f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fed180bcc40 tx=0x7fed180f8a80 comp rx=0 tx=0).stop 2024-04-22T13:34:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 -- 172.21.15.5:0/743148015 shutdown_connections 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85c5a00 0x55a5d85c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85bce80 0x55a5d85b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 --2- 172.21.15.5:0/743148015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85d15f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 -- 172.21.15.5:0/743148015 >> 172.21.15.5:0/743148015 conn(0x55a5d85b6ba0 msgr2=0x55a5d85bdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 -- 172.21.15.5:0/743148015 shutdown_connections 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 -- 172.21.15.5:0/743148015 wait complete. 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 -- start start 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.122+0000 7fed26438740 1 Processor -- start 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1aef70 con 0x55a5d85bc4d0 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1ae8a0 con 0x55a5d85bce80 2024-04-22T13:34:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1abb00 con 0x55a5d85c5a00 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed26438740 10 client.0 _ll_register_callbacks cb 0x7fed180df0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37064/0 (socket says 172.21.15.5:37064) 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 -- 172.21.15.5:0/1576682232 learned_addr learned my addr 172.21.15.5:0/1576682232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed21de0640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 -- 172.21.15.5:0/1576682232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 msgr2=0x55a5d85c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed20dde640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 -- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 msgr2=0x55a5d85ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 -- 172.21.15.5:0/1576682232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d85c98b0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed21de0640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed215df640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85cdb50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fed18006010 tx=0x7fed180beea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c610ee6beb09daa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.123+0000 7fed04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed04ff9640 1 -- 172.21.15.5:0/1576682232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180be2f0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed04ff9640 1 -- 172.21.15.5:0/1576682232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fed180c7170 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed180bf090 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed04ff9640 1 -- 172.21.15.5:0/1576682232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed180be4d0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 10 client.18288 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5d85c98b0 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed04ff9640 1 -- 172.21.15.5:0/1576682232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed180bf090 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed04ff9640 1 -- 172.21.15.5:0/1576682232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed18089330 con 0x55a5d85bc4d0 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 10 client.18288 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 10 client.18288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 20 client.18288 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d863ba30 0x55a5d863de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 4 client.18288 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5d852ec10 con 0x55a5d863ba30 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.124+0000 7fed20dde640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d863ba30 0x55a5d863de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed20dde640 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d863ba30 0x55a5d863de20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01f813ca840a68c server_cookie=46e8827479f8a282 in_seq=0 out_seq=0 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed04ff9640 10 client.18288 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed04ff9640 1 -- 172.21.15.5:0/1576682232 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5d852ec10 con 0x55a5d863ba30 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed04ff9640 10 client.18288 handle_command_reply: tid=0 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 client.18288 shutdown 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 msgr2=0x55a5d85cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85cdb50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fed18006010 tx=0x7fed180beea0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 shutdown_connections 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a5d863ba30 0x55a5d863de20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5d85c5a00 0x55a5d85c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5d85bce80 0x55a5d85ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 --2- 172.21.15.5:0/1576682232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5d85bc4d0 0x55a5d85cdb50 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed04ff9640 0 client.18288 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:22.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 >> 172.21.15.5:0/1576682232 conn(0x55a5d85b6ba0 msgr2=0x55a5d85b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:22.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 shutdown_connections 2024-04-22T13:34:22.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 1 -- 172.21.15.5:0/1576682232 wait complete. 2024-04-22T13:34:22.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.125+0000 7fed26438740 20 client.18288 trim_cache size 0 max 16384 2024-04-22T13:34:22.128 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:22.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed080a3d20 msgr2=0x7fed080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed080a3d20 0x7fed080a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed14005fd0 tx=0x7fed14009600 comp rx=0 tx=0).stop 2024-04-22T13:34:22.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 msgr2=0x7fed1c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:22.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c198700 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fed18005940 tx=0x7fed1802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 shutdown_connections 2024-04-22T13:34:22.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed080a3d20 0x7fed080a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed1c19cfc0 0x7fed1c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c107250 0x7fed1c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 --2- 172.21.15.5:0/3491380374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1c102180 0x7fed1c198700 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:22.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 >> 172.21.15.5:0/3491380374 conn(0x7fed1c0fda00 msgr2=0x7fed1c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:22.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 shutdown_connections 2024-04-22T13:34:22.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:22.126+0000 7fed23843640 1 -- 172.21.15.5:0/3491380374 wait complete. 2024-04-22T13:34:23.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:22 smithi049 ceph-mon[31379]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:22 smithi107 ceph-mon[31859]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:23.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:22 smithi005 ceph-mon[28006]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:23.133 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:34:23.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:34:23.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:34:23.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 -- 172.21.15.5:0/3693061042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f64101960 msgr2=0x7f0f64105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3693061042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f64101960 0x7f0f64105d90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009900 tx=0x7f0f5402bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 -- 172.21.15.5:0/3693061042 shutdown_connections 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3693061042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f64101960 0x7f0f64105d90 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3693061042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f64106470 0x7f0f64106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3693061042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f64100fb0 0x7f0f64101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.257+0000 7f0f6b606640 1 -- 172.21.15.5:0/3693061042 >> 172.21.15.5:0/3693061042 conn(0x7f0f64077f70 msgr2=0x7f0f64078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 -- 172.21.15.5:0/3693061042 shutdown_connections 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 -- 172.21.15.5:0/3693061042 wait complete. 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 Processor -- start 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 -- start start 2024-04-22T13:34:23.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f64100fb0 0x7f0f6419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f64106470 0x7f0f6419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 0x7f0f641a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641189c0 con 0x7f0f641a3880 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64118840 con 0x7f0f64106470 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f6b606640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64118b40 con 0x7f0f64100fb0 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.258+0000 7f0f69ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 0x7f0f641a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 0x7f0f641a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37084/0 (socket says 172.21.15.5:37084) 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 -- 172.21.15.5:0/3590358565 learned_addr learned my addr 172.21.15.5:0/3590358565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f64106470 0x7f0f6419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f693a2640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f64100fb0 0x7f0f6419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 -- 172.21.15.5:0/3590358565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f64100fb0 msgr2=0x7f0f6419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f64100fb0 0x7f0f6419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 -- 172.21.15.5:0/3590358565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f64106470 msgr2=0x7f0f6419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f64106470 0x7f0f6419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 -- 172.21.15.5:0/3590358565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f641a4370 con 0x7f0f641a3880 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f69ba3640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 0x7f0f641a3d30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009fd0 tx=0x7f0f5c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f5a7fc640 1 -- 172.21.15.5:0/3590358565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c017070 con 0x7f0f641a3880 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f641a4660 con 0x7f0f641a3880 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f5a7fc640 1 -- 172.21.15.5:0/3590358565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f5c003e20 con 0x7f0f641a3880 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f5a7fc640 1 -- 172.21.15.5:0/3590358565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c010460 con 0x7f0f641a3880 2024-04-22T13:34:23.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.259+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641abe20 con 0x7f0f641a3880 2024-04-22T13:34:23.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f5a7fc640 1 -- 172.21.15.5:0/3590358565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0f5c003970 con 0x7f0f641a3880 2024-04-22T13:34:23.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 Processor -- start 2024-04-22T13:34:23.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 -- start start 2024-04-22T13:34:23.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 0x55798e8db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55798e836f00 con 0x55798e8dbe10 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55798e7d4590 con 0x55798e8d28e0 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55798e832ec0 con 0x55798e8d3290 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.262+0000 7f0f5a7fc640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f500a3d20 0x7f0f500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f68ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 0x55798e8db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f68ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8de260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37092/0 (socket says 172.21.15.5:37092) 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f68ba1640 1 -- 172.21.15.5:0/400302313 learned_addr learned my addr 172.21.15.5:0/400302313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 0x55798e8db840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:51976/0 (socket says 172.21.15.5:51976) 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f69ba3640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f500a3d20 0x7f0f500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f5a7fc640 1 -- 172.21.15.5:0/3590358565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f0f5c0d0540 con 0x7f0f641a3880 2024-04-22T13:34:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 -- 172.21.15.5:0/400302313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 msgr2=0x55798e8d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 -- 172.21.15.5:0/400302313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 msgr2=0x55798e8de260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8de260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 -- 172.21.15.5:0/400302313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798e6117e0 con 0x55798e8d3290 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8de260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f69ba3640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f500a3d20 0x7f0f500a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0ce290 tx=0x7f0f5c004020 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f693a2640 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 0x55798e8db840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009b70 tx=0x7f0f600117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa21a0ae62f479a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f4f7fe640 1 -- 172.21.15.5:0/400302313 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60009e60 con 0x55798e8d3290 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f4f7fe640 1 -- 172.21.15.5:0/400302313 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f60012900 con 0x55798e8d3290 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.263+0000 7f0f4f7fe640 1 -- 172.21.15.5:0/400302313 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000f9b0 con 0x55798e8d3290 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/400302313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 msgr2=0x55798e8db840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 0x55798e8db840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009b70 tx=0x7f0f600117b0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/400302313 shutdown_connections 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8de260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d3290 0x55798e8db840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/400302313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/400302313 >> 172.21.15.5:0/400302313 conn(0x55798e8ccfb0 msgr2=0x55798e8cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/400302313 shutdown_connections 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.264+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/400302313 wait complete. 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 -- start start 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 Processor -- start 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 0x55798e8d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55798e8ded80 con 0x55798e8d3290 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55798e7f0580 con 0x55798e8d28e0 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55798e8def00 con 0x55798e8dbe10 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f6e1fc740 10 client.0 _ll_register_callbacks cb 0x55798e7f6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f693a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 0x55798e8d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f693a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 0x55798e8d92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37094/0 (socket says 172.21.15.5:37094) 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f693a2640 1 -- 172.21.15.5:0/1409071634 learned_addr learned my addr 172.21.15.5:0/1409071634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.265+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f693a2640 1 -- 172.21.15.5:0/1409071634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 msgr2=0x55798e8d9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f693a2640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f693a2640 1 -- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 msgr2=0x55798e8d8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f693a2640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f693a2640 1 -- 172.21.15.5:0/1409071634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798e671f70 con 0x55798e8d3290 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f693a2640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 0x55798e8d92f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0f6001b000 tx=0x7f0f600126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662f3d0235e159a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f4cff9640 1 -- 172.21.15.5:0/1409071634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60028070 con 0x55798e8d3290 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f4cff9640 1 -- 172.21.15.5:0/1409071634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f0f60005040 con 0x55798e8d3290 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f60012900 con 0x55798e8d3290 2024-04-22T13:34:23.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f4cff9640 1 -- 172.21.15.5:0/1409071634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f60034050 con 0x55798e8d3290 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f6e1fc740 10 client.18300 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.266+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55798e509170 con 0x55798e8d3290 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f4cff9640 1 -- 172.21.15.5:0/1409071634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f60012900 con 0x55798e8d3290 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f4cff9640 1 -- 172.21.15.5:0/1409071634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0f6000fbf0 con 0x55798e8d3290 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f6e1fc740 10 client.18300 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f6e1fc740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f6e1fc740 20 client.18300 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e8ec230 0x55798e8ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f6e1fc740 4 client.18300 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798e844920 con 0x55798e8ec230 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e8ec230 0x55798e8ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f4cff9640 10 client.18300 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.267+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e8ec230 0x55798e8ee620 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2070f539ae23f3b2 server_cookie=fb07d66f9302de5c in_seq=0 out_seq=0 2024-04-22T13:34:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f4cff9640 1 -- 172.21.15.5:0/1409071634 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55798e844920 con 0x55798e8ec230 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f4cff9640 10 client.18300 handle_command_reply: tid=0 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 client.18300 shutdown 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 msgr2=0x55798e8d92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 0x55798e8d92f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0f6001b000 tx=0x7f0f600126e0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 shutdown_connections 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e8ec230 0x55798e8ee620 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d3290 0x55798e8d92f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f4cff9640 0 client.18300 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/1409071634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d28e0 0x55798e8d8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 >> 172.21.15.5:0/1409071634 conn(0x55798e8ccfb0 msgr2=0x55798e8ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 shutdown_connections 2024-04-22T13:34:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.268+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/1409071634 wait complete. 2024-04-22T13:34:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.269+0000 7f0f6e1fc740 20 client.18300 trim_cache size 0 max 16384 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 Processor -- start 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 -- start start 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 0x55798e8db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641b4230 con 0x55798e8d28e0 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641b5d70 con 0x55798e8d3290 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641b4900 con 0x55798e8dbe10 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 0x55798e8db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 0x55798e8db4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37096/0 (socket says 172.21.15.5:37096) 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 -- 172.21.15.5:0/914779564 learned_addr learned my addr 172.21.15.5:0/914779564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f693a2640 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 -- 172.21.15.5:0/914779564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 msgr2=0x55798e8d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 -- 172.21.15.5:0/914779564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 msgr2=0x55798e8e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.287+0000 7f0f68ba1640 1 -- 172.21.15.5:0/914779564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798e8deee0 con 0x55798e8d28e0 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 0x55798e8db4e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009230 tx=0x7f0f54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358cd9160dcdbc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f4effd640 1 -- 172.21.15.5:0/914779564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5402cb90 con 0x55798e8d28e0 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f4effd640 1 -- 172.21.15.5:0/914779564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f54053070 con 0x55798e8d28e0 2024-04-22T13:34:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f4effd640 1 -- 172.21.15.5:0/914779564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540046c0 con 0x55798e8d28e0 2024-04-22T13:34:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/914779564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 msgr2=0x55798e8db4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 0x55798e8db4e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009230 tx=0x7f0f54000bc0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/914779564 shutdown_connections 2024-04-22T13:34:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8dbe10 0x55798e8d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8e7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/914779564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8d28e0 0x55798e8db4e0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.288+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/914779564 >> 172.21.15.5:0/914779564 conn(0x55798e8ccfb0 msgr2=0x55798e8d8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/914779564 shutdown_connections 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/914779564 wait complete. 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 -- start start 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 Processor -- start 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d28e0 0x55798e8e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641b56a0 con 0x55798e8dbe10 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641b4fd0 con 0x55798e8d3290 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f641b2230 con 0x55798e8d28e0 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f6e1fc740 10 client.0 _ll_register_callbacks cb 0x7f0f54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f68ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f68ba1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8d5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37098/0 (socket says 172.21.15.5:37098) 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.289+0000 7f0f68ba1640 1 -- 172.21.15.5:0/844399417 learned_addr learned my addr 172.21.15.5:0/844399417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f693a2640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f68ba1640 1 -- 172.21.15.5:0/844399417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d28e0 msgr2=0x55798e8e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d28e0 0x55798e8e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f68ba1640 1 -- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 msgr2=0x55798e8d5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8d5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f68ba1640 1 -- 172.21.15.5:0/844399417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798e951490 con 0x55798e8dbe10 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f693a2640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f68ba1640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8d5f60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402c6a0 tx=0x7f0f5404e360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2add8f7dc83f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f4cff9640 1 -- 172.21.15.5:0/844399417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54005040 con 0x55798e8dbe10 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f4cff9640 1 -- 172.21.15.5:0/844399417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f0f540051e0 con 0x55798e8dbe10 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f54053070 con 0x55798e8dbe10 2024-04-22T13:34:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f4cff9640 1 -- 172.21.15.5:0/844399417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f54067050 con 0x55798e8dbe10 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f6e1fc740 10 client.18306 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55798e952620 con 0x55798e8dbe10 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.290+0000 7f0f4cff9640 1 -- 172.21.15.5:0/844399417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f54053070 con 0x55798e8dbe10 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f4cff9640 1 -- 172.21.15.5:0/844399417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0f54043450 con 0x55798e8dbe10 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f6e1fc740 10 client.18306 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f6e1fc740 10 client.18306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f6e1fc740 20 client.18306 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e952c80 0x55798e9550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f6e1fc740 4 client.18306 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55798e83e790 con 0x55798e952c80 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f693a2640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e952c80 0x55798e9550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f693a2640 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e952c80 0x55798e9550a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee87cc4a70eaa5e server_cookie=790c7e33623299cc in_seq=0 out_seq=0 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.291+0000 7f0f4cff9640 10 client.18306 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.292+0000 7f0f4cff9640 1 -- 172.21.15.5:0/844399417 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55798e83e790 con 0x55798e952c80 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.292+0000 7f0f4cff9640 10 client.18306 handle_command_reply: tid=0 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.292+0000 7f0f6e1fc740 1 client.18306 shutdown 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.292+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 msgr2=0x55798e8d5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.292+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8d5f60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402c6a0 tx=0x7f0f5404e360 comp rx=0 tx=0).stop 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 shutdown_connections 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55798e952c80 0x55798e9550a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798e8dbe10 0x55798e8d5f60 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55798e8d3290 0x55798e8d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f4cff9640 0 client.18306 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 --2- 172.21.15.5:0/844399417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798e8d28e0 0x55798e8e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 >> 172.21.15.5:0/844399417 conn(0x55798e8ccfb0 msgr2=0x55798e8ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 shutdown_connections 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 1 -- 172.21.15.5:0/844399417 wait complete. 2024-04-22T13:34:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6e1fc740 20 client.18306 trim_cache size 0 max 16384 2024-04-22T13:34:23.294 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:34:23.294 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:34:23.294 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "356d394d-083f-40cd-bf69-4f1d222113cb", 2024-04-22T13:34:23.294 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:34:23.294 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f500a3d20 msgr2=0x7f0f500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f500a3d20 0x7f0f500a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0ce290 tx=0x7f0f5c004020 comp rx=0 tx=0).stop 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 msgr2=0x7f0f641a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 0x7f0f641a3d30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009fd0 tx=0x7f0f5c00b260 comp rx=0 tx=0).stop 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 shutdown_connections 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.293+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f641a3880 0x7f0f641a3d30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.294+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0f500a3d20 0x7f0f500a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.294+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f64106470 0x7f0f6419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.294+0000 7f0f6b606640 1 --2- 172.21.15.5:0/3590358565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f64100fb0 0x7f0f6419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.294+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 >> 172.21.15.5:0/3590358565 conn(0x7f0f64077f70 msgr2=0x7f0f64103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.294+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 shutdown_connections 2024-04-22T13:34:23.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.294+0000 7f0f6b606640 1 -- 172.21.15.5:0/3590358565 wait complete. 2024-04-22T13:34:23.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:23.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 -- 172.21.15.5:0/769095612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 msgr2=0x7f61c810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 --2- 172.21.15.5:0/769095612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c810ec30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009900 tx=0x7f61c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 -- 172.21.15.5:0/769095612 shutdown_connections 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 --2- 172.21.15.5:0/769095612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c810ec30 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 --2- 172.21.15.5:0/769095612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c8102940 0x7f61c8106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 --2- 172.21.15.5:0/769095612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8101f90 0x7f61c8102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 -- 172.21.15.5:0/769095612 >> 172.21.15.5:0/769095612 conn(0x7f61c80fd810 msgr2=0x7f61c80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 -- 172.21.15.5:0/769095612 shutdown_connections 2024-04-22T13:34:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.409+0000 7f61cca00640 1 -- 172.21.15.5:0/769095612 wait complete. 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 Processor -- start 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 -- start start 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8101f90 0x7f61c8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c8074cd0 0x7f61c81a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8112020 con 0x7f61c8107060 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8111ea0 con 0x7f61c8074cd0 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61cca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81121a0 con 0x7f61c8101f90 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37112/0 (socket says 172.21.15.5:37112) 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61c659c640 1 -- 172.21.15.5:0/2513445641 learned_addr learned my addr 172.21.15.5:0/2513445641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.410+0000 7f61c759e640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c8074cd0 0x7f61c81a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8101f90 0x7f61c8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c659c640 1 -- 172.21.15.5:0/2513445641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8101f90 msgr2=0x7f61c8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c659c640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8101f90 0x7f61c8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c659c640 1 -- 172.21.15.5:0/2513445641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c8074cd0 msgr2=0x7f61c81a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c659c640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c8074cd0 0x7f61c81a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c659c640 1 -- 172.21.15.5:0/2513445641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c81a6db0 con 0x7f61c8107060 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61c659c640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c8074790 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00d770 tx=0x7f61bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61affff640 1 -- 172.21.15.5:0/2513445641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc017070 con 0x7f61c8107060 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c81a70a0 con 0x7f61c8107060 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61affff640 1 -- 172.21.15.5:0/2513445641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61bc009c30 con 0x7f61c8107060 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61affff640 1 -- 172.21.15.5:0/2513445641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00f620 con 0x7f61c8107060 2024-04-22T13:34:23.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.411+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c81a75f0 con 0x7f61c8107060 2024-04-22T13:34:23.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.413+0000 7f61affff640 1 -- 172.21.15.5:0/2513445641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f61bc01a070 con 0x7f61c8107060 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 Processor -- start 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 -- start start 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665c5030 0x55bc665cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 0x55bc665d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc665291e0 con 0x55bc665cdc10 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bc664c6390 con 0x55bc665c4710 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61cfdea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bc665251a0 con 0x55bc665c5030 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 0x55bc665d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 0x55bc665d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37126/0 (socket says 172.21.15.5:37126) 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61c659c640 1 -- 172.21.15.5:0/1188443846 learned_addr learned my addr 172.21.15.5:0/1188443846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61affff640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f61b00a3d10 0x7f61b00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665c5030 0x55bc665cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61c759e640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.414+0000 7f61c759e640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f61b00a3d10 0x7f61b00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61affff640 1 -- 172.21.15.5:0/2513445641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f61bc0d02d0 con 0x7f61c8107060 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c659c640 1 -- 172.21.15.5:0/1188443846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665c5030 msgr2=0x55bc665cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c659c640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665c5030 0x55bc665cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c659c640 1 -- 172.21.15.5:0/1188443846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 msgr2=0x55bc665c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c659c640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c659c640 1 -- 172.21.15.5:0/1188443846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc663057d0 con 0x55bc665cdc10 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c759e640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665c5030 0x55bc665cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c659c640 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 0x55bc665d0000 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f61bc09bf70 tx=0x7f61bc010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72097c255146d08f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61ad7fa640 1 -- 172.21.15.5:0/1188443846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0cf070 con 0x55bc665cdc10 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61c759e640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f61b00a3d10 0x7f61b00a61d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005ab0 tx=0x7f61c0005a40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61ad7fa640 1 -- 172.21.15.5:0/1188443846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61bc009de0 con 0x55bc665cdc10 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.415+0000 7f61ad7fa640 1 -- 172.21.15.5:0/1188443846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc021050 con 0x55bc665cdc10 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- 172.21.15.5:0/1188443846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 msgr2=0x55bc665d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 0x55bc665d0000 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f61bc09bf70 tx=0x7f61bc010c00 comp rx=0 tx=0).stop 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- 172.21.15.5:0/1188443846 shutdown_connections 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665cdc10 0x55bc665d0000 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665c5030 0x55bc665cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1188443846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- 172.21.15.5:0/1188443846 >> 172.21.15.5:0/1188443846 conn(0x55bc665bee40 msgr2=0x55bc665bf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- 172.21.15.5:0/1188443846 shutdown_connections 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- 172.21.15.5:0/1188443846 wait complete. 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- start start 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 Processor -- start 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 0x55bc665cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc665d0940 con 0x55bc665c5030 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bc664e22f0 con 0x55bc665c4710 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.416+0000 7f61cfdea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bc665d0ac0 con 0x55bc665cdc10 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61cfdea740 10 client.0 _ll_register_callbacks cb 0x55bc664ef580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 0x55bc665cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 0x55bc665cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37130/0 (socket says 172.21.15.5:37130) 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 -- 172.21.15.5:0/1842558888 learned_addr learned my addr 172.21.15.5:0/1842558888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c759e640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 -- 172.21.15.5:0/1842558888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 msgr2=0x55bc665cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 -- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 msgr2=0x55bc665cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 -- 172.21.15.5:0/1842558888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6623b3f0 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61c659c640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 0x55bc665cb170 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0ce020 tx=0x7f61bc071c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0b3dc285dbb759 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61aaff5640 1 -- 172.21.15.5:0/1842558888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0cf050 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61aaff5640 1 -- 172.21.15.5:0/1842558888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f61bc0988e0 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61bc009de0 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61aaff5640 1 -- 172.21.15.5:0/1842558888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61bc09c270 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61cfdea740 10 client.18318 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.417+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc661fb0f0 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61aaff5640 1 -- 172.21.15.5:0/1842558888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61bc009de0 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61aaff5640 1 -- 172.21.15.5:0/1842558888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f61bc0a4a50 con 0x55bc665c5030 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61cfdea740 10 client.18318 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61cfdea740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61cfdea740 20 client.18318 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc665dda60 0x55bc665dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61cfdea740 4 client.18318 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc66536c00 con 0x55bc665dda60 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.418+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc665dda60 0x55bc665dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61aaff5640 10 client.18318 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc665dda60 0x55bc665dfe50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c350ce404ee9b755 server_cookie=a76458c73c72376a in_seq=0 out_seq=0 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61aaff5640 1 -- 172.21.15.5:0/1842558888 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc66536c00 con 0x55bc665dda60 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61aaff5640 10 client.18318 handle_command_reply: tid=0 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 client.18318 shutdown 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 msgr2=0x55bc665cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 0x55bc665cb170 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0ce020 tx=0x7f61bc071c20 comp rx=0 tx=0).stop 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 shutdown_connections 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc665dda60 0x55bc665dfe50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c5030 0x55bc665cb170 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61aaff5640 0 client.18318 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.419+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1842558888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c4710 0x55bc665cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.420+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 >> 172.21.15.5:0/1842558888 conn(0x55bc665bee40 msgr2=0x55bc665be120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.420+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 shutdown_connections 2024-04-22T13:34:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.420+0000 7f61cfdea740 1 -- 172.21.15.5:0/1842558888 wait complete. 2024-04-22T13:34:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.420+0000 7f61cfdea740 20 client.18318 trim_cache size 0 max 16384 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 Processor -- start 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 -- start start 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81afbb0 con 0x55bc665c4710 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b16f0 con 0x55bc665c5030 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61cfdea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b0280 con 0x55bc665cdc10 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37146/0 (socket says 172.21.15.5:37146) 2024-04-22T13:34:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61c6d9d640 1 -- 172.21.15.5:0/1973330452 learned_addr learned my addr 172.21.15.5:0/1973330452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.437+0000 7f61c659c640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c759e640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c6d9d640 1 -- 172.21.15.5:0/1973330452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 msgr2=0x55bc665e2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665e2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c6d9d640 1 -- 172.21.15.5:0/1973330452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 msgr2=0x55bc665bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c6d9d640 1 -- 172.21.15.5:0/1973330452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc665d0000 con 0x55bc665c4710 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c759e640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665e2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c659c640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665c9b70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005fd0 tx=0x7f61b80139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32a0aa04d9aff75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61acff9640 1 -- 172.21.15.5:0/1973330452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8004d20 con 0x55bc665c4710 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61acff9640 1 -- 172.21.15.5:0/1973330452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b8023030 con 0x55bc665c4710 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61acff9640 1 -- 172.21.15.5:0/1973330452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8010ea0 con 0x55bc665c4710 2024-04-22T13:34:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.438+0000 7f61cfdea740 1 -- 172.21.15.5:0/1973330452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 msgr2=0x55bc665c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665c9b70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005fd0 tx=0x7f61b80139f0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- 172.21.15.5:0/1973330452 shutdown_connections 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc665e2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- 172.21.15.5:0/1973330452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665c9b70 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- 172.21.15.5:0/1973330452 >> 172.21.15.5:0/1973330452 conn(0x55bc665bee40 msgr2=0x55bc665be120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- 172.21.15.5:0/1973330452 shutdown_connections 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- 172.21.15.5:0/1973330452 wait complete. 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- start start 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 Processor -- start 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665e1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc66642c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b1020 con 0x55bc665c4710 2024-04-22T13:34:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81b0950 con 0x55bc665c5030 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81adbb0 con 0x55bc665cdc10 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61cfdea740 10 client.0 _ll_register_callbacks cb 0x7f61b8009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665e1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37154/0 (socket says 172.21.15.5:37154) 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.439+0000 7f61c6d9d640 1 -- 172.21.15.5:0/3155769981 learned_addr learned my addr 172.21.15.5:0/3155769981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c659c640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c759e640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc66642c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c6d9d640 1 -- 172.21.15.5:0/3155769981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 msgr2=0x55bc66642c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc66642c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c6d9d640 1 -- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 msgr2=0x55bc665e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665e1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c6d9d640 1 -- 172.21.15.5:0/3155769981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc66643180 con 0x55bc665c4710 2024-04-22T13:34:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c659c640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61c6d9d640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665e1000 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f61b8015010 tx=0x7f61b8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30be595b7ef98be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61aaff5640 1 -- 172.21.15.5:0/3155769981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80107d0 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61aaff5640 1 -- 172.21.15.5:0/3155769981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f61b8004a80 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b8023030 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61aaff5640 1 -- 172.21.15.5:0/3155769981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61b8004c80 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61cfdea740 10 client.18324 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc66644400 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.440+0000 7f61aaff5640 1 -- 172.21.15.5:0/3155769981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b8023030 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61aaff5640 1 -- 172.21.15.5:0/3155769981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f61b8030020 con 0x55bc665c4710 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61cfdea740 10 client.18324 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61cfdea740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61cfdea740 20 client.18324 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61cfdea740 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc66644b20 0x55bc66646f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61cfdea740 4 client.18324 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc66530a70 con 0x55bc66644b20 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61c659c640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc66644b20 0x55bc66646f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61c659c640 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc66644b20 0x55bc66646f40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db5ddff472c5ae5 server_cookie=67501e2adc9a2a00 in_seq=0 out_seq=0 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.441+0000 7f61aaff5640 10 client.18324 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61aaff5640 1 -- 172.21.15.5:0/3155769981 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bc66530a70 con 0x55bc66644b20 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61aaff5640 10 client.18324 handle_command_reply: tid=0 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 client.18324 shutdown 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 msgr2=0x55bc665e1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665e1000 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f61b8015010 tx=0x7f61b8010da0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 shutdown_connections 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55bc66644b20 0x55bc66646f40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc665cdc10 0x55bc66642c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bc665c5030 0x55bc665e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61aaff5640 0 client.18324 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 --2- 172.21.15.5:0/3155769981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc665c4710 0x55bc665e1000 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 >> 172.21.15.5:0/3155769981 conn(0x55bc665bee40 msgr2=0x55bc665cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 shutdown_connections 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 1 -- 172.21.15.5:0/3155769981 wait complete. 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cfdea740 20 client.18324 trim_cache size 0 max 16384 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stdout: "356d394d-083f-40cd-bf69-4f1d222113cb": { 2024-04-22T13:34:23.445 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:34:23.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "356d394d-083f-40cd-bf69-4f1d222113cb", 2024-04-22T13:34:23.446 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:34:23.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:34:23.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:34:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:34:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f61b00a3d10 msgr2=0x7f61b00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cca00640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f61b00a3d10 0x7f61b00a61d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005ab0 tx=0x7f61c0005a40 comp rx=0 tx=0).stop 2024-04-22T13:34:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 msgr2=0x7f61c8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cca00640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c8074790 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00d770 tx=0x7f61bc00dc30 comp rx=0 tx=0).stop 2024-04-22T13:34:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 shutdown_connections 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.442+0000 7f61cca00640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c8107060 0x7f61c8074790 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.443+0000 7f61cca00640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f61b00a3d10 0x7f61b00a61d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.443+0000 7f61cca00640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c8074cd0 0x7f61c81a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.443+0000 7f61cca00640 1 --2- 172.21.15.5:0/2513445641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61c8101f90 0x7f61c8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.443+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 >> 172.21.15.5:0/2513445641 conn(0x7f61c80fd810 msgr2=0x7f61c80ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.443+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 shutdown_connections 2024-04-22T13:34:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:23.443+0000 7f61cca00640 1 -- 172.21.15.5:0/2513445641 wait complete. 2024-04-22T13:34:23.451 INFO:tasks.cephfs.filesystem:scrub status for tag:356d394d-083f-40cd-bf69-4f1d222113cb - {'path': '/', 'tag': '356d394d-083f-40cd-bf69-4f1d222113cb', 'options': 'recursive,force'} 2024-04-22T13:34:25.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:24 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:34:25.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:24 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:25.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:24 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:34:25.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:24 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:25.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:24 smithi049 ceph-mon[31379]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:25.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:24 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:34:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:24 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:24 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:34:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:24 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:24 smithi107 ceph-mon[31859]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:24 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:34:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:24 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:24 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:34:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:24 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:24 smithi005 ceph-mon[28006]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:34:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:26 smithi049 ceph-mon[31379]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-22T13:34:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:26 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:34:27.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:26 smithi107 ceph-mon[31859]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-22T13:34:27.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:26 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:34:27.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:26 smithi005 ceph-mon[28006]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-22T13:34:27.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:26 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:34:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:28 smithi049 ceph-mon[31379]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 100 B/s wr, 11 op/s 2024-04-22T13:34:29.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:28 smithi107 ceph-mon[31859]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 100 B/s wr, 11 op/s 2024-04-22T13:34:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:28 smithi005 ceph-mon[28006]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 100 B/s wr, 11 op/s 2024-04-22T13:34:31.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:30 smithi049 ceph-mon[31379]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:34:31.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:30 smithi107 ceph-mon[31859]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:34:31.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:30 smithi005 ceph-mon[28006]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:34:33.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:32 smithi049 ceph-mon[31379]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:34:33.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:32 smithi107 ceph-mon[31859]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:34:33.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:32 smithi005 ceph-mon[28006]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:34:35.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:34 smithi049 ceph-mon[31379]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:35.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:34 smithi107 ceph-mon[31859]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:35.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:34 smithi005 ceph-mon[28006]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:34:37.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:36 smithi049 ceph-mon[31379]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:34:37.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:36 smithi107 ceph-mon[31859]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:34:37.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:36 smithi005 ceph-mon[28006]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:34:39.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:38 smithi049 ceph-mon[31379]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:39.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:38 smithi107 ceph-mon[31859]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:39.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:38 smithi005 ceph-mon[28006]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:41.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:40 smithi049 ceph-mon[31379]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:41.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:40 smithi107 ceph-mon[31859]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:41.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:40 smithi005 ceph-mon[28006]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:43.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:42 smithi049 ceph-mon[31379]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:43.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:42 smithi107 ceph-mon[31859]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:43.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:42 smithi005 ceph-mon[28006]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:45.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:44 smithi049 ceph-mon[31379]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:45.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:44 smithi107 ceph-mon[31859]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:44 smithi005 ceph-mon[28006]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:47.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:46 smithi049 ceph-mon[31379]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:47.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:46 smithi107 ceph-mon[31859]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:47.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:46 smithi005 ceph-mon[28006]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:49.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:48 smithi049 ceph-mon[31379]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:49.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:48 smithi107 ceph-mon[31859]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:48 smithi005 ceph-mon[28006]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:51.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:50 smithi049 ceph-mon[31379]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:51.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:50 smithi107 ceph-mon[31859]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:51.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:50 smithi005 ceph-mon[28006]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:53.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:52 smithi049 ceph-mon[31379]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:53.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:52 smithi107 ceph-mon[31859]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:52 smithi005 ceph-mon[28006]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:53.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:53.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 -- 172.21.15.5:0/4185735773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 msgr2=0x7fab341035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 --2- 172.21.15.5:0/4185735773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab341035a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fab30009900 tx=0x7fab3002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 -- 172.21.15.5:0/4185735773 shutdown_connections 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 --2- 172.21.15.5:0/4185735773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab3410c590 0x7fab3410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 --2- 172.21.15.5:0/4185735773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab34103ae0 0x7fab3410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 --2- 172.21.15.5:0/4185735773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab341035a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.574+0000 7fab3ac81640 1 -- 172.21.15.5:0/4185735773 >> 172.21.15.5:0/4185735773 conn(0x7fab34077fa0 msgr2=0x7fab340783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 -- 172.21.15.5:0/4185735773 shutdown_connections 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 -- 172.21.15.5:0/4185735773 wait complete. 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 Processor -- start 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 -- start start 2024-04-22T13:34:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab34198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab34103ae0 0x7fab34198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab3410c590 0x7fab3419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab341121d0 con 0x7fab341031c0 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab34112050 con 0x7fab3410c590 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab3ac81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab34112350 con 0x7fab34103ae0 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab38a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab34198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab38a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab34198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44680/0 (socket says 172.21.15.5:44680) 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.575+0000 7fab38a1d640 1 -- 172.21.15.5:0/2392566286 learned_addr learned my addr 172.21.15.5:0/2392566286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab2bfff640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab34103ae0 0x7fab34198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab3921e640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab3410c590 0x7fab3419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab38a1d640 1 -- 172.21.15.5:0/2392566286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab34103ae0 msgr2=0x7fab34198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab38a1d640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab34103ae0 0x7fab34198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab38a1d640 1 -- 172.21.15.5:0/2392566286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab3410c590 msgr2=0x7fab3419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab38a1d640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab3410c590 0x7fab3419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab38a1d640 1 -- 172.21.15.5:0/2392566286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419d720 con 0x7fab341031c0 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab38a1d640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab34198730 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fab30005940 tx=0x7fab3002c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab29ffb640 1 -- 172.21.15.5:0/2392566286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30043020 con 0x7fab341031c0 2024-04-22T13:34:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab29ffb640 1 -- 172.21.15.5:0/2392566286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab30004e00 con 0x7fab341031c0 2024-04-22T13:34:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419d9b0 con 0x7fab341031c0 2024-04-22T13:34:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab29ffb640 1 -- 172.21.15.5:0/2392566286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30003970 con 0x7fab341031c0 2024-04-22T13:34:53.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.576+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a52a0 con 0x7fab341031c0 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab29ffb640 1 -- 172.21.15.5:0/2392566286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fab30046020 con 0x7fab341031c0 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 Processor -- start 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 -- start start 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab29ffb640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab1c0a3d20 0x7fab1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d45b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d468950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d46b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e86d3c01e0 con 0x55e86d45b2b0 2024-04-22T13:34:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e86d35d390 con 0x55e86d45bbd0 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.579+0000 7fab3d874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e86d3bc1a0 con 0x55e86d468e90 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab3921e640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab1c0a3d20 0x7fab1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab29ffb640 1 -- 172.21.15.5:0/2392566286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fab300f2620 con 0x7fab341031c0 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d468950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab38a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d45b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab3921e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d46b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d468950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:41104/0 (socket says 172.21.15.5:41104) 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 -- 172.21.15.5:0/464012416 learned_addr learned my addr 172.21.15.5:0/464012416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 -- 172.21.15.5:0/464012416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 msgr2=0x55e86d46b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d46b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 -- 172.21.15.5:0/464012416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 msgr2=0x55e86d45b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d45b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 -- 172.21.15.5:0/464012416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86d19c7d0 con 0x55e86d45bbd0 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab38a1d640 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d45b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab3921e640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab1c0a3d20 0x7fab1c0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009fd0 tx=0x7fab2c013b00 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.580+0000 7fab2bfff640 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d468950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab24009c20 tx=0x7fab24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a7139663540efa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab1affd640 1 -- 172.21.15.5:0/464012416 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2401e070 con 0x55e86d45bbd0 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab1affd640 1 -- 172.21.15.5:0/464012416 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24019ae0 con 0x55e86d45bbd0 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab1affd640 1 -- 172.21.15.5:0/464012416 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400e8e0 con 0x55e86d45bbd0 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 -- 172.21.15.5:0/464012416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 msgr2=0x55e86d468950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d468950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab24009c20 tx=0x7fab24011810 comp rx=0 tx=0).stop 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 -- 172.21.15.5:0/464012416 shutdown_connections 2024-04-22T13:34:53.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d46b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d468950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 --2- 172.21.15.5:0/464012416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d45b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.581+0000 7fab3d874740 1 -- 172.21.15.5:0/464012416 >> 172.21.15.5:0/464012416 conn(0x55e86d455cb0 msgr2=0x55e86d456090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 -- 172.21.15.5:0/464012416 shutdown_connections 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 -- 172.21.15.5:0/464012416 wait complete. 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 -- start start 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 Processor -- start 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d465e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 0x55e86d4668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e86d46bc50 con 0x55e86d468e90 2024-04-22T13:34:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e86d3792f0 con 0x55e86d45bbd0 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e86d46bdd0 con 0x55e86d45b2b0 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3d874740 10 client.0 _ll_register_callbacks cb 0x55e86d386580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.582+0000 7fab3921e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 0x55e86d4668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 0x55e86d4668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44692/0 (socket says 172.21.15.5:44692) 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 -- 172.21.15.5:0/630775848 learned_addr learned my addr 172.21.15.5:0/630775848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab38a1d640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d465e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 -- 172.21.15.5:0/630775848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 msgr2=0x55e86d465e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d465e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 -- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 msgr2=0x55e86d4663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 -- 172.21.15.5:0/630775848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86d0d23f0 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab2bfff640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3921e640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 0x55e86d4668f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ff80 tx=0x7fab2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4627d6c2aa9df26 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab13fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab13fff640 1 -- 172.21.15.5:0/630775848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010430 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab13fff640 1 -- 172.21.15.5:0/630775848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fab2c0105d0 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab24019ae0 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab13fff640 1 -- 172.21.15.5:0/630775848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab2c0107d0 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3d874740 10 client.18336 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.583+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e86d0920f0 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab13fff640 1 -- 172.21.15.5:0/630775848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24019ae0 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab13fff640 1 -- 172.21.15.5:0/630775848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab2c01f070 con 0x55e86d468e90 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab3d874740 10 client.18336 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab3d874740 10 client.18336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab3d874740 20 client.18336 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab3d874740 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d478f30 0x55e86d47b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab3d874740 4 client.18336 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e86d3cdc00 con 0x55e86d478f30 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.584+0000 7fab38a1d640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d478f30 0x55e86d47b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.585+0000 7fab38a1d640 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d478f30 0x55e86d47b350 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752bb3bf3b9eb44a server_cookie=276b26d439d86103 in_seq=0 out_seq=0 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.585+0000 7fab13fff640 10 client.18336 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.585+0000 7fab13fff640 1 -- 172.21.15.5:0/630775848 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e86d3cdc00 con 0x55e86d478f30 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.585+0000 7fab13fff640 10 client.18336 handle_command_reply: tid=0 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.585+0000 7fab3d874740 1 client.18336 shutdown 2024-04-22T13:34:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 msgr2=0x55e86d4668f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 0x55e86d4668f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ff80 tx=0x7fab2c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 shutdown_connections 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d478f30 0x55e86d47b350 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d468e90 0x55e86d4668f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab13fff640 0 client.18336 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 --2- 172.21.15.5:0/630775848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d465e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 >> 172.21.15.5:0/630775848 conn(0x55e86d455cb0 msgr2=0x55e86d457a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 shutdown_connections 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 1 -- 172.21.15.5:0/630775848 wait complete. 2024-04-22T13:34:53.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.586+0000 7fab3d874740 20 client.18336 trim_cache size 0 max 16384 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.603+0000 7fab3d874740 1 Processor -- start 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.603+0000 7fab3d874740 1 -- start start 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab3d874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d474bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab3d874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 0x55e86d454bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab3d874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d468e90 0x55e86d480c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab3d874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ab900 con 0x55e86d45bbd0 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab3d874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ad440 con 0x55e86d468e90 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab3d874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab341abfd0 con 0x55e86d45b2b0 2024-04-22T13:34:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 0x55e86d454bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 0x55e86d454bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44704/0 (socket says 172.21.15.5:44704) 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 -- 172.21.15.5:0/177937697 learned_addr learned my addr 172.21.15.5:0/177937697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab38a1d640 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d474bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 -- 172.21.15.5:0/177937697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 msgr2=0x55e86d474bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d474bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 -- 172.21.15.5:0/177937697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d468e90 msgr2=0x55e86d480c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d468e90 0x55e86d480c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 -- 172.21.15.5:0/177937697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86d4643c0 con 0x55e86d45bbd0 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab2bfff640 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 0x55e86d454bf0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fab24009c20 tx=0x7fab24005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a850ec7810687c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab1a7fc640 1 -- 172.21.15.5:0/177937697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2401e040 con 0x55e86d45bbd0 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab1a7fc640 1 -- 172.21.15.5:0/177937697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24005580 con 0x55e86d45bbd0 2024-04-22T13:34:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.604+0000 7fab1a7fc640 1 -- 172.21.15.5:0/177937697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400e5d0 con 0x55e86d45bbd0 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 -- 172.21.15.5:0/177937697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 msgr2=0x55e86d454bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 0x55e86d454bf0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fab24009c20 tx=0x7fab24005d30 comp rx=0 tx=0).stop 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 -- 172.21.15.5:0/177937697 shutdown_connections 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45bbd0 0x55e86d454bf0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d468e90 0x55e86d480c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 --2- 172.21.15.5:0/177937697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d45b2b0 0x55e86d474bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 -- 172.21.15.5:0/177937697 >> 172.21.15.5:0/177937697 conn(0x55e86d455cb0 msgr2=0x55e86d4544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 -- 172.21.15.5:0/177937697 shutdown_connections 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 -- 172.21.15.5:0/177937697 wait complete. 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.605+0000 7fab3d874740 1 -- start start 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 Processor -- start 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d462e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d4de490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab341acd70 con 0x55e86d45b2b0 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ac6a0 con 0x55e86d45bbd0 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab341a9900 con 0x55e86d468e90 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab3d874740 10 client.0 _ll_register_callbacks cb 0x7fab2c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d462e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d462e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44718/0 (socket says 172.21.15.5:44718) 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 -- 172.21.15.5:0/2874826198 learned_addr learned my addr 172.21.15.5:0/2874826198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 -- 172.21.15.5:0/2874826198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 msgr2=0x55e86d4de490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d4de490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 -- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 msgr2=0x55e86d4633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 -- 172.21.15.5:0/2874826198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86d463c70 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab38a1d640 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d462e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fab300be780 tx=0x7fab300baa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96b792e3816f064 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.606+0000 7fab13fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab13fff640 1 -- 172.21.15.5:0/2874826198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab300bfb20 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab24005580 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab13fff640 1 -- 172.21.15.5:0/2874826198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fab300efff0 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab13fff640 1 -- 172.21.15.5:0/2874826198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab300c2ba0 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 10 client.18342 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e86d4dfb10 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab13fff640 1 -- 172.21.15.5:0/2874826198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24005580 con 0x55e86d45b2b0 2024-04-22T13:34:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab13fff640 1 -- 172.21.15.5:0/2874826198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab300052f0 con 0x55e86d45b2b0 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 10 client.18342 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 20 client.18342 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d4e0290 0x55e86d4e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 4 client.18342 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e86d3c7a70 con 0x55e86d4e0290 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.607+0000 7fab2bfff640 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d4e0290 0x55e86d4e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab2bfff640 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d4e0290 0x55e86d4e26f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574dfd73253ebe0a server_cookie=e7f6f411ec4b8347 in_seq=0 out_seq=0 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab13fff640 10 client.18342 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab13fff640 1 -- 172.21.15.5:0/2874826198 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e86d3c7a70 con 0x55e86d4e0290 2024-04-22T13:34:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab13fff640 10 client.18342 handle_command_reply: tid=0 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 client.18342 shutdown 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 msgr2=0x55e86d462e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d462e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fab300be780 tx=0x7fab300baa90 comp rx=0 tx=0).stop 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 shutdown_connections 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e86d4e0290 0x55e86d4e26f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e86d468e90 0x55e86d4de490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e86d45bbd0 0x55e86d4633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab13fff640 0 client.18342 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 --2- 172.21.15.5:0/2874826198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86d45b2b0 0x55e86d462e60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 >> 172.21.15.5:0/2874826198 conn(0x55e86d455cb0 msgr2=0x55e86d4666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 shutdown_connections 2024-04-22T13:34:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.608+0000 7fab3d874740 1 -- 172.21.15.5:0/2874826198 wait complete. 2024-04-22T13:34:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3d874740 20 client.18342 trim_cache size 0 max 16384 2024-04-22T13:34:53.612 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:34:53.612 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:34:53.612 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:34:53.612 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab1c0a3d20 msgr2=0x7fab1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab1c0a3d20 0x7fab1c0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009fd0 tx=0x7fab2c013b00 comp rx=0 tx=0).stop 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 msgr2=0x7fab34198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab34198730 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fab30005940 tx=0x7fab3002c690 comp rx=0 tx=0).stop 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 shutdown_connections 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fab1c0a3d20 0x7fab1c0a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab3410c590 0x7fab3419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab34103ae0 0x7fab34198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 --2- 172.21.15.5:0/2392566286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341031c0 0x7fab34198730 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 >> 172.21.15.5:0/2392566286 conn(0x7fab34077fa0 msgr2=0x7fab340fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 shutdown_connections 2024-04-22T13:34:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.609+0000 7fab3ac81640 1 -- 172.21.15.5:0/2392566286 wait complete. 2024-04-22T13:34:53.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:34:53.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 -- 172.21.15.5:0/814298467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0107240 msgr2=0x7f11f010ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 --2- 172.21.15.5:0/814298467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0107240 0x7f11f010ee10 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009ca0 tx=0x7f11e802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 -- 172.21.15.5:0/814298467 shutdown_connections 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 --2- 172.21.15.5:0/814298467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0107240 0x7f11f010ee10 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 --2- 172.21.15.5:0/814298467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11f0102b20 0x7f11f0106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 --2- 172.21.15.5:0/814298467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0102170 0x7f11f0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.723+0000 7f11f676d640 1 -- 172.21.15.5:0/814298467 >> 172.21.15.5:0/814298467 conn(0x7f11f00fd9d0 msgr2=0x7f11f00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.724+0000 7f11f676d640 1 -- 172.21.15.5:0/814298467 shutdown_connections 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.724+0000 7f11f676d640 1 -- 172.21.15.5:0/814298467 wait complete. 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.724+0000 7f11f676d640 1 Processor -- start 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.724+0000 7f11f676d640 1 -- start start 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.724+0000 7f11f676d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 0x7f11f0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11f0107240 0x7f11f0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f019cf10 0x7f11f019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01121b0 con 0x7f11f0102170 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0112030 con 0x7f11f0107240 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0112330 con 0x7f11f019cf10 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 0x7f11f0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 0x7f11f0198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44738/0 (socket says 172.21.15.5:44738) 2024-04-22T13:34:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 -- 172.21.15.5:0/2071040563 learned_addr learned my addr 172.21.15.5:0/2071040563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11ef7fe640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11f0107240 0x7f11f0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f4d0a640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f019cf10 0x7f11f019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 -- 172.21.15.5:0/2071040563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f019cf10 msgr2=0x7f11f019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f019cf10 0x7f11f019d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 -- 172.21.15.5:0/2071040563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11f0107240 msgr2=0x7f11f0198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11f0107240 0x7f11f0198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 -- 172.21.15.5:0/2071040563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f019da00 con 0x7f11f0102170 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11effff640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 0x7f11f0198650 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800ccc0 con 0x7f11f0102170 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d800ce60 con 0x7f11f0102170 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f019dcf0 con 0x7f11f0102170 2024-04-22T13:34:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8010430 con 0x7f11f0102170 2024-04-22T13:34:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.725+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f01a54e0 con 0x7f11f0102170 2024-04-22T13:34:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.726+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f0069250 con 0x7f11f0102170 2024-04-22T13:34:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.727+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f11d80046c0 con 0x7f11f0102170 2024-04-22T13:34:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.728+0000 7f11ed7fa640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11dc0a3da0 0x7f11dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.728+0000 7f11ef7fe640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11dc0a3da0 0x7f11dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.728+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f11d80d0bb0 con 0x7f11f0102170 2024-04-22T13:34:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.729+0000 7f11ef7fe640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11dc0a3da0 0x7f11dc0a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f11f0101100 tx=0x7f11e000a2d0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.733+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11d809d1f0 con 0x7f11f0102170 2024-04-22T13:34:53.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.961+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11f010a530 con 0x7f11f0102170 2024-04-22T13:34:53.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.962+0000 7f11ed7fa640 1 -- 172.21.15.5:0/2071040563 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f11d809caf0 con 0x7f11f0102170 2024-04-22T13:34:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:34:53.957 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:34:53.958 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:34:53.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.963+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11dc0a3da0 msgr2=0x7f11dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.963+0000 7f11f676d640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11dc0a3da0 0x7f11dc0a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f11f0101100 tx=0x7f11e000a2d0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.963+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 msgr2=0x7f11f0198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:53.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.963+0000 7f11f676d640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 0x7f11f0198650 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800edf0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 shutdown_connections 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11dc0a3da0 0x7f11dc0a6260 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f019cf10 0x7f11f019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11f0107240 0x7f11f0198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 --2- 172.21.15.5:0/2071040563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11f0102170 0x7f11f0198650 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 >> 172.21.15.5:0/2071040563 conn(0x7f11f00fd9d0 msgr2=0x7f11f00ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 shutdown_connections 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:53.964+0000 7f11f676d640 1 -- 172.21.15.5:0/2071040563 wait complete. 2024-04-22T13:34:53.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:34:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 -- 172.21.15.5:0/1084532204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a141044f0 msgr2=0x7f5a1410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1084532204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a141044f0 0x7f5a1410eef0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009930 tx=0x7f5a0c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 -- 172.21.15.5:0/1084532204 shutdown_connections 2024-04-22T13:34:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1084532204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a141044f0 0x7f5a1410eef0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1084532204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a14103b50 0x7f5a14103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1084532204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14102270 0x7f5a14102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 -- 172.21.15.5:0/1084532204 >> 172.21.15.5:0/1084532204 conn(0x7f5a140fda10 msgr2=0x7f5a140ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 -- 172.21.15.5:0/1084532204 shutdown_connections 2024-04-22T13:34:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.082+0000 7f5a19c81640 1 -- 172.21.15.5:0/1084532204 wait complete. 2024-04-22T13:34:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 Processor -- start 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 -- start start 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 0x7f5a1419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14103b50 0x7f5a1419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a141044f0 0x7f5a141985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14112340 con 0x7f5a14102270 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141121c0 con 0x7f5a141044f0 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.083+0000 7f5a19c81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141124c0 con 0x7f5a14103b50 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 0x7f5a1419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 0x7f5a1419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44764/0 (socket says 172.21.15.5:44764) 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 -- 172.21.15.5:0/1409356793 learned_addr learned my addr 172.21.15.5:0/1409356793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a141044f0 0x7f5a141985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14103b50 0x7f5a1419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 -- 172.21.15.5:0/1409356793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14103b50 msgr2=0x7f5a1419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14103b50 0x7f5a1419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 -- 172.21.15.5:0/1409356793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a141044f0 msgr2=0x7f5a141985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a141044f0 0x7f5a141985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 -- 172.21.15.5:0/1409356793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a14198d60 con 0x7f5a14102270 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14103b50 0x7f5a1419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 0x7f5a1419e3e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400ba50 tx=0x7f5a0400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a10ff9640 1 -- 172.21.15.5:0/1409356793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04019070 con 0x7f5a14102270 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a14199050 con 0x7f5a14102270 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a10ff9640 1 -- 172.21.15.5:0/1409356793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0400cbe0 con 0x7f5a14102270 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a10ff9640 1 -- 172.21.15.5:0/1409356793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a040075d0 con 0x7f5a14102270 2024-04-22T13:34:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.084+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141a5320 con 0x7f5a14102270 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.086+0000 7f5a10ff9640 1 -- 172.21.15.5:0/1409356793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5a0401c070 con 0x7f5a14102270 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 Processor -- start 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 -- start start 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b4298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b432430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 0x560c7b434e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7b38e1f0 con 0x560c7b432a00 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c7b32b4b0 con 0x560c7b429e80 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a1c873740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c7b38a1b0 con 0x560c7b4294d0 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 0x560c7b434e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a13fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b4298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 0x560c7b434e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44770/0 (socket says 172.21.15.5:44770) 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2706050512 learned_addr learned my addr 172.21.15.5:0/2706050512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a10ff9640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f59fc0a3d20 0x7f59fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a137fe640 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b432430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2706050512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 msgr2=0x560c7b4298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b4298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2706050512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 msgr2=0x560c7b432430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b432430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f59fc0a3d20 0x7f59fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2706050512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7b31d1b0 con 0x560c7b432a00 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.087+0000 7f5a10ff9640 1 -- 172.21.15.5:0/1409356793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5a040cfcc0 con 0x7f5a14102270 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 0x560c7b434e50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800bbb0 tx=0x7f5a0800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe6e7c9615b7f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f59f9ffb640 1 -- 172.21.15.5:0/2706050512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017620 con 0x560c7b432a00 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f59fc0a3d20 0x7f59fc0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c036040 tx=0x7f5a0c009260 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f59f9ffb640 1 -- 172.21.15.5:0/2706050512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a08017c70 con 0x560c7b432a00 2024-04-22T13:34:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f59f9ffb640 1 -- 172.21.15.5:0/2706050512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800fc20 con 0x560c7b432a00 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f5a1c873740 1 -- 172.21.15.5:0/2706050512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 msgr2=0x560c7b434e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.088+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 0x560c7b434e50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800bbb0 tx=0x7f5a0800d710 comp rx=0 tx=0).stop 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- 172.21.15.5:0/2706050512 shutdown_connections 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b432a00 0x560c7b434e50 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b432430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2706050512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b4298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- 172.21.15.5:0/2706050512 >> 172.21.15.5:0/2706050512 conn(0x560c7b423ba0 msgr2=0x560c7b423f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- 172.21.15.5:0/2706050512 shutdown_connections 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- 172.21.15.5:0/2706050512 wait complete. 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- start start 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 Processor -- start 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 0x560c7b432240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b42f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b432a00 0x560c7b42fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7b435970 con 0x560c7b4294d0 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560c7b347500 con 0x560c7b429e80 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c7b435af0 con 0x560c7b432a00 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a1c873740 10 client.0 _ll_register_callbacks cb 0x560c7b315b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 0x560c7b432240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 0x560c7b432240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44774/0 (socket says 172.21.15.5:44774) 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.089+0000 7f5a137fe640 1 -- 172.21.15.5:0/1421021704 learned_addr learned my addr 172.21.15.5:0/1421021704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b42f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b432a00 0x560c7b42fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a137fe640 1 -- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b432a00 msgr2=0x560c7b42fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b432a00 0x560c7b42fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a137fe640 1 -- 172.21.15.5:0/1421021704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 msgr2=0x560c7b42f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b42f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a137fe640 1 -- 172.21.15.5:0/1421021704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7b0a03f0 con 0x560c7b4294d0 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b432a00 0x560c7b42fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b42f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 0x560c7b432240 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5a040cd580 tx=0x7f5a0409b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fa179a7c614ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f59f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1421021704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a040d1040 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1421021704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5a040a4550 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a08017c70 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1421021704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a040a49e0 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a1c873740 10 client.18360 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c7b16a7d0 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.090+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1421021704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a08017c70 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1421021704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5a040cdc80 con 0x560c7b4294d0 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a1c873740 10 client.18360 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a1c873740 10 client.18360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a1c873740 20 client.18360 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b442af0 0x560c7b444f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a1c873740 4 client.18360 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7b32d930 con 0x560c7b442af0 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.091+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b442af0 0x560c7b444f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b442af0 0x560c7b444f20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=199c29d7d08dfc5c server_cookie=c1d9fb231488d873 in_seq=0 out_seq=0 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f59f2ffd640 10 client.18360 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1421021704 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c7b32d930 con 0x560c7b442af0 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f59f2ffd640 10 client.18360 handle_command_reply: tid=0 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f5a1c873740 1 client.18360 shutdown 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 msgr2=0x560c7b432240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.092+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 0x560c7b432240 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5a040cd580 tx=0x7f5a0409b840 comp rx=0 tx=0).stop 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 shutdown_connections 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b432a00 0x560c7b42fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b429e80 0x560c7b42f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b442af0 0x560c7b444f20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1421021704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b4294d0 0x560c7b432240 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 >> 172.21.15.5:0/1421021704 conn(0x560c7b423ba0 msgr2=0x560c7b4258f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f59f2ffd640 0 client.18360 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 shutdown_connections 2024-04-22T13:34:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 1 -- 172.21.15.5:0/1421021704 wait complete. 2024-04-22T13:34:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.093+0000 7f5a1c873740 20 client.18360 trim_cache size 0 max 16384 2024-04-22T13:34:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 Processor -- start 2024-04-22T13:34:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 -- start start 2024-04-22T13:34:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b425ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b431d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141adbb0 con 0x560c7b429e80 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141af6f0 con 0x560c7b432a00 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a1c873740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141ae280 con 0x560c7b4294d0 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a12ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b425ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a12ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b425ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44778/0 (socket says 172.21.15.5:44778) 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.106+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2117084929 learned_addr learned my addr 172.21.15.5:0/2117084929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a137fe640 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2117084929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 msgr2=0x560c7b43e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2117084929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 msgr2=0x560c7b431d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b431d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a12ffd640 1 -- 172.21.15.5:0/2117084929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7b42db20 con 0x560c7b429e80 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b425ac0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5a08019040 tx=0x7f5a0800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17760ca430c0b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f59f97fa640 1 -- 172.21.15.5:0/2117084929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800b660 con 0x560c7b429e80 2024-04-22T13:34:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f59f97fa640 1 -- 172.21.15.5:0/2117084929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0802e070 con 0x560c7b429e80 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f59f97fa640 1 -- 172.21.15.5:0/2117084929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08005280 con 0x560c7b429e80 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a1c873740 1 -- 172.21.15.5:0/2117084929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 msgr2=0x560c7b425ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.107+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b425ac0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5a08019040 tx=0x7f5a0800baf0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- 172.21.15.5:0/2117084929 shutdown_connections 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b425ac0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b431d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 --2- 172.21.15.5:0/2117084929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- 172.21.15.5:0/2117084929 >> 172.21.15.5:0/2117084929 conn(0x560c7b423ba0 msgr2=0x560c7b422c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- 172.21.15.5:0/2117084929 shutdown_connections 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- 172.21.15.5:0/2117084929 wait complete. 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- start start 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 Processor -- start 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b436460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b4369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141af020 con 0x560c7b429e80 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141ae950 con 0x560c7b432a00 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141abbb0 con 0x560c7b4294d0 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a1c873740 10 client.0 _ll_register_callbacks cb 0x7f5a0409bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a12ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b436460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a12ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b436460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44794/0 (socket says 172.21.15.5:44794) 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.108+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1341056474 learned_addr learned my addr 172.21.15.5:0/1341056474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b4369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 msgr2=0x560c7b43b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1341056474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 msgr2=0x560c7b4369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b4369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1341056474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7b4371d0 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a13fff640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b4369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b436460 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800f7c0 tx=0x7f5a0800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde1cfa9f08ddbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f59f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1341056474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800fad0 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1341056474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5a0800fc70 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0802e070 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1341056474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a0800b370 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a1c873740 10 client.18366 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c7b4a8ff0 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1341056474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0802e070 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.109+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1341056474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5a08029040 con 0x560c7b429e80 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 10 client.18366 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 10 client.18366 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:34:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 20 client.18366 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b4a9590 0x560c7b4ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 4 client.18366 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c7b39bc10 con 0x560c7b4a9590 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b4a9590 0x560c7b4ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f59f2ffd640 10 client.18366 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a137fe640 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b4a9590 0x560c7b4ab9b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6326b21cf6560155 server_cookie=e2966caa791a6358 in_seq=0 out_seq=0 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f59f2ffd640 1 -- 172.21.15.5:0/1341056474 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c7b39bc10 con 0x560c7b4a9590 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f59f2ffd640 10 client.18366 handle_command_reply: tid=0 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.110+0000 7f5a1c873740 1 client.18366 shutdown 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 msgr2=0x560c7b436460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b436460 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800f7c0 tx=0x7f5a0800bed0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 shutdown_connections 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x560c7b4a9590 0x560c7b4ab9b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7b429e80 0x560c7b436460 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560c7b432a00 0x560c7b4369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f59f2ffd640 0 client.18366 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 --2- 172.21.15.5:0/1341056474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7b4294d0 0x560c7b43b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 >> 172.21.15.5:0/1341056474 conn(0x560c7b423ba0 msgr2=0x560c7b422370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 shutdown_connections 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 1 -- 172.21.15.5:0/1341056474 wait complete. 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a1c873740 20 client.18366 trim_cache size 0 max 16384 2024-04-22T13:34:54.113 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f59fc0a3d20 msgr2=0x7f59fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f59fc0a3d20 0x7f59fc0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c036040 tx=0x7f5a0c009260 comp rx=0 tx=0).stop 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 msgr2=0x7f5a1419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 0x7f5a1419e3e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400ba50 tx=0x7f5a0400bf10 comp rx=0 tx=0).stop 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 shutdown_connections 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f59fc0a3d20 0x7f59fc0a61e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a141044f0 0x7f5a141985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a14103b50 0x7f5a1419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 --2- 172.21.15.5:0/1409356793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a14102270 0x7f5a1419e3e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 >> 172.21.15.5:0/1409356793 conn(0x7f5a140fda10 msgr2=0x7f5a140fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 shutdown_connections 2024-04-22T13:34:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.111+0000 7f5a19c81640 1 -- 172.21.15.5:0/1409356793 wait complete. 2024-04-22T13:34:54.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:34:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 -- 172.21.15.5:0/2608332533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 msgr2=0x7f54b8101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 --2- 172.21.15.5:0/2608332533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8101560 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009930 tx=0x7f54b402bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 -- 172.21.15.5:0/2608332533 shutdown_connections 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 --2- 172.21.15.5:0/2608332533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 0x7f54b810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 --2- 172.21.15.5:0/2608332533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54b8102a60 0x7f54b8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 --2- 172.21.15.5:0/2608332533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8101560 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.225+0000 7f54bf6df640 1 -- 172.21.15.5:0/2608332533 >> 172.21.15.5:0/2608332533 conn(0x7f54b8077f70 msgr2=0x7f54b8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 -- 172.21.15.5:0/2608332533 shutdown_connections 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 -- 172.21.15.5:0/2608332533 wait complete. 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 Processor -- start 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 -- start start 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54b8102a60 0x7f54b8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 0x7f54b81a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8111180 con 0x7f54b8101180 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8111000 con 0x7f54b8102a60 2024-04-22T13:34:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bf6df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8111300 con 0x7f54b8103400 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.226+0000 7f54bd47b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44810/0 (socket says 172.21.15.5:44810) 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 -- 172.21.15.5:0/4079289034 learned_addr learned my addr 172.21.15.5:0/4079289034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 0x7f54b81a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54b8102a60 0x7f54b8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 -- 172.21.15.5:0/4079289034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 msgr2=0x7f54b81a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 0x7f54b81a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 -- 172.21.15.5:0/4079289034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54b8102a60 msgr2=0x7f54b8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54b8102a60 0x7f54b8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 -- 172.21.15.5:0/4079289034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b81a4c00 con 0x7f54b8101180 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 0x7f54b81a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bd47b640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8199640 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009230 tx=0x7f54b40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54ae7fc640 1 -- 172.21.15.5:0/4079289034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4043020 con 0x7f54b8101180 2024-04-22T13:34:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54ae7fc640 1 -- 172.21.15.5:0/4079289034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b402cb70 con 0x7f54b8101180 2024-04-22T13:34:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b81a4dd0 con 0x7f54b8101180 2024-04-22T13:34:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54ae7fc640 1 -- 172.21.15.5:0/4079289034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003b50 con 0x7f54b8101180 2024-04-22T13:34:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.227+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b81a5280 con 0x7f54b8101180 2024-04-22T13:34:54.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54ae7fc640 1 -- 172.21.15.5:0/4079289034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f54b4046020 con 0x7f54b8101180 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 Processor -- start 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 -- start start 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c22e80 0x558ab6c2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ab6b871f0 con 0x558ab6c224d0 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558ab6b244b0 con 0x558ab6c2ba00 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54c23a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ab6b831b0 con 0x558ab6c22e80 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54bdc7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54bdc7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44820/0 (socket says 172.21.15.5:44820) 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54bdc7c640 1 -- 172.21.15.5:0/1605558660 learned_addr learned my addr 172.21.15.5:0/1605558660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54bd47b640 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c22e80 0x558ab6c2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.230+0000 7f54ae7fc640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f54a40a3da0 0x7f54a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 -- 172.21.15.5:0/1605558660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c22e80 msgr2=0x558ab6c2b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c22e80 0x558ab6c2b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 -- 172.21.15.5:0/1605558660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 msgr2=0x558ab6c2de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c2de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 -- 172.21.15.5:0/1605558660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab6b161b0 con 0x558ab6c224d0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f54a40a3da0 0x7f54a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54ae7fc640 1 -- 172.21.15.5:0/4079289034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f54b40f28b0 con 0x7f54b8101180 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c228b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f54b000f4b0 tx=0x7f54b000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8e4d5732b89e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54a37fe640 1 -- 172.21.15.5:0/1605558660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b001c400 con 0x558ab6c224d0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54a37fe640 1 -- 172.21.15.5:0/1605558660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b00124b0 con 0x558ab6c224d0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.231+0000 7f54a37fe640 1 -- 172.21.15.5:0/1605558660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0010610 con 0x558ab6c224d0 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 -- 172.21.15.5:0/1605558660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 msgr2=0x558ab6c228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f54a40a3da0 0x7f54a40a6260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f54b0020000 tx=0x7f54b001d000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c228b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f54b000f4b0 tx=0x7f54b000f970 comp rx=0 tx=0).stop 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 -- 172.21.15.5:0/1605558660 shutdown_connections 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c22e80 0x558ab6c2b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1605558660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c228b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 -- 172.21.15.5:0/1605558660 >> 172.21.15.5:0/1605558660 conn(0x558ab6c1cba0 msgr2=0x558ab6c1cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 -- 172.21.15.5:0/1605558660 shutdown_connections 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 -- 172.21.15.5:0/1605558660 wait complete. 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 -- start start 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.232+0000 7f54c23a8740 1 Processor -- start 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c2b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c22e80 0x558ab6c286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c28cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ab6c2e970 con 0x558ab6c224d0 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558ab6b40500 con 0x558ab6c22e80 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ab6c2eaf0 con 0x558ab6c2ba00 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 10 client.0 _ll_register_callbacks cb 0x558ab6b0eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c2b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c2b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44822/0 (socket says 172.21.15.5:44822) 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 -- 172.21.15.5:0/554579016 learned_addr learned my addr 172.21.15.5:0/554579016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c22e80 0x558ab6c286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c28cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 -- 172.21.15.5:0/554579016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 msgr2=0x558ab6c28cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c28cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 -- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c22e80 msgr2=0x558ab6c286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c22e80 0x558ab6c286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 -- 172.21.15.5:0/554579016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab68993f0 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c28cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54bd47b640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c2b330 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f54b40f0170 tx=0x7f54b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe5a8937da80092 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54a0ff9640 1 -- 172.21.15.5:0/554579016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40be2c0 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54a0ff9640 1 -- 172.21.15.5:0/554579016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f54b4005490 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.233+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b00124b0 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54a0ff9640 1 -- 172.21.15.5:0/554579016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54b4005690 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 10 client.18378 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ab69637d0 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54a0ff9640 1 -- 172.21.15.5:0/554579016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b00124b0 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54a0ff9640 1 -- 172.21.15.5:0/554579016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f54b40c7140 con 0x558ab6c224d0 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 10 client.18378 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 10 client.18378 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 20 client.18378 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6c3df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 4 client.18378 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab6b26930 con 0x558ab6c3bb40 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.234+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6c3df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.235+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6c3df70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3079efd0e8f7fa70 server_cookie=88b02e5baaa890b3 in_seq=0 out_seq=0 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.235+0000 7f54a0ff9640 10 client.18378 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.235+0000 7f54a0ff9640 1 -- 172.21.15.5:0/554579016 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ab6b26930 con 0x558ab6c3bb40 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.235+0000 7f54a0ff9640 10 client.18378 handle_command_reply: tid=0 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.235+0000 7f54c23a8740 1 client.18378 shutdown 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 msgr2=0x558ab6c2b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c2b330 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f54b40f0170 tx=0x7f54b4005b50 comp rx=0 tx=0).stop 2024-04-22T13:34:54.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 shutdown_connections 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6c3df70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c28cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54a0ff9640 0 client.18378 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c22e80 0x558ab6c286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 --2- 172.21.15.5:0/554579016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c224d0 0x558ab6c2b330 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 >> 172.21.15.5:0/554579016 conn(0x558ab6c1cba0 msgr2=0x558ab6c1e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 shutdown_connections 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 1 -- 172.21.15.5:0/554579016 wait complete. 2024-04-22T13:34:54.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.236+0000 7f54c23a8740 20 client.18378 trim_cache size 0 max 16384 2024-04-22T13:34:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.249+0000 7f54c23a8740 1 Processor -- start 2024-04-22T13:34:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.249+0000 7f54c23a8740 1 -- start start 2024-04-22T13:34:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c224d0 0x558ab6c37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c34120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c2adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54c23a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54b81adb10 con 0x558ab6c22e80 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54c23a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54b81af650 con 0x558ab6c224d0 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54c23a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54b81ae1e0 con 0x558ab6c2ba00 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c34120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c34120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44834/0 (socket says 172.21.15.5:44834) 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 -- 172.21.15.5:0/1097860788 learned_addr learned my addr 172.21.15.5:0/1097860788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bd47b640 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c224d0 0x558ab6c37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 -- 172.21.15.5:0/1097860788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 msgr2=0x558ab6c2adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c2adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 -- 172.21.15.5:0/1097860788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c224d0 msgr2=0x558ab6c37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c224d0 0x558ab6c37830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 -- 172.21.15.5:0/1097860788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab6c1ee90 con 0x558ab6c22e80 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c34120 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x558ab6c27f20 tx=0x7f54a800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57057515f27f7a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54a2ffd640 1 -- 172.21.15.5:0/1097860788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a80140d0 con 0x558ab6c22e80 2024-04-22T13:34:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54a2ffd640 1 -- 172.21.15.5:0/1097860788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a8014270 con 0x558ab6c22e80 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.250+0000 7f54a2ffd640 1 -- 172.21.15.5:0/1097860788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8004730 con 0x558ab6c22e80 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 -- 172.21.15.5:0/1097860788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 msgr2=0x558ab6c34120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c34120 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x558ab6c27f20 tx=0x7f54a800b4d0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 -- 172.21.15.5:0/1097860788 shutdown_connections 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c34120 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c2ba00 0x558ab6c2adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 --2- 172.21.15.5:0/1097860788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c224d0 0x558ab6c37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 -- 172.21.15.5:0/1097860788 >> 172.21.15.5:0/1097860788 conn(0x558ab6c1cba0 msgr2=0x558ab6c1b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 -- 172.21.15.5:0/1097860788 shutdown_connections 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 -- 172.21.15.5:0/1097860788 wait complete. 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 -- start start 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.251+0000 7f54c23a8740 1 Processor -- start 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c224d0 0x558ab6c24930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c24e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54b81aef80 con 0x558ab6c22e80 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f54b81ae8b0 con 0x558ab6c2ba00 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54b81abb10 con 0x558ab6c224d0 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54c23a8740 10 client.0 _ll_register_callbacks cb 0x7f54b40df0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c24e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c24e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44848/0 (socket says 172.21.15.5:44848) 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 -- 172.21.15.5:0/4057248933 learned_addr learned my addr 172.21.15.5:0/4057248933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bd47b640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c224d0 0x558ab6c24930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 -- 172.21.15.5:0/4057248933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c224d0 msgr2=0x558ab6c24930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c224d0 0x558ab6c24930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 -- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 msgr2=0x558ab6c254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 -- 172.21.15.5:0/4057248933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab6c25ca0 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bdc7c640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54bcc7a640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c24e70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b840 tx=0x7f54a801b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5397c55953dd34b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.252+0000 7f54a0ff9640 1 -- 172.21.15.5:0/4057248933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a801ba00 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8014270 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54a0ff9640 1 -- 172.21.15.5:0/4057248933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f54a8005020 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54a0ff9640 1 -- 172.21.15.5:0/4057248933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54a8005220 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 10 client.18384 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ab6ca6770 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54a0ff9640 1 -- 172.21.15.5:0/4057248933 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a8014270 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54a0ff9640 1 -- 172.21.15.5:0/4057248933 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f54a800f040 con 0x558ab6c22e80 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 10 client.18384 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 10 client.18384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 20 client.18384 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6ca8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 4 client.18384 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ab6b94c10 con 0x558ab6c3bb40 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.253+0000 7f54bd47b640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6ca8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54bd47b640 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6ca8eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ace74a0e55d7d48 server_cookie=a9b069c0d31ece64 in_seq=0 out_seq=0 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54a0ff9640 10 client.18384 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54a0ff9640 1 -- 172.21.15.5:0/4057248933 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ab6b94c10 con 0x558ab6c3bb40 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54a0ff9640 10 client.18384 handle_command_reply: tid=0 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 client.18384 shutdown 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 msgr2=0x558ab6c24e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c24e70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b840 tx=0x7f54a801b330 comp rx=0 tx=0).stop 2024-04-22T13:34:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 shutdown_connections 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x558ab6c3bb40 0x558ab6ca8eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ab6c22e80 0x558ab6c24e70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558ab6c2ba00 0x558ab6c254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 --2- 172.21.15.5:0/4057248933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab6c224d0 0x558ab6c24930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54a0ff9640 0 client.18384 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 >> 172.21.15.5:0/4057248933 conn(0x558ab6c1cba0 msgr2=0x558ab6c1e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 shutdown_connections 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 1 -- 172.21.15.5:0/4057248933 wait complete. 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.254+0000 7f54c23a8740 20 client.18384 trim_cache size 0 max 16384 2024-04-22T13:34:54.256 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f54a40a3da0 msgr2=0x7f54a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f54a40a3da0 0x7f54a40a6260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f54b0020000 tx=0x7f54b001d000 comp rx=0 tx=0).stop 2024-04-22T13:34:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 msgr2=0x7f54b8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8199640 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009230 tx=0x7f54b40045e0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 shutdown_connections 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f54a40a3da0 0x7f54a40a6260 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b8103400 0x7f54b81a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54b8102a60 0x7f54b8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 --2- 172.21.15.5:0/4079289034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54b8101180 0x7f54b8199640 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 >> 172.21.15.5:0/4079289034 conn(0x7f54b8077f70 msgr2=0x7f54b8104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 shutdown_connections 2024-04-22T13:34:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.255+0000 7f54bf6df640 1 -- 172.21.15.5:0/4079289034 wait complete. 2024-04-22T13:34:54.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/537436379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f410c6d0 msgr2=0x7fb3f410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/537436379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f410c6d0 0x7fb3f410eb60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009ca0 tx=0x7fb3ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/537436379 shutdown_connections 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/537436379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f410c6d0 0x7fb3f410eb60 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/537436379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3f4103b70 0x7fb3f410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/537436379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f41031c0 0x7fb3f41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/537436379 >> 172.21.15.5:0/537436379 conn(0x7fb3f4077f70 msgr2=0x7fb3f4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/537436379 shutdown_connections 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.369+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/537436379 wait complete. 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 Processor -- start 2024-04-22T13:34:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 -- start start 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 0x7fb3f4100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f410c6d0 0x7fb3f4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3f4101890 0x7fb3f41a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4112380 con 0x7fb3f41031c0 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4112200 con 0x7fb3f4101890 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fc1f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4112500 con 0x7fb3f410c6d0 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 0x7fb3f4100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 0x7fb3f4100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44864/0 (socket says 172.21.15.5:44864) 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 -- 172.21.15.5:0/1578727932 learned_addr learned my addr 172.21.15.5:0/1578727932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3fa791640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3f4101890 0x7fb3f41a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f978f640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f410c6d0 0x7fb3f4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 -- 172.21.15.5:0/1578727932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f410c6d0 msgr2=0x7fb3f4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f410c6d0 0x7fb3f4101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 -- 172.21.15.5:0/1578727932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3f4101890 msgr2=0x7fb3f41a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3f4101890 0x7fb3f41a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f9f90640 1 -- 172.21.15.5:0/1578727932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f41a7190 con 0x7fb3f41031c0 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.370+0000 7fb3f978f640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f410c6d0 0x7fb3f4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.371+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 0x7fb3f4100e10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800d950 tx=0x7fb3e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.371+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1578727932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8017030 con 0x7fb3f41031c0 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.371+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f41a7480 con 0x7fb3f41031c0 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.371+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1578727932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e8004020 con 0x7fb3f41031c0 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.371+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1578727932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80047a0 con 0x7fb3f41031c0 2024-04-22T13:34:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.371+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f41a7a20 con 0x7fb3f41031c0 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.373+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1578727932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb3e8009a30 con 0x7fb3f41031c0 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.373+0000 7fb3fedeb740 1 Processor -- start 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.373+0000 7fb3fedeb740 1 -- start start 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f6261f0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f5c34b0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f6221b0 con 0x55eb0f6caa00 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44870/0 (socket says 172.21.15.5:44870) 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2274457806 learned_addr learned my addr 172.21.15.5:0/2274457806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fa791640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f978f640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3e6ffd640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3e00a3d50 0x7fb3e00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fa791640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3e00a3d50 0x7fb3e00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1578727932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb3e80d01f0 con 0x7fb3f41031c0 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2274457806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 msgr2=0x55eb0f6cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2274457806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 msgr2=0x55eb0f6c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2274457806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0f5b51b0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f978f640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3fa791640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.374+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6ca430 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80cd8b0 tx=0x7fb3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9330a511f08a4fba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3dffff640 1 -- 172.21.15.5:0/2274457806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e809baf0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fa791640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3e00a3d50 0x7fb3e00a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec004af0 tx=0x7fb3ec02c0f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3dffff640 1 -- 172.21.15.5:0/2274457806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80056a0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3dffff640 1 -- 172.21.15.5:0/2274457806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8009d20 con 0x55eb0f6c1e80 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2274457806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 msgr2=0x55eb0f6ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6ca430 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80cd8b0 tx=0x7fb3e80046c0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2274457806 shutdown_connections 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6ca430 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2274457806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2274457806 >> 172.21.15.5:0/2274457806 conn(0x55eb0f6bbba0 msgr2=0x55eb0f6bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2274457806 shutdown_connections 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.375+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2274457806 wait complete. 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 -- start start 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 Processor -- start 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f6cd8b0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f5df500 con 0x55eb0f6c14d0 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f6cda30 con 0x55eb0f6caa00 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fedeb740 10 client.0 _ll_register_callbacks cb 0x55eb0f5adb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3f978f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3f978f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44878/0 (socket says 172.21.15.5:44878) 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3f978f640 1 -- 172.21.15.5:0/1644081609 learned_addr learned my addr 172.21.15.5:0/1644081609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.376+0000 7fb3fa791640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f978f640 1 -- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 msgr2=0x55eb0f6c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f978f640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f978f640 1 -- 172.21.15.5:0/1644081609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 msgr2=0x55eb0f6ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f978f640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f978f640 1 -- 172.21.15.5:0/1644081609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0f3383f0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3f978f640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c76e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0011730 tx=0x7fb3f0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cc9c319512d4df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/1644081609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000fe70 con 0x55eb0f6c1e80 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e80056a0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/1644081609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb3f00193e0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/1644081609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f00195e0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3fedeb740 10 client.18396 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb0f4027d0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/1644081609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80056a0 con 0x55eb0f6c1e80 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.377+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/1644081609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3f0018070 con 0x55eb0f6c1e80 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3fedeb740 10 client.18396 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3fedeb740 10 client.18396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3fedeb740 20 client.18396 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f6dab90 0x55eb0f6dd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3fedeb740 4 client.18396 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb0f5c5930 con 0x55eb0f6dab90 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f6dab90 0x55eb0f6dd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f6dab90 0x55eb0f6dd010 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34bc9357bde958d5 server_cookie=d28f57c1375acae8 in_seq=0 out_seq=0 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.378+0000 7fb3dd7fa640 10 client.18396 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/1644081609 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb0f5c5930 con 0x55eb0f6dab90 2024-04-22T13:34:54.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3dd7fa640 10 client.18396 handle_command_reply: tid=0 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 client.18396 shutdown 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 msgr2=0x55eb0f6c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c76e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0011730 tx=0x7fb3f0011bf0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 shutdown_connections 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f6dab90 0x55eb0f6dd010 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c76e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3dd7fa640 0 client.18396 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/1644081609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 >> 172.21.15.5:0/1644081609 conn(0x55eb0f6bbba0 msgr2=0x55eb0f6bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 shutdown_connections 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 1 -- 172.21.15.5:0/1644081609 wait complete. 2024-04-22T13:34:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.379+0000 7fb3fedeb740 20 client.18396 trim_cache size 0 max 16384 2024-04-22T13:34:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 Processor -- start 2024-04-22T13:34:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 -- start start 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f41afe40 con 0x55eb0f6c14d0 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f41b1980 con 0x55eb0f6caa00 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f41b0510 con 0x55eb0f6c1e80 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6d6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44886/0 (socket says 172.21.15.5:44886) 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 -- 172.21.15.5:0/751064601 learned_addr learned my addr 172.21.15.5:0/751064601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f978f640 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fa791640 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 -- 172.21.15.5:0/751064601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 msgr2=0x55eb0f6c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 -- 172.21.15.5:0/751064601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 msgr2=0x55eb0f6ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 -- 172.21.15.5:0/751064601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0f6d97e0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3fa791640 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6d6770 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80985f0 tx=0x7fb3e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46efabd9263b97a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.393+0000 7fb3df7fe640 1 -- 172.21.15.5:0/751064601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b4020 con 0x55eb0f6c14d0 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3df7fe640 1 -- 172.21.15.5:0/751064601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80056a0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3df7fe640 1 -- 172.21.15.5:0/751064601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80041d0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 -- 172.21.15.5:0/751064601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 msgr2=0x55eb0f6d6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6d6770 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80985f0 tx=0x7fb3e8004730 comp rx=0 tx=0).stop 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 -- 172.21.15.5:0/751064601 shutdown_connections 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/751064601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6d6770 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 -- 172.21.15.5:0/751064601 >> 172.21.15.5:0/751064601 conn(0x55eb0f6bbba0 msgr2=0x55eb0f6bac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 -- 172.21.15.5:0/751064601 shutdown_connections 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 -- 172.21.15.5:0/751064601 wait complete. 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 -- start start 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.394+0000 7fb3fedeb740 1 Processor -- start 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f41b12b0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f41b0be0 con 0x55eb0f6caa00 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f41ade40 con 0x55eb0f6c1e80 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fedeb740 10 client.0 _ll_register_callbacks cb 0x7fb3f0008f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44898/0 (socket says 172.21.15.5:44898) 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2254562479 learned_addr learned my addr 172.21.15.5:0/2254562479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f978f640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fa791640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 msgr2=0x55eb0f6c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2254562479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 msgr2=0x55eb0f6c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 -- 172.21.15.5:0/2254562479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0f7401a0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3fa791640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f978f640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3f9f90640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c43f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8004700 tx=0x7fb3e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a6340e46afe274 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.395+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/2254562479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b4040 con 0x55eb0f6c14d0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/2254562479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fb3e8009d20 con 0x55eb0f6c14d0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e80056a0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/2254562479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3e809bf60 con 0x55eb0f6c14d0 2024-04-22T13:34:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 10 client.18402 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb0f7413f0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/2254562479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80056a0 con 0x55eb0f6c14d0 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/2254562479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3e8098800 con 0x55eb0f6c14d0 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 10 client.18402 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 10 client.18402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 20 client.18402 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f7413f0 0x55eb0f7437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 4 client.18402 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb0f633c10 con 0x55eb0f7413f0 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.396+0000 7fb3f978f640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f7413f0 0x55eb0f7437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3f978f640 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f7413f0 0x55eb0f7437e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53cf00cea74143c1 server_cookie=5879ec5af73244a8 in_seq=0 out_seq=0 2024-04-22T13:34:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3dd7fa640 10 client.18402 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3dd7fa640 1 -- 172.21.15.5:0/2254562479 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb0f633c10 con 0x55eb0f7413f0 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3dd7fa640 10 client.18402 handle_command_reply: tid=0 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 client.18402 shutdown 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 msgr2=0x55eb0f6c43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c43f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8004700 tx=0x7fb3e8009780 comp rx=0 tx=0).stop 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 shutdown_connections 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55eb0f7413f0 0x55eb0f7437e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eb0f6caa00 0x55eb0f6c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb0f6c1e80 0x55eb0f6c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3dd7fa640 0 client.18402 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 --2- 172.21.15.5:0/2254562479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb0f6c14d0 0x55eb0f6c43f0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 >> 172.21.15.5:0/2254562479 conn(0x55eb0f6bbba0 msgr2=0x55eb0f6bcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 shutdown_connections 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 1 -- 172.21.15.5:0/2254562479 wait complete. 2024-04-22T13:34:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.397+0000 7fb3fedeb740 20 client.18402 trim_cache size 0 max 16384 2024-04-22T13:34:54.400 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3e00a3d50 msgr2=0x7fb3e00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3e00a3d50 0x7fb3e00a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec004af0 tx=0x7fb3ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 msgr2=0x7fb3f4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 0x7fb3f4100e10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800d950 tx=0x7fb3e800de10 comp rx=0 tx=0).stop 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 shutdown_connections 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb3e00a3d50 0x7fb3e00a6210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3f4101890 0x7fb3f41a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f410c6d0 0x7fb3f4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 --2- 172.21.15.5:0/1578727932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f41031c0 0x7fb3f4100e10 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 >> 172.21.15.5:0/1578727932 conn(0x7fb3f4077f70 msgr2=0x7fb3f40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 shutdown_connections 2024-04-22T13:34:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.398+0000 7fb3fc1f4640 1 -- 172.21.15.5:0/1578727932 wait complete. 2024-04-22T13:34:54.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:34:54.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 -- 172.21.15.5:0/264096250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8107250 msgr2=0x7f36e810ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 --2- 172.21.15.5:0/264096250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8107250 0x7f36e810ee20 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009ca0 tx=0x7f36e002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 -- 172.21.15.5:0/264096250 shutdown_connections 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 --2- 172.21.15.5:0/264096250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8107250 0x7f36e810ee20 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 --2- 172.21.15.5:0/264096250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8102b30 0x7f36e8106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 --2- 172.21.15.5:0/264096250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e8102180 0x7f36e8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.512+0000 7f36ef05c640 1 -- 172.21.15.5:0/264096250 >> 172.21.15.5:0/264096250 conn(0x7f36e80fda00 msgr2=0x7f36e80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 -- 172.21.15.5:0/264096250 shutdown_connections 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 -- 172.21.15.5:0/264096250 wait complete. 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 Processor -- start 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 -- start start 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 0x7f36e8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e8107250 0x7f36e8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e819cf70 0x7f36e819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81121c0 con 0x7f36e8102180 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36e8112040 con 0x7f36e819cf70 2024-04-22T13:34:54.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ef05c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36e8112340 con 0x7f36e8107250 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ecdf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 0x7f36e8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ecdf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 0x7f36e8198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44916/0 (socket says 172.21.15.5:44916) 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.513+0000 7f36ecdf8640 1 -- 172.21.15.5:0/1107450506 learned_addr learned my addr 172.21.15.5:0/1107450506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e819cf70 0x7f36e819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36e7fff640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e8107250 0x7f36e8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ecdf8640 1 -- 172.21.15.5:0/1107450506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e8107250 msgr2=0x7f36e8198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e8107250 0x7f36e8198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ecdf8640 1 -- 172.21.15.5:0/1107450506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e819cf70 msgr2=0x7f36e819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e819cf70 0x7f36e819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ecdf8640 1 -- 172.21.15.5:0/1107450506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e819da60 con 0x7f36e8102180 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 0x7f36e8198700 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f36d800d770 tx=0x7f36d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36e5ffb640 1 -- 172.21.15.5:0/1107450506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8017070 con 0x7f36e8102180 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36e5ffb640 1 -- 172.21.15.5:0/1107450506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80054d0 con 0x7f36e8102180 2024-04-22T13:34:54.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e819dd50 con 0x7f36e8102180 2024-04-22T13:34:54.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36e5ffb640 1 -- 172.21.15.5:0/1107450506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004c30 con 0x7f36e8102180 2024-04-22T13:34:54.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.514+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e81a54f0 con 0x7f36e8102180 2024-04-22T13:34:54.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.516+0000 7f36e5ffb640 1 -- 172.21.15.5:0/1107450506 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f36d8004e10 con 0x7f36e8102180 2024-04-22T13:34:54.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 Processor -- start 2024-04-22T13:34:54.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 -- start start 2024-04-22T13:34:54.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d7f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee36ce41f0 con 0x55ee36d88a00 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee36c814b0 con 0x55ee36d7fe80 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ee36ce01b0 con 0x55ee36d7f4d0 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36e5ffb640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f36d00a3d20 0x7f36d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44928/0 (socket says 172.21.15.5:44928) 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.517+0000 7f36e7fff640 1 -- 172.21.15.5:0/1563725336 learned_addr learned my addr 172.21.15.5:0/1563725336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f36d00a3d20 0x7f36d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d7f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e5ffb640 1 -- 172.21.15.5:0/1107450506 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f36d80d0330 con 0x7f36e8102180 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e7fff640 1 -- 172.21.15.5:0/1563725336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 msgr2=0x55ee36d7f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e7fff640 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d7f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e7fff640 1 -- 172.21.15.5:0/1563725336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 msgr2=0x55ee36d88430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e7fff640 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d88430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e7fff640 1 -- 172.21.15.5:0/1563725336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee36c731b0 con 0x55ee36d88a00 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d88430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36e7fff640 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8ae50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00cb90 tx=0x7f36dc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d998cab419db06 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f36d00a3d20 0x7f36d00a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005e30 tx=0x7f36e002e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ceffd640 1 -- 172.21.15.5:0/1563725336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc005220 con 0x55ee36d88a00 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ceffd640 1 -- 172.21.15.5:0/1563725336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc004c50 con 0x55ee36d88a00 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.518+0000 7f36ceffd640 1 -- 172.21.15.5:0/1563725336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0107d0 con 0x55ee36d88a00 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 -- 172.21.15.5:0/1563725336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 msgr2=0x55ee36d8ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8ae50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00cb90 tx=0x7f36dc00e720 comp rx=0 tx=0).stop 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 -- 172.21.15.5:0/1563725336 shutdown_connections 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8ae50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d88430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/1563725336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d7f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 -- 172.21.15.5:0/1563725336 >> 172.21.15.5:0/1563725336 conn(0x55ee36d79ba0 msgr2=0x55ee36d79f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 -- 172.21.15.5:0/1563725336 shutdown_connections 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 -- 172.21.15.5:0/1563725336 wait complete. 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 -- start start 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.519+0000 7f36f1c4e740 1 Processor -- start 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee36d8b970 con 0x55ee36d88a00 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ee36c9d500 con 0x55ee36d7fe80 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ee36d8baf0 con 0x55ee36d7f4d0 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36f1c4e740 10 client.0 _ll_register_callbacks cb 0x55ee36c6bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d85bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44940/0 (socket says 172.21.15.5:44940) 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 -- 172.21.15.5:0/141735263 learned_addr learned my addr 172.21.15.5:0/141735263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36e7fff640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 -- 172.21.15.5:0/141735263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 msgr2=0x55ee36d882d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d882d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 -- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 msgr2=0x55ee36d85660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d85660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 -- 172.21.15.5:0/141735263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee369f63f0 con 0x55ee36d88a00 2024-04-22T13:34:54.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d85bc0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f36e0036000 tx=0x7f36e002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7cac4252323fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36c7fff640 1 -- 172.21.15.5:0/141735263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0053060 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.520+0000 7f36c7fff640 1 -- 172.21.15.5:0/141735263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f36e0003d00 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc004c50 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36c7fff640 1 -- 172.21.15.5:0/141735263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e0039040 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 10 client.18414 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee36ac07d0 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36c7fff640 1 -- 172.21.15.5:0/141735263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc004c50 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36c7fff640 1 -- 172.21.15.5:0/141735263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f36e0043720 con 0x55ee36d88a00 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 10 client.18414 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 10 client.18414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 20 client.18414 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36d98c20 0x55ee36d9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 4 client.18414 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee36c83930 con 0x55ee36d98c20 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.521+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36d98c20 0x55ee36d9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.522+0000 7f36c7fff640 10 client.18414 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.522+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36d98c20 0x55ee36d9b050 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59768f16077f00b9 server_cookie=34a0c51efb58913e in_seq=0 out_seq=0 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.522+0000 7f36c7fff640 1 -- 172.21.15.5:0/141735263 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee36c83930 con 0x55ee36d98c20 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.522+0000 7f36c7fff640 10 client.18414 handle_command_reply: tid=0 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.522+0000 7f36f1c4e740 1 client.18414 shutdown 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 msgr2=0x55ee36d85bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d85bc0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f36e0036000 tx=0x7f36e002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 shutdown_connections 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36d98c20 0x55ee36d9b050 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d85bc0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d85660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/141735263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 >> 172.21.15.5:0/141735263 conn(0x55ee36d79ba0 msgr2=0x55ee36d7b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36c7fff640 0 client.18414 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 shutdown_connections 2024-04-22T13:34:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 1 -- 172.21.15.5:0/141735263 wait complete. 2024-04-22T13:34:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.523+0000 7f36f1c4e740 20 client.18414 trim_cache size 0 max 16384 2024-04-22T13:34:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 Processor -- start 2024-04-22T13:34:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 -- start start 2024-04-22T13:34:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 0x55ee36d94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7fe80 0x55ee36d7b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d88a00 0x55ee36d87d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81ada50 con 0x55ee36d7f4d0 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81af590 con 0x55ee36d88a00 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81ae120 con 0x55ee36d7fe80 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36ecdf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 0x55ee36d94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36ecdf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 0x55ee36d94830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44956/0 (socket says 172.21.15.5:44956) 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.536+0000 7f36ecdf8640 1 -- 172.21.15.5:0/3337838413 learned_addr learned my addr 172.21.15.5:0/3337838413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d88a00 0x55ee36d87d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36e7fff640 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7fe80 0x55ee36d7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ecdf8640 1 -- 172.21.15.5:0/3337838413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7fe80 msgr2=0x55ee36d7b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7fe80 0x55ee36d7b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ecdf8640 1 -- 172.21.15.5:0/3337838413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d88a00 msgr2=0x55ee36d87d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d88a00 0x55ee36d87d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ecdf8640 1 -- 172.21.15.5:0/3337838413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee36d83bb0 con 0x55ee36d7f4d0 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 0x55ee36d94830 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f36d80cda80 tx=0x7f36d8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdd8f06ae3ebaeb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ce7fc640 1 -- 172.21.15.5:0/3337838413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80cf040 con 0x55ee36d7f4d0 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ce7fc640 1 -- 172.21.15.5:0/3337838413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80b5070 con 0x55ee36d7f4d0 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.537+0000 7f36ce7fc640 1 -- 172.21.15.5:0/3337838413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8020040 con 0x55ee36d7f4d0 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- 172.21.15.5:0/3337838413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 msgr2=0x55ee36d94830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 0x55ee36d94830 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f36d80cda80 tx=0x7f36d8009f90 comp rx=0 tx=0).stop 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- 172.21.15.5:0/3337838413 shutdown_connections 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d88a00 0x55ee36d87d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7fe80 0x55ee36d7b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/3337838413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d7f4d0 0x55ee36d94830 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- 172.21.15.5:0/3337838413 >> 172.21.15.5:0/3337838413 conn(0x55ee36d79ba0 msgr2=0x55ee36d7aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- 172.21.15.5:0/3337838413 shutdown_connections 2024-04-22T13:34:54.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- 172.21.15.5:0/3337838413 wait complete. 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- start start 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 Processor -- start 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d8c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81aeec0 con 0x55ee36d88a00 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81ae7f0 con 0x55ee36d7fe80 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.538+0000 7f36f1c4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81aba50 con 0x55ee36d7f4d0 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36f1c4e740 10 client.0 _ll_register_callbacks cb 0x7f36e00324c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44968/0 (socket says 172.21.15.5:44968) 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 -- 172.21.15.5:0/907439663 learned_addr learned my addr 172.21.15.5:0/907439663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36e7fff640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d8c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 -- 172.21.15.5:0/907439663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 msgr2=0x55ee36d910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 -- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 msgr2=0x55ee36d8c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d8c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 -- 172.21.15.5:0/907439663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee36d8d170 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36ed5f9640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8c970 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x55ee36d7a9e0 tx=0x7f36e0034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d48f487ec39ffa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36c7fff640 1 -- 172.21.15.5:0/907439663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0053040 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36c7fff640 1 -- 172.21.15.5:0/907439663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f36e0034a70 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d80b5070 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36c7fff640 1 -- 172.21.15.5:0/907439663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e002c770 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36f1c4e740 10 client.18420 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.539+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee36e07970 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36c7fff640 1 -- 172.21.15.5:0/907439663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80b5070 con 0x55ee36d88a00 2024-04-22T13:34:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36c7fff640 1 -- 172.21.15.5:0/907439663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f36e004f930 con 0x55ee36d88a00 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36f1c4e740 10 client.18420 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36f1c4e740 10 client.18420 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36f1c4e740 20 client.18420 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36e08060 0x55ee36e0a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36f1c4e740 4 client.18420 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee36cf1c10 con 0x55ee36e08060 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36e08060 0x55ee36e0a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36c7fff640 10 client.18420 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.540+0000 7f36ecdf8640 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36e08060 0x55ee36e0a480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6784e7786edf44c3 server_cookie=325cb0e1fd4f7e49 in_seq=0 out_seq=0 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36c7fff640 1 -- 172.21.15.5:0/907439663 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee36cf1c10 con 0x55ee36e08060 2024-04-22T13:34:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36c7fff640 10 client.18420 handle_command_reply: tid=0 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 client.18420 shutdown 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 msgr2=0x55ee36d8c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8c970 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x55ee36d7a9e0 tx=0x7f36e0034670 comp rx=0 tx=0).stop 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 shutdown_connections 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55ee36e08060 0x55ee36e0a480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee36d88a00 0x55ee36d8c970 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36c7fff640 0 client.18420 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee36d7fe80 0x55ee36d8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 --2- 172.21.15.5:0/907439663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee36d7f4d0 0x55ee36d910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 >> 172.21.15.5:0/907439663 conn(0x55ee36d79ba0 msgr2=0x55ee36d7b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 shutdown_connections 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 1 -- 172.21.15.5:0/907439663 wait complete. 2024-04-22T13:34:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.541+0000 7f36f1c4e740 20 client.18420 trim_cache size 0 max 16384 2024-04-22T13:34:54.544 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f36d00a3d20 msgr2=0x7f36d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f36d00a3d20 0x7f36d00a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005e30 tx=0x7f36e002e040 comp rx=0 tx=0).stop 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 msgr2=0x7f36e8198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 0x7f36e8198700 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f36d800d770 tx=0x7f36d800dc30 comp rx=0 tx=0).stop 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 shutdown_connections 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f36d00a3d20 0x7f36d00a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e819cf70 0x7f36e819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e8107250 0x7f36e8198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 --2- 172.21.15.5:0/1107450506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e8102180 0x7f36e8198700 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 >> 172.21.15.5:0/1107450506 conn(0x7f36e80fda00 msgr2=0x7f36e80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 shutdown_connections 2024-04-22T13:34:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.542+0000 7f36ef05c640 1 -- 172.21.15.5:0/1107450506 wait complete. 2024-04-22T13:34:54.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:34:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 -- 172.21.15.5:0/871287920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070102a60 msgr2=0x7fc070102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 --2- 172.21.15.5:0/871287920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070102a60 0x7fc070102ec0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc064009930 tx=0x7fc06402bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 -- 172.21.15.5:0/871287920 shutdown_connections 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 --2- 172.21.15.5:0/871287920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070102a60 0x7fc070102ec0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 --2- 172.21.15.5:0/871287920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc070103400 0x7fc07010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 --2- 172.21.15.5:0/871287920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc070101180 0x7fc070101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.656+0000 7fc077585640 1 -- 172.21.15.5:0/871287920 >> 172.21.15.5:0/871287920 conn(0x7fc070077f70 msgr2=0x7fc070078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 -- 172.21.15.5:0/871287920 shutdown_connections 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 -- 172.21.15.5:0/871287920 wait complete. 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 Processor -- start 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 -- start start 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc070101180 0x7fc07019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc070102a60 0x7fc07019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 0x7fc070198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701111a0 con 0x7fc070103400 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc070111020 con 0x7fc070102a60 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc077585640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc070111320 con 0x7fc070101180 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc075b22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 0x7fc070198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.657+0000 7fc075b22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 0x7fc070198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44982/0 (socket says 172.21.15.5:44982) 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 -- 172.21.15.5:0/2110135481 learned_addr learned my addr 172.21.15.5:0/2110135481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075321640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc070101180 0x7fc07019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc074b20640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc070102a60 0x7fc07019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 -- 172.21.15.5:0/2110135481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc070101180 msgr2=0x7fc07019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc070101180 0x7fc07019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 -- 172.21.15.5:0/2110135481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc070102a60 msgr2=0x7fc07019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc070102a60 0x7fc07019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 -- 172.21.15.5:0/2110135481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc070198f50 con 0x7fc070103400 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc075b22640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 0x7fc070198690 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc06000e970 tx=0x7fc06000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc06e7fc640 1 -- 172.21.15.5:0/2110135481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000ccc0 con 0x7fc070103400 2024-04-22T13:34:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc06e7fc640 1 -- 172.21.15.5:0/2110135481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06000ce60 con 0x7fc070103400 2024-04-22T13:34:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc070199240 con 0x7fc070103400 2024-04-22T13:34:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc06e7fc640 1 -- 172.21.15.5:0/2110135481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060010430 con 0x7fc070103400 2024-04-22T13:34:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.658+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0701a5550 con 0x7fc070103400 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc06e7fc640 1 -- 172.21.15.5:0/2110135481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc060004ff0 con 0x7fc070103400 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 Processor -- start 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 -- start start 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ff9b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9bd60 0x55e26ffa4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffa52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e26ff001f0 con 0x55e26ffa4e60 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e26fe9d4b0 con 0x55e26ff9b3b0 2024-04-22T13:34:54.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc07a171740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e26fefc1b0 con 0x55e26ff9bd60 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc074b20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffa52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc074b20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffa52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44992/0 (socket says 172.21.15.5:44992) 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.661+0000 7fc074b20640 1 -- 172.21.15.5:0/3796446709 learned_addr learned my addr 172.21.15.5:0/3796446709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc075321640 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9bd60 0x55e26ffa4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ff9b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc06e7fc640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0580a3d20 0x7fc0580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 -- 172.21.15.5:0/3796446709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9bd60 msgr2=0x55e26ffa4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9bd60 0x55e26ffa4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 -- 172.21.15.5:0/3796446709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 msgr2=0x55e26ff9b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ff9b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 -- 172.21.15.5:0/3796446709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26fe8f1b0 con 0x55e26ffa4e60 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc075b22640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0580a3d20 0x7fc0580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc06e7fc640 1 -- 172.21.15.5:0/2110135481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fc0600d0ce0 con 0x7fc070103400 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc074b20640 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffa52c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc064036040 tx=0x7fc0640058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7676fb77a11498cd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc0577fe640 1 -- 172.21.15.5:0/3796446709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004480 con 0x55e26ffa4e60 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc0577fe640 1 -- 172.21.15.5:0/3796446709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc064048070 con 0x55e26ffa4e60 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.662+0000 7fc0577fe640 1 -- 172.21.15.5:0/3796446709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004920 con 0x55e26ffa4e60 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 -- 172.21.15.5:0/3796446709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 msgr2=0x55e26ffa52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc075b22640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0580a3d20 0x7fc0580a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0600ce4e0 tx=0x7fc0600bd040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffa52c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc064036040 tx=0x7fc0640058d0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 -- 172.21.15.5:0/3796446709 shutdown_connections 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffa52c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9bd60 0x55e26ffa4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 --2- 172.21.15.5:0/3796446709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ff9b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 -- 172.21.15.5:0/3796446709 >> 172.21.15.5:0/3796446709 conn(0x55e26ff95ba0 msgr2=0x55e26ff95f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 -- 172.21.15.5:0/3796446709 shutdown_connections 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 -- 172.21.15.5:0/3796446709 wait complete. 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 -- start start 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.663+0000 7fc07a171740 1 Processor -- start 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffa8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ffa4e60 0x55e26ffa8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e26ffabd40 con 0x55e26ff9bd60 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55e26feb9500 con 0x55e26ffa4e60 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e26ffabec0 con 0x55e26ff9b3b0 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc07a171740 10 client.0 _ll_register_callbacks cb 0x55e26fe87b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffa8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffa8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44994/0 (socket says 172.21.15.5:44994) 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 -- 172.21.15.5:0/3185582807 learned_addr learned my addr 172.21.15.5:0/3185582807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc074b20640 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 -- 172.21.15.5:0/3185582807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 msgr2=0x55e26ffab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 -- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ffa4e60 msgr2=0x55e26ffa8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ffa4e60 0x55e26ffa8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 -- 172.21.15.5:0/3185582807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26fc123f0 con 0x55e26ff9bd60 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc075321640 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffa8610 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc068012cc0 tx=0x7fc06800c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85905c357b17f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.664+0000 7fc054ff9640 1 -- 172.21.15.5:0/3185582807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06801b440 con 0x55e26ff9bd60 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc054ff9640 1 -- 172.21.15.5:0/3185582807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fc06801a3e0 con 0x55e26ff9bd60 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc064048070 con 0x55e26ff9bd60 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc054ff9640 1 -- 172.21.15.5:0/3185582807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc06801b5e0 con 0x55e26ff9bd60 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 10 client.18432 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e26fcdc7d0 con 0x55e26ff9bd60 2024-04-22T13:34:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc054ff9640 1 -- 172.21.15.5:0/3185582807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc064048070 con 0x55e26ff9bd60 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc054ff9640 1 -- 172.21.15.5:0/3185582807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc068019070 con 0x55e26ff9bd60 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 10 client.18432 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 10 client.18432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 20 client.18432 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e26ffb90a0 0x55e26ffbb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 4 client.18432 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26fe9f930 con 0x55e26ffb90a0 2024-04-22T13:34:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.665+0000 7fc074b20640 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e26ffb90a0 0x55e26ffbb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.666+0000 7fc074b20640 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e26ffb90a0 0x55e26ffbb4b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98a5d14efa37294 server_cookie=102222c7a1aa765e in_seq=0 out_seq=0 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.666+0000 7fc054ff9640 10 client.18432 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.666+0000 7fc054ff9640 1 -- 172.21.15.5:0/3185582807 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e26fe9f930 con 0x55e26ffb90a0 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.666+0000 7fc054ff9640 10 client.18432 handle_command_reply: tid=0 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.666+0000 7fc07a171740 1 client.18432 shutdown 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 msgr2=0x55e26ffa8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffa8610 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc068012cc0 tx=0x7fc06800c700 comp rx=0 tx=0).stop 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 shutdown_connections 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e26ffb90a0 0x55e26ffbb4b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffa8610 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ffa4e60 0x55e26ffa8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 --2- 172.21.15.5:0/3185582807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 >> 172.21.15.5:0/3185582807 conn(0x55e26ff95ba0 msgr2=0x55e26ff978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc054ff9640 0 client.18432 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 shutdown_connections 2024-04-22T13:34:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 1 -- 172.21.15.5:0/3185582807 wait complete. 2024-04-22T13:34:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.667+0000 7fc07a171740 20 client.18432 trim_cache size 0 max 16384 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 Processor -- start 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 -- start start 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffa6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9bd60 0x55e26ffb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffaad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ab980 con 0x55e26ffa4e60 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ad4c0 con 0x55e26ff9bd60 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc07a171740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ac050 con 0x55e26ff9b3b0 2024-04-22T13:34:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffaad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffaad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45006/0 (socket says 172.21.15.5:45006) 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 -- 172.21.15.5:0/374418331 learned_addr learned my addr 172.21.15.5:0/374418331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc075321640 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9bd60 0x55e26ffb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffa6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 -- 172.21.15.5:0/374418331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 msgr2=0x55e26ffa6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffa6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 -- 172.21.15.5:0/374418331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9bd60 msgr2=0x55e26ffb4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9bd60 0x55e26ffb4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.681+0000 7fc074b20640 1 -- 172.21.15.5:0/374418331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26ffa75e0 con 0x55e26ffa4e60 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc074b20640 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffaad40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc064006010 tx=0x7fc064005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33e32c7a3f98937 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc056ffd640 1 -- 172.21.15.5:0/374418331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06402c7e0 con 0x55e26ffa4e60 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc056ffd640 1 -- 172.21.15.5:0/374418331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc064048070 con 0x55e26ffa4e60 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc056ffd640 1 -- 172.21.15.5:0/374418331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064003bd0 con 0x55e26ffa4e60 2024-04-22T13:34:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 -- 172.21.15.5:0/374418331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 msgr2=0x55e26ffaad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffaad40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc064006010 tx=0x7fc064005410 comp rx=0 tx=0).stop 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 -- 172.21.15.5:0/374418331 shutdown_connections 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ffa4e60 0x55e26ffaad40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9bd60 0x55e26ffb4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 --2- 172.21.15.5:0/374418331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ff9b3b0 0x55e26ffa6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 -- 172.21.15.5:0/374418331 >> 172.21.15.5:0/374418331 conn(0x55e26ff95ba0 msgr2=0x55e26ff94c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 -- 172.21.15.5:0/374418331 shutdown_connections 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.682+0000 7fc07a171740 1 -- 172.21.15.5:0/374418331 wait complete. 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 -- start start 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 Processor -- start 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ffaa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffaab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ffa4e60 0x55e26ffab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701acdf0 con 0x55e26ff9bd60 2024-04-22T13:34:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ac720 con 0x55e26ff9b3b0 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701a99a0 con 0x55e26ffa4e60 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc07a171740 10 client.0 _ll_register_callbacks cb 0x7fc0680128d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc075321640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffaab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc075321640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffaab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45010/0 (socket says 172.21.15.5:45010) 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc075321640 1 -- 172.21.15.5:0/1783557540 learned_addr learned my addr 172.21.15.5:0/1783557540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc074b20640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ffa4e60 0x55e26ffab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.683+0000 7fc074b20640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ffaa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc075321640 1 -- 172.21.15.5:0/1783557540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ffa4e60 msgr2=0x55e26ffab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc075321640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ffa4e60 0x55e26ffab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc075321640 1 -- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 msgr2=0x55e26ffaa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc075321640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ffaa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc075321640 1 -- 172.21.15.5:0/1783557540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27001e7f0 con 0x55e26ff9bd60 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc075321640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffaab50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc06801ada0 tx=0x7fc068009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c07441f322d81b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc054ff9640 1 -- 172.21.15.5:0/1783557540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068004be0 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc054ff9640 1 -- 172.21.15.5:0/1783557540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fc068004d80 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc064048070 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc054ff9640 1 -- 172.21.15.5:0/1783557540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc06801b780 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc07a171740 10 client.18438 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e27001fae0 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.684+0000 7fc054ff9640 1 -- 172.21.15.5:0/1783557540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc064048070 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc054ff9640 1 -- 172.21.15.5:0/1783557540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc068019070 con 0x55e26ff9bd60 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 10 client.18438 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 10 client.18438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 20 client.18438 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e2700202b0 0x55e2700226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 4 client.18438 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e26ff0dc10 con 0x55e2700202b0 2024-04-22T13:34:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc074b20640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e2700202b0 0x55e2700226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc054ff9640 10 client.18438 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc074b20640 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e2700202b0 0x55e2700226d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62877c244fb1fbb7 server_cookie=5769c34c6384bee in_seq=0 out_seq=0 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc054ff9640 1 -- 172.21.15.5:0/1783557540 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e26ff0dc10 con 0x55e2700202b0 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc054ff9640 10 client.18438 handle_command_reply: tid=0 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 1 client.18438 shutdown 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 msgr2=0x55e26ffaab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.685+0000 7fc07a171740 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffaab50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc06801ada0 tx=0x7fc068009510 comp rx=0 tx=0).stop 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 shutdown_connections 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55e2700202b0 0x55e2700226d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e26ff9bd60 0x55e26ffaab50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc054ff9640 0 client.18438 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26ffa4e60 0x55e26ffab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 --2- 172.21.15.5:0/1783557540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e26ff9b3b0 0x55e26ffaa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 >> 172.21.15.5:0/1783557540 conn(0x55e26ff95ba0 msgr2=0x55e26ffa7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 shutdown_connections 2024-04-22T13:34:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 1 -- 172.21.15.5:0/1783557540 wait complete. 2024-04-22T13:34:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc07a171740 20 client.18438 trim_cache size 0 max 16384 2024-04-22T13:34:54.688 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0580a3d20 msgr2=0x7fc0580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0580a3d20 0x7fc0580a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0600ce4e0 tx=0x7fc0600bd040 comp rx=0 tx=0).stop 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 msgr2=0x7fc070198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 0x7fc070198690 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc06000e970 tx=0x7fc06000ed70 comp rx=0 tx=0).stop 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 shutdown_connections 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc070103400 0x7fc070198690 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fc0580a3d20 0x7fc0580a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc070102a60 0x7fc07019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 --2- 172.21.15.5:0/2110135481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc070101180 0x7fc07019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 >> 172.21.15.5:0/2110135481 conn(0x7fc070077f70 msgr2=0x7fc070100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 shutdown_connections 2024-04-22T13:34:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:54.686+0000 7fc077585640 1 -- 172.21.15.5:0/2110135481 wait complete. 2024-04-22T13:34:54.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:54 smithi005 ceph-mon[28006]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:54.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:54 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2071040563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:54 smithi049 ceph-mon[31379]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:54 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2071040563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:55.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:54 smithi107 ceph-mon[31859]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:55.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:54 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2071040563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:55.694 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:34:55.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:34:55.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 -- 172.21.15.5:0/3182231771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d41010a0 msgr2=0x7f83d4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 --2- 172.21.15.5:0/3182231771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d41010a0 0x7f83d4101480 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009900 tx=0x7f83d002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 -- 172.21.15.5:0/3182231771 shutdown_connections 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 --2- 172.21.15.5:0/3182231771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d4106170 0x7f83d410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 --2- 172.21.15.5:0/3182231771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4101a50 0x7f83d4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 --2- 172.21.15.5:0/3182231771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d41010a0 0x7f83d4101480 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 -- 172.21.15.5:0/3182231771 >> 172.21.15.5:0/3182231771 conn(0x7f83d4077f40 msgr2=0x7f83d4078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 -- 172.21.15.5:0/3182231771 shutdown_connections 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.818+0000 7f83dc715640 1 -- 172.21.15.5:0/3182231771 wait complete. 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 Processor -- start 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 -- start start 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 0x7f83d41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4106170 0x7f83d41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d419a770 0x7f83d419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41121f0 con 0x7f83d419a770 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4112070 con 0x7f83d4106170 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dc715640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4112370 con 0x7f83d41010a0 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83dacb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d419a770 0x7f83d419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83da4b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 0x7f83d41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83da4b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 0x7f83d41a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:46276/0 (socket says 172.21.15.5:46276) 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83da4b1640 1 -- 172.21.15.5:0/770753846 learned_addr learned my addr 172.21.15.5:0/770753846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.819+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4106170 0x7f83d41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dacb2640 1 -- 172.21.15.5:0/770753846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 msgr2=0x7f83d41a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dacb2640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 0x7f83d41a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dacb2640 1 -- 172.21.15.5:0/770753846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4106170 msgr2=0x7f83d41a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dacb2640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4106170 0x7f83d41a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dacb2640 1 -- 172.21.15.5:0/770753846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d419b370 con 0x7f83d419a770 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83da4b1640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 0x7f83d41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4106170 0x7f83d41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dacb2640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d419a770 0x7f83d419abf0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00cab0 tx=0x7f83cc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83cb7fe640 1 -- 172.21.15.5:0/770753846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00ae20 con 0x7f83d419a770 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83cb7fe640 1 -- 172.21.15.5:0/770753846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83cc00f900 con 0x7f83d419a770 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d41a7450 con 0x7f83d419a770 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83cb7fe640 1 -- 172.21.15.5:0/770753846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0189f0 con 0x7f83d419a770 2024-04-22T13:34:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.820+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d41a7800 con 0x7f83d419a770 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83cb7fe640 1 -- 172.21.15.5:0/770753846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f83cc00f450 con 0x7f83d419a770 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 Processor -- start 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 -- start start 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a083135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a08313ee10 0x55a083141260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a083099f00 con 0x55a083136290 2024-04-22T13:34:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a083037590 con 0x55a0831358e0 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83df3e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a083095ec0 con 0x55a08313ee10 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83da4b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83da4b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45048/0 (socket says 172.21.15.5:45048) 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83da4b1640 1 -- 172.21.15.5:0/2527289242 learned_addr learned my addr 172.21.15.5:0/2527289242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.823+0000 7f83cb7fe640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f83c00a3d20 0x7f83c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a083135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a08313ee10 0x55a083141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83dacb2640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f83c00a3d20 0x7f83c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83da4b1640 1 -- 172.21.15.5:0/2527289242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a08313ee10 msgr2=0x55a083141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83da4b1640 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a08313ee10 0x55a083141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83da4b1640 1 -- 172.21.15.5:0/2527289242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 msgr2=0x55a083135cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83da4b1640 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a083135cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83cb7fe640 1 -- 172.21.15.5:0/770753846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f83cc0d0f20 con 0x7f83d419a770 2024-04-22T13:34:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83da4b1640 1 -- 172.21.15.5:0/2527289242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a082e747e0 con 0x55a083136290 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83da4b1640 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313e840 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f83d0036040 tx=0x7f83d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eee7c53fa915ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83dacb2640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f83c00a3d20 0x7f83c00a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0ce720 tx=0x7f83cc092350 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83c8ff9640 1 -- 172.21.15.5:0/2527289242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0045070 con 0x55a083136290 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83c8ff9640 1 -- 172.21.15.5:0/2527289242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d002c7e0 con 0x55a083136290 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.824+0000 7f83c8ff9640 1 -- 172.21.15.5:0/2527289242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0003ce0 con 0x55a083136290 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 -- 172.21.15.5:0/2527289242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 msgr2=0x55a08313e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313e840 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f83d0036040 tx=0x7f83d0004040 comp rx=0 tx=0).stop 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 -- 172.21.15.5:0/2527289242 shutdown_connections 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313e840 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a08313ee10 0x55a083141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 --2- 172.21.15.5:0/2527289242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a083135cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 -- 172.21.15.5:0/2527289242 >> 172.21.15.5:0/2527289242 conn(0x55a08312ffb0 msgr2=0x55a083130390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 -- 172.21.15.5:0/2527289242 shutdown_connections 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.825+0000 7f83df3e8740 1 -- 172.21.15.5:0/2527289242 wait complete. 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 -- start start 2024-04-22T13:34:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 Processor -- start 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0831358e0 0x55a08313bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a08313c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a083141d20 con 0x55a083136290 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a083053580 con 0x55a08313ee10 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a083141ea0 con 0x55a0831358e0 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83df3e8740 10 client.0 _ll_register_callbacks cb 0x55a083059650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45062/0 (socket says 172.21.15.5:45062) 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 -- 172.21.15.5:0/1486518354 learned_addr learned my addr 172.21.15.5:0/1486518354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0831358e0 0x55a08313bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a08313c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 -- 172.21.15.5:0/1486518354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0831358e0 msgr2=0x55a08313bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0831358e0 0x55a08313bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 -- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 msgr2=0x55a08313c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a08313c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83da4b1640 1 -- 172.21.15.5:0/1486518354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a082ed4f70 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.826+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a08313c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83da4b1640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313c290 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000c00 tx=0x7f83d00037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0d5560cc8cb923 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83bdffb640 1 -- 172.21.15.5:0/1486518354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0045040 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d002c7e0 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83bdffb640 1 -- 172.21.15.5:0/1486518354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f83d0003e10 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83bdffb640 1 -- 172.21.15.5:0/1486518354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83d0058050 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 10 client.18450 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a082d6c170 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83bdffb640 1 -- 172.21.15.5:0/1486518354 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d002c7e0 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83bdffb640 1 -- 172.21.15.5:0/1486518354 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f83d003f440 con 0x55a083136290 2024-04-22T13:34:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 10 client.18450 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 20 client.18450 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a08314f140 0x55a083151530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 4 client.18450 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.827+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0830a7920 con 0x55a08314f140 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.828+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a08314f140 0x55a083151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.828+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a08314f140 0x55a083151530 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc649ec3327e1a8 server_cookie=e081d101d37de75b in_seq=0 out_seq=0 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.828+0000 7f83bdffb640 10 client.18450 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83bdffb640 1 -- 172.21.15.5:0/1486518354 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0830a7920 con 0x55a08314f140 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83bdffb640 10 client.18450 handle_command_reply: tid=0 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 client.18450 shutdown 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 msgr2=0x55a08313c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313c290 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000c00 tx=0x7f83d00037e0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 shutdown_connections 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a08314f140 0x55a083151530 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a083136290 0x55a08313c290 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a08313c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1486518354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0831358e0 0x55a08313bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83bdffb640 0 client.18450 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 >> 172.21.15.5:0/1486518354 conn(0x55a08312ffb0 msgr2=0x55a083131d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 shutdown_connections 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 1 -- 172.21.15.5:0/1486518354 wait complete. 2024-04-22T13:34:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.829+0000 7f83df3e8740 20 client.18450 trim_cache size 0 max 16384 2024-04-22T13:34:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.847+0000 7f83df3e8740 1 Processor -- start 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.847+0000 7f83df3e8740 1 -- start start 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a08313e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a08313ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 0x55a08314ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83df3e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41afc40 con 0x55a08313ee10 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83df3e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b1780 con 0x55a0831358e0 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83df3e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b0310 con 0x55a083136290 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 0x55a08314ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 0x55a08314ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45066/0 (socket says 172.21.15.5:45066) 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 -- 172.21.15.5:0/1098098412 learned_addr learned my addr 172.21.15.5:0/1098098412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a08313e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83da4b1640 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a08313ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 -- 172.21.15.5:0/1098098412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 msgr2=0x55a08313ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a08313ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 -- 172.21.15.5:0/1098098412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 msgr2=0x55a08313e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a08313e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 -- 172.21.15.5:0/1098098412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a083136030 con 0x55a08313ee10 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 0x55a08314ac40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f83c4014440 tx=0x7f83c4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a753ca133ea2db51 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83bffff640 1 -- 172.21.15.5:0/1098098412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c401f850 con 0x55a08313ee10 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.848+0000 7f83bffff640 1 -- 172.21.15.5:0/1098098412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c4014ab0 con 0x55a08313ee10 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83bffff640 1 -- 172.21.15.5:0/1098098412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4011a20 con 0x55a08313ee10 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 -- 172.21.15.5:0/1098098412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 msgr2=0x55a08314ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 0x55a08314ac40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f83c4014440 tx=0x7f83c4014040 comp rx=0 tx=0).stop 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 -- 172.21.15.5:0/1098098412 shutdown_connections 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a08313ee10 0x55a08314ac40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a08313ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 --2- 172.21.15.5:0/1098098412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a0831358e0 0x55a08313e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 -- 172.21.15.5:0/1098098412 >> 172.21.15.5:0/1098098412 conn(0x55a08312ffb0 msgr2=0x55a08313c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 -- 172.21.15.5:0/1098098412 shutdown_connections 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 -- 172.21.15.5:0/1098098412 wait complete. 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 -- start start 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.849+0000 7f83df3e8740 1 Processor -- start 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 0x55a0831470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a083147630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a083143540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b10b0 con 0x55a0831358e0 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b09e0 con 0x55a08313ee10 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41adc40 con 0x55a083136290 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83df3e8740 10 client.0 _ll_register_callbacks cb 0x7f83c400cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 0x55a0831470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 0x55a0831470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45074/0 (socket says 172.21.15.5:45074) 2024-04-22T13:34:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 -- 172.21.15.5:0/86452617 learned_addr learned my addr 172.21.15.5:0/86452617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83da4b1640 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a083147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 -- 172.21.15.5:0/86452617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 msgr2=0x55a083147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a083147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 -- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 msgr2=0x55a083143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a083143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 -- 172.21.15.5:0/86452617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a083143d70 con 0x55a0831358e0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83d9cb0640 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 0x55a0831470f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f83c4014010 tx=0x7f83c4014160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e968068ab36e8516 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.850+0000 7f83bdffb640 1 -- 172.21.15.5:0/86452617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4014600 con 0x55a0831358e0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c4014ab0 con 0x55a0831358e0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83bdffb640 1 -- 172.21.15.5:0/86452617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f83c40147a0 con 0x55a0831358e0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83bdffb640 1 -- 172.21.15.5:0/86452617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83c4004020 con 0x55a0831358e0 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 10 client.18456 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a0831b5fc0 con 0x55a0831358e0 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83bdffb640 1 -- 172.21.15.5:0/86452617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c4014ab0 con 0x55a0831358e0 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83bdffb640 1 -- 172.21.15.5:0/86452617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f83c4004700 con 0x55a0831358e0 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 10 client.18456 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 20 client.18456 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a0831b65c0 0x55a0831b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 4 client.18456 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.851+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0830a1790 con 0x55a0831b65c0 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.852+0000 7f83da4b1640 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a0831b65c0 0x55a0831b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.852+0000 7f83bdffb640 10 client.18456 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.852+0000 7f83da4b1640 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a0831b65c0 0x55a0831b89e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b511a9b064922d server_cookie=7c1f6875f38a5bac in_seq=0 out_seq=0 2024-04-22T13:34:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83bdffb640 1 -- 172.21.15.5:0/86452617 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0830a1790 con 0x55a0831b65c0 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83bdffb640 10 client.18456 handle_command_reply: tid=0 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 client.18456 shutdown 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 msgr2=0x55a0831470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 0x55a0831470f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f83c4014010 tx=0x7f83c4014160 comp rx=0 tx=0).stop 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 shutdown_connections 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a0831b65c0 0x55a0831b89e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a08313ee10 0x55a083143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a083136290 0x55a083147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 --2- 172.21.15.5:0/86452617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0831358e0 0x55a0831470f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83bdffb640 0 client.18456 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 >> 172.21.15.5:0/86452617 conn(0x55a08312ffb0 msgr2=0x55a08312e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 shutdown_connections 2024-04-22T13:34:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 1 -- 172.21.15.5:0/86452617 wait complete. 2024-04-22T13:34:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.853+0000 7f83df3e8740 20 client.18456 trim_cache size 0 max 16384 2024-04-22T13:34:55.855 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:34:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:34:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5b5ad6ca-995f-40a3-ba86-763bf0fea388", 2024-04-22T13:34:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:34:55.855 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f83c00a3d20 msgr2=0x7f83c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f83c00a3d20 0x7f83c00a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0ce720 tx=0x7f83cc092350 comp rx=0 tx=0).stop 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d419a770 msgr2=0x7f83d419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d419a770 0x7f83d419abf0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00cab0 tx=0x7f83cc00cf70 comp rx=0 tx=0).stop 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 shutdown_connections 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83d419a770 0x7f83d419abf0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f83c00a3d20 0x7f83c00a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83d4106170 0x7f83d41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 --2- 172.21.15.5:0/770753846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83d41010a0 0x7f83d41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 >> 172.21.15.5:0/770753846 conn(0x7f83d4077f40 msgr2=0x7f83d40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 shutdown_connections 2024-04-22T13:34:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.854+0000 7f83dc715640 1 -- 172.21.15.5:0/770753846 wait complete. 2024-04-22T13:34:55.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:55.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 -- 172.21.15.5:0/533536221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0106170 msgr2=0x7f5fd010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/533536221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0106170 0x7f5fd010dd40 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009ca0 tx=0x7f5fc802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 -- 172.21.15.5:0/533536221 shutdown_connections 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/533536221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0106170 0x7f5fd010dd40 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/533536221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd0101a50 0x7f5fd0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/533536221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fd01010a0 0x7f5fd0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 -- 172.21.15.5:0/533536221 >> 172.21.15.5:0/533536221 conn(0x7f5fd0077f40 msgr2=0x7f5fd0078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 -- 172.21.15.5:0/533536221 shutdown_connections 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.969+0000 7f5fd77d0640 1 -- 172.21.15.5:0/533536221 wait complete. 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 Processor -- start 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 -- start start 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 0x7f5fd01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fd0106170 0x7f5fd0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd019cf60 0x7f5fd019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01110c0 con 0x7f5fd01010a0 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0110f40 con 0x7f5fd0106170 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd77d0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0111240 con 0x7f5fd019cf60 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 0x7f5fd01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 0x7f5fd01986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45104/0 (socket says 172.21.15.5:45104) 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 -- 172.21.15.5:0/697707052 learned_addr learned my addr 172.21.15.5:0/697707052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd019cf60 0x7f5fd019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 -- 172.21.15.5:0/697707052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd019cf60 msgr2=0x7f5fd019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd019cf60 0x7f5fd019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 -- 172.21.15.5:0/697707052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fd0106170 msgr2=0x7f5fd0198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fd0106170 0x7f5fd0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd556c640 1 -- 172.21.15.5:0/697707052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd019da50 con 0x7f5fd01010a0 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.970+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd019cf60 0x7f5fd019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.971+0000 7f5fd556c640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 0x7f5fd01986a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00d950 tx=0x7f5fcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.971+0000 7f5fc27fc640 1 -- 172.21.15.5:0/697707052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc017030 con 0x7f5fd01010a0 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.971+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd019dd40 con 0x7f5fd01010a0 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.971+0000 7f5fc27fc640 1 -- 172.21.15.5:0/697707052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fcc004020 con 0x7f5fd01010a0 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.971+0000 7f5fc27fc640 1 -- 172.21.15.5:0/697707052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0047a0 con 0x7f5fd01010a0 2024-04-22T13:34:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.971+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a5580 con 0x7f5fd01010a0 2024-04-22T13:34:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.973+0000 7f5fda3bd740 1 Processor -- start 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.973+0000 7f5fda3bd740 1 -- start start 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.973+0000 7f5fc27fc640 1 -- 172.21.15.5:0/697707052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5fcc009a30 con 0x7f5fd01010a0 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 0x564789532340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789532910 0x56478953f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789542280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647894971e0 con 0x564789531f60 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x564789434390 con 0x56478953fe90 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5647894931a0 con 0x564789532910 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 0x564789532340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 0x564789532340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45106/0 (socket says 172.21.15.5:45106) 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 -- 172.21.15.5:0/2295322065 learned_addr learned my addr 172.21.15.5:0/2295322065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789542280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd556c640 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789532910 0x56478953f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 -- 172.21.15.5:0/2295322065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789532910 msgr2=0x56478953f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789532910 0x56478953f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 -- 172.21.15.5:0/2295322065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 msgr2=0x564789542280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789542280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 -- 172.21.15.5:0/2295322065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647892737d0 con 0x564789531f60 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fc27fc640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5fbc0a3d20 0x7f5fbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 0x564789532340 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004af0 tx=0x7f5fc8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a451f72d7133c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fd556c640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5fbc0a3d20 0x7f5fbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.974+0000 7f5fbb7fe640 1 -- 172.21.15.5:0/2295322065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8039050 con 0x564789531f60 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fbb7fe640 1 -- 172.21.15.5:0/2295322065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc8045070 con 0x564789531f60 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fbb7fe640 1 -- 172.21.15.5:0/2295322065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004040 con 0x564789531f60 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fc27fc640 1 -- 172.21.15.5:0/697707052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5fcc0d0310 con 0x7f5fd01010a0 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 -- 172.21.15.5:0/2295322065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 msgr2=0x564789532340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fd556c640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5fbc0a3d20 0x7f5fbc0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0cdb10 tx=0x7f5fcc0d5040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 0x564789532340 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004af0 tx=0x7f5fc8009800 comp rx=0 tx=0).stop 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 -- 172.21.15.5:0/2295322065 shutdown_connections 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789542280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789532910 0x56478953f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/2295322065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789531f60 0x564789532340 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 -- 172.21.15.5:0/2295322065 >> 172.21.15.5:0/2295322065 conn(0x56478952ccb0 msgr2=0x56478952d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 -- 172.21.15.5:0/2295322065 shutdown_connections 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.975+0000 7f5fda3bd740 1 -- 172.21.15.5:0/2295322065 wait complete. 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 -- start start 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 Processor -- start 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478953d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x56478953d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564789542bf0 con 0x564789532910 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5647894502f0 con 0x56478953fe90 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564789542d70 con 0x564789531f60 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fda3bd740 10 client.0 _ll_register_callbacks cb 0x56478945d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fd4d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478953d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fd4d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478953d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45108/0 (socket says 172.21.15.5:45108) 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/4052772998 learned_addr learned my addr 172.21.15.5:0/4052772998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.976+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x56478953d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/4052772998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 msgr2=0x56478953ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 msgr2=0x56478953d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x56478953d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/4052772998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647891a93f0 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478953d3d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400f670 tx=0x7f5fc400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3c9fb56d437007 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/4052772998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc401d060 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/4052772998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5fc400db20 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc8045070 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/4052772998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fc400dd20 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fda3bd740 10 client.18468 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647891690f0 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/4052772998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc8045070 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.977+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/4052772998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fc40203b0 con 0x564789532910 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fda3bd740 10 client.18468 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fda3bd740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fda3bd740 20 client.18468 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56478954fea0 0x564789552290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fda3bd740 4 client.18468 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647894a4c00 con 0x56478954fea0 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56478954fea0 0x564789552290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56478954fea0 0x564789552290 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790c572bce2e7b21 server_cookie=8cbdaeff3f636e9e in_seq=0 out_seq=0 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.978+0000 7f5fb8ff9640 10 client.18468 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/4052772998 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647894a4c00 con 0x56478954fea0 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fb8ff9640 10 client.18468 handle_command_reply: tid=0 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 client.18468 shutdown 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 msgr2=0x56478953d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478953d3d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400f670 tx=0x7f5fc400fb30 comp rx=0 tx=0).stop 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 shutdown_connections 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x56478954fea0 0x564789552290 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478953d3d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fb8ff9640 0 client.18468 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:55.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x56478953d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/4052772998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 >> 172.21.15.5:0/4052772998 conn(0x56478952ccb0 msgr2=0x56478952ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:55.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 shutdown_connections 2024-04-22T13:34:55.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 1 -- 172.21.15.5:0/4052772998 wait complete. 2024-04-22T13:34:55.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.979+0000 7f5fda3bd740 20 client.18468 trim_cache size 0 max 16384 2024-04-22T13:34:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 Processor -- start 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 -- start start 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478954baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789557b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ab980 con 0x564789532910 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ad4c0 con 0x56478953fe90 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ac050 con 0x564789531f60 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478954baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478954baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45112/0 (socket says 172.21.15.5:45112) 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1391319225 learned_addr learned my addr 172.21.15.5:0/1391319225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789557b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1391319225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 msgr2=0x56478953f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1391319225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 msgr2=0x564789557b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789557b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1391319225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647895326b0 con 0x564789532910 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789557b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.997+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478954baf0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4001eb0 tx=0x7f5fc400ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9920f6aea1c54434 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fbaffd640 1 -- 172.21.15.5:0/1391319225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc402d020 con 0x564789532910 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fbaffd640 1 -- 172.21.15.5:0/1391319225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc400d710 con 0x564789532910 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fbaffd640 1 -- 172.21.15.5:0/1391319225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4004a40 con 0x564789532910 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1391319225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 msgr2=0x56478954baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478954baf0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4001eb0 tx=0x7f5fc400ffb0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1391319225 shutdown_connections 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x56478954baf0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56478953fe90 0x564789557b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1391319225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564789531f60 0x56478953f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1391319225 >> 172.21.15.5:0/1391319225 conn(0x56478952ccb0 msgr2=0x564789538210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1391319225 shutdown_connections 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1391319225 wait complete. 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.998+0000 7f5fda3bd740 1 -- start start 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 Processor -- start 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564789531f60 0x564789543980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x564789543ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56478953fe90 0x564789544400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01acdf0 con 0x564789532910 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ac720 con 0x564789531f60 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01a9980 con 0x56478953fe90 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fda3bd740 10 client.0 _ll_register_callbacks cb 0x7f5fc400a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x564789543ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x564789543ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45114/0 (socket says 172.21.15.5:45114) 2024-04-22T13:34:56.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1034702785 learned_addr learned my addr 172.21.15.5:0/1034702785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56478953fe90 0x564789544400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564789531f60 0x564789543980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1034702785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56478953fe90 msgr2=0x564789544400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56478953fe90 0x564789544400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564789531f60 msgr2=0x564789543980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564789531f60 0x564789543980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd4d6b640 1 -- 172.21.15.5:0/1034702785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647895b53f0 con 0x564789532910 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:55.999+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564789531f60 0x564789543980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fd4d6b640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x564789543ec0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40208f0 tx=0x7f5fc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238b25b0fa831fec server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/1034702785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc401d040 con 0x564789532910 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/1034702785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5fc40047e0 con 0x564789532910 2024-04-22T13:34:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc400d710 con 0x564789532910 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/1034702785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fc4040050 con 0x564789532910 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 10 client.18474 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647895b6640 con 0x564789532910 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/1034702785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc400d710 con 0x564789532910 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/1034702785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fc4032450 con 0x564789532910 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 10 client.18474 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 20 client.18474 populate_metadata read hostname 'smithi005' 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5647895b6da0 0x5647895b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 4 client.18474 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.000+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56478949ea70 con 0x5647895b6da0 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5647895b6da0 0x5647895b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fd5d6d640 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5647895b6da0 0x5647895b91c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f79e3ddd8d7d04b server_cookie=4e39821e0a364d9 in_seq=0 out_seq=0 2024-04-22T13:34:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fb8ff9640 10 client.18474 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fb8ff9640 1 -- 172.21.15.5:0/1034702785 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56478949ea70 con 0x5647895b6da0 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fb8ff9640 10 client.18474 handle_command_reply: tid=0 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 client.18474 shutdown 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 msgr2=0x564789543ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x564789543ec0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40208f0 tx=0x7f5fc400dc60 comp rx=0 tx=0).stop 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 shutdown_connections 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5647895b6da0 0x5647895b91c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564789532910 0x564789543ec0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56478953fe90 0x564789544400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fb8ff9640 0 client.18474 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 --2- 172.21.15.5:0/1034702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564789531f60 0x564789543980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 >> 172.21.15.5:0/1034702785 conn(0x56478952ccb0 msgr2=0x56478953cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 shutdown_connections 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.001+0000 7f5fda3bd740 1 -- 172.21.15.5:0/1034702785 wait complete. 2024-04-22T13:34:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fda3bd740 20 client.18474 trim_cache size 0 max 16384 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: "5b5ad6ca-995f-40a3-ba86-763bf0fea388": { 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5b5ad6ca-995f-40a3-ba86-763bf0fea388", 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:34:56.004 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:34:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5fbc0a3d20 msgr2=0x7f5fbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5fbc0a3d20 0x7f5fbc0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0cdb10 tx=0x7f5fcc0d5040 comp rx=0 tx=0).stop 2024-04-22T13:34:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 msgr2=0x7f5fd01986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 0x7f5fd01986a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00d950 tx=0x7f5fcc00de10 comp rx=0 tx=0).stop 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 shutdown_connections 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5fbc0a3d20 0x7f5fbc0a61e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fd019cf60 0x7f5fd019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fd0106170 0x7f5fd0198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 --2- 172.21.15.5:0/697707052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01010a0 0x7f5fd01986a0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 >> 172.21.15.5:0/697707052 conn(0x7f5fd0077f40 msgr2=0x7f5fd006a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 shutdown_connections 2024-04-22T13:34:56.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:34:56.002+0000 7f5fd77d0640 1 -- 172.21.15.5:0/697707052 wait complete. 2024-04-22T13:34:56.008 INFO:tasks.cephfs.filesystem:scrub status for tag:5b5ad6ca-995f-40a3-ba86-763bf0fea388 - {'path': '/', 'tag': '5b5ad6ca-995f-40a3-ba86-763bf0fea388', 'options': 'recursive,force'} 2024-04-22T13:34:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:56 smithi049 ceph-mon[31379]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:56 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:34:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:56 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:56 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:34:57.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:56 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:56 smithi107 ceph-mon[31859]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:56 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:34:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:56 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:56 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:34:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:56 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:56 smithi005 ceph-mon[28006]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:56 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:34:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:56 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:56 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:34:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:56 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:59.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:34:58 smithi049 ceph-mon[31379]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:34:58 smithi107 ceph-mon[31859]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:59.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:34:58 smithi005 ceph-mon[28006]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:01.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:00 smithi049 ceph-mon[31379]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:01.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:00 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:35:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:00 smithi107 ceph-mon[31859]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:00 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:35:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:00 smithi005 ceph-mon[28006]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:00 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:35:03.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:02 smithi049 ceph-mon[31379]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:35:03.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:02 smithi107 ceph-mon[31859]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:35:03.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:02 smithi005 ceph-mon[28006]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:35:05.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:04 smithi049 ceph-mon[31379]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:35:05.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:04 smithi107 ceph-mon[31859]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:35:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:04 smithi005 ceph-mon[28006]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:35:07.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:06 smithi049 ceph-mon[31379]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:35:07.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:06 smithi049 ceph-mon[31379]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T13:35:07.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:06 smithi107 ceph-mon[31859]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:35:07.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:06 smithi107 ceph-mon[31859]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T13:35:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:06 smithi005 ceph-mon[28006]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:35:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:06 smithi005 ceph-mon[28006]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T13:35:09.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:08 smithi049 ceph-mon[31379]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:35:09.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:08 smithi107 ceph-mon[31859]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:35:09.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:08 smithi005 ceph-mon[28006]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:35:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:10 smithi049 ceph-mon[31379]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:35:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:10 smithi107 ceph-mon[31859]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:35:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:10 smithi005 ceph-mon[28006]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:35:13.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:12 smithi049 ceph-mon[31379]: pgmap v248: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:13.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:12 smithi049 ceph-mon[31379]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T13:35:13.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:12 smithi107 ceph-mon[31859]: pgmap v248: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:13.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:12 smithi107 ceph-mon[31859]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T13:35:13.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:12 smithi005 ceph-mon[28006]: pgmap v248: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:13.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:12 smithi005 ceph-mon[28006]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T13:35:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:14 smithi049 ceph-mon[31379]: pgmap v250: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:35:15.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:14 smithi107 ceph-mon[31859]: pgmap v250: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:35:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:14 smithi005 ceph-mon[28006]: pgmap v250: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:35:17.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:16 smithi049 ceph-mon[31379]: pgmap v251: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-22T13:35:17.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:16 smithi107 ceph-mon[31859]: pgmap v251: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-22T13:35:17.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:16 smithi005 ceph-mon[28006]: pgmap v251: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-22T13:35:19.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:18 smithi049 ceph-mon[31379]: pgmap v252: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:19.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:18 smithi107 ceph-mon[31859]: pgmap v252: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:18 smithi005 ceph-mon[28006]: pgmap v252: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:21.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:20 smithi049 ceph-mon[31379]: pgmap v253: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:21.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:20 smithi107 ceph-mon[31859]: pgmap v253: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:21.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:20 smithi005 ceph-mon[28006]: pgmap v253: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:22.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:21 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:35:22.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:21 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:35:22.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:21 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:35:23.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:22 smithi049 ceph-mon[31379]: pgmap v254: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:35:23.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:22 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:35:23.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:22 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:35:23.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:22 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:35:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:22 smithi107 ceph-mon[31859]: pgmap v254: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:35:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:22 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:35:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:22 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:35:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:22 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:35:23.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:22 smithi005 ceph-mon[28006]: pgmap v254: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:35:23.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:22 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:35:23.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:22 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:35:23.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:22 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:35:25.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:24 smithi049 ceph-mon[31379]: pgmap v255: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-22T13:35:25.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:24 smithi107 ceph-mon[31859]: pgmap v255: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-22T13:35:25.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:24 smithi005 ceph-mon[28006]: pgmap v255: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-22T13:35:26.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:35:26.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:35:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 -- 172.21.15.5:0/2944966396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48102b30 msgr2=0x7f5d48106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 --2- 172.21.15.5:0/2944966396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48102b30 0x7f5d48106b70 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009900 tx=0x7f5d3802bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 -- 172.21.15.5:0/2944966396 shutdown_connections 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 --2- 172.21.15.5:0/2944966396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48102b30 0x7f5d48106b70 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 --2- 172.21.15.5:0/2944966396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d48107250 0x7f5d4810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 --2- 172.21.15.5:0/2944966396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d48102180 0x7f5d48102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.130+0000 7f5d503e6640 1 -- 172.21.15.5:0/2944966396 >> 172.21.15.5:0/2944966396 conn(0x7f5d480fda00 msgr2=0x7f5d480ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 -- 172.21.15.5:0/2944966396 shutdown_connections 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 -- 172.21.15.5:0/2944966396 wait complete. 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 Processor -- start 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 -- start start 2024-04-22T13:35:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d48102180 0x7f5d4819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 0x7f5d4819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d48198680 0x7f5d48198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481121e0 con 0x7f5d48107250 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48112060 con 0x7f5d48102180 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d503e6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48112360 con 0x7f5d48198680 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.131+0000 7f5d4d981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 0x7f5d4819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 0x7f5d4819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58702/0 (socket says 172.21.15.5:58702) 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 -- 172.21.15.5:0/4181306551 learned_addr learned my addr 172.21.15.5:0/4181306551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 -- 172.21.15.5:0/4181306551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d48198680 msgr2=0x7f5d48198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4e182640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d48102180 0x7f5d4819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4e983640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d48198680 0x7f5d48198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d48198680 0x7f5d48198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 -- 172.21.15.5:0/4181306551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d48102180 msgr2=0x7f5d4819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d48102180 0x7f5d4819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 -- 172.21.15.5:0/4181306551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d48199260 con 0x7f5d48107250 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d4d981640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 0x7f5d4819ea70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000c00 tx=0x7f5d380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d3f7fe640 1 -- 172.21.15.5:0/4181306551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38043020 con 0x7f5d48107250 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d3f7fe640 1 -- 172.21.15.5:0/4181306551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3802cb70 con 0x7f5d48107250 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d481a5340 con 0x7f5d48107250 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d3f7fe640 1 -- 172.21.15.5:0/4181306551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38003b80 con 0x7f5d48107250 2024-04-22T13:35:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.132+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d481a56f0 con 0x7f5d48107250 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.135+0000 7f5d3f7fe640 1 -- 172.21.15.5:0/4181306551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5d38046020 con 0x7f5d48107250 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 Processor -- start 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 -- start start 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986852650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b98685f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986862280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9867b71e0 con 0x55b986852b90 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b986754390 con 0x55b98685fe90 2024-04-22T13:35:26.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9867b31a0 con 0x55b986852270 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d4e182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b98685f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d4e182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b98685f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58718/0 (socket says 172.21.15.5:58718) 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d4e983640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986852650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d4e182640 1 -- 172.21.15.5:0/966788644 learned_addr learned my addr 172.21.15.5:0/966788644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.136+0000 7f5d4d981640 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986862280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e182640 1 -- 172.21.15.5:0/966788644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 msgr2=0x55b986852650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e182640 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986852650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e182640 1 -- 172.21.15.5:0/966788644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 msgr2=0x55b986862280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e182640 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986862280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e182640 1 -- 172.21.15.5:0/966788644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9865937d0 con 0x55b986852b90 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d3f7fe640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d340a3da0 0x7f5d340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e983640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d340a3da0 0x7f5d340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d4e182640 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b98685f950 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009c50 tx=0x7f5d44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77f3612c37475f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d3f7fe640 1 -- 172.21.15.5:0/4181306551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f5d380f27e0 con 0x7f5d48107250 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d3cbfe640 1 -- 172.21.15.5:0/966788644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4401b480 con 0x55b986852b90 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d3cbfe640 1 -- 172.21.15.5:0/966788644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d44012900 con 0x55b986852b90 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.137+0000 7f5d3cbfe640 1 -- 172.21.15.5:0/966788644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d440118b0 con 0x55b986852b90 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d4e983640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d340a3da0 0x7f5d340a6260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d40005fd0 tx=0x7f5d400095a0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 -- 172.21.15.5:0/966788644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 msgr2=0x55b98685f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b98685f950 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009c50 tx=0x7f5d44010740 comp rx=0 tx=0).stop 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 -- 172.21.15.5:0/966788644 shutdown_connections 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b98685f950 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986862280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/966788644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986852650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 -- 172.21.15.5:0/966788644 >> 172.21.15.5:0/966788644 conn(0x55b98684ccb0 msgr2=0x55b98684d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 -- 172.21.15.5:0/966788644 shutdown_connections 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 -- 172.21.15.5:0/966788644 wait complete. 2024-04-22T13:35:26.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.138+0000 7f5d52fd4740 1 -- start start 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 Processor -- start 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98685ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98685d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b986862c80 con 0x55b986852270 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b9867702f0 con 0x55b98685fe90 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b986862e00 con 0x55b986852b90 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d52fd4740 10 client.0 _ll_register_callbacks cb 0x55b98677d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98685ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98685ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58734/0 (socket says 172.21.15.5:58734) 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 -- 172.21.15.5:0/190082273 learned_addr learned my addr 172.21.15.5:0/190082273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e983640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98685d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4d981640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 -- 172.21.15.5:0/190082273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 msgr2=0x55b98685d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 -- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 msgr2=0x55b98685d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98685d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 -- 172.21.15.5:0/190082273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9864c93f0 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4d981640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d4e182640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98685ceb0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009fa0 tx=0x7f5d44009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31adc822c9a67960 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.139+0000 7f5d31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d31ffb640 1 -- 172.21.15.5:0/190082273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44012cc0 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d31ffb640 1 -- 172.21.15.5:0/190082273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f5d44005040 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d44012900 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d31ffb640 1 -- 172.21.15.5:0/190082273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d44005240 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 10 client.18486 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9864890f0 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d31ffb640 1 -- 172.21.15.5:0/190082273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d44012900 con 0x55b986852270 2024-04-22T13:35:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d31ffb640 1 -- 172.21.15.5:0/190082273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d4400f040 con 0x55b986852270 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 10 client.18486 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 20 client.18486 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b98686fd30 0x55b986872120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 4 client.18486 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.140+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9867c4c00 con 0x55b98686fd30 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.141+0000 7f5d4d981640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b98686fd30 0x55b986872120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.141+0000 7f5d4d981640 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b98686fd30 0x55b986872120 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796cc65fc2f63c8c server_cookie=e3501a23328b326b in_seq=0 out_seq=0 2024-04-22T13:35:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.141+0000 7f5d31ffb640 10 client.18486 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d31ffb640 1 -- 172.21.15.5:0/190082273 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9867c4c00 con 0x55b98686fd30 2024-04-22T13:35:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d31ffb640 10 client.18486 handle_command_reply: tid=0 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 client.18486 shutdown 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 msgr2=0x55b98685ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98685ceb0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009fa0 tx=0x7f5d44009fd0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 shutdown_connections 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b98686fd30 0x55b986872120 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98685d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d31ffb640 0 client.18486 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/190082273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98685ceb0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 >> 172.21.15.5:0/190082273 conn(0x55b98684ccb0 msgr2=0x55b98684ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 shutdown_connections 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 1 -- 172.21.15.5:0/190082273 wait complete. 2024-04-22T13:35:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.142+0000 7f5d52fd4740 20 client.18486 trim_cache size 0 max 16384 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 Processor -- start 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 -- start start 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98686bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98684e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481ab9c0 con 0x55b986852270 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481ad500 con 0x55b98685fe90 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481ac090 con 0x55b986852b90 2024-04-22T13:35:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98686bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98686bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58744/0 (socket says 172.21.15.5:58744) 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 -- 172.21.15.5:0/3484332213 learned_addr learned my addr 172.21.15.5:0/3484332213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 -- 172.21.15.5:0/3484332213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 msgr2=0x55b98685c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 -- 172.21.15.5:0/3484332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 msgr2=0x55b98684e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98684e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.160+0000 7f5d4e182640 1 -- 172.21.15.5:0/3484332213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9868532f0 con 0x55b986852270 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d4e182640 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98686bb80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5d44001eb0 tx=0x7f5d44005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68637ef616bc486e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d33fff640 1 -- 172.21.15.5:0/3484332213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44034070 con 0x55b986852270 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d33fff640 1 -- 172.21.15.5:0/3484332213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d44011e40 con 0x55b986852270 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d33fff640 1 -- 172.21.15.5:0/3484332213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44020030 con 0x55b986852270 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 -- 172.21.15.5:0/3484332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 msgr2=0x55b98686bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98686bb80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5d44001eb0 tx=0x7f5d44005b10 comp rx=0 tx=0).stop 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 -- 172.21.15.5:0/3484332213 shutdown_connections 2024-04-22T13:35:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b98684e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852b90 0x55b98685c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/3484332213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852270 0x55b98686bb80 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 -- 172.21.15.5:0/3484332213 >> 172.21.15.5:0/3484332213 conn(0x55b98684ccb0 msgr2=0x55b98684e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 -- 172.21.15.5:0/3484332213 shutdown_connections 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.161+0000 7f5d52fd4740 1 -- 172.21.15.5:0/3484332213 wait complete. 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 -- start start 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 Processor -- start 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986868520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b986859940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986859ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481ace30 con 0x55b986852b90 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481ac760 con 0x55b98685fe90 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481a99c0 con 0x55b986852270 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d52fd4740 10 client.0 _ll_register_callbacks cb 0x7f5d380f8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b986859940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b986859940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58748/0 (socket says 172.21.15.5:58748) 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 -- 172.21.15.5:0/2992637797 learned_addr learned my addr 172.21.15.5:0/2992637797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4e182640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986868520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4e983640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986859ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 -- 172.21.15.5:0/2992637797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 msgr2=0x55b986868520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986868520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 -- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 msgr2=0x55b986859ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986859ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 -- 172.21.15.5:0/2992637797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98685a6f0 con 0x55b986852b90 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4e983640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986859ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d4d981640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b986859940 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5d380bb260 tx=0x7f5d380035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d029fbe9a92261 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.162+0000 7f5d31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d31ffb640 1 -- 172.21.15.5:0/2992637797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380bef90 con 0x55b986852b90 2024-04-22T13:35:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d31ffb640 1 -- 172.21.15.5:0/2992637797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f5d380f01b0 con 0x55b986852b90 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9867bea70 con 0x55b986852b90 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d31ffb640 1 -- 172.21.15.5:0/2992637797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d380be890 con 0x55b986852b90 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 10 client.18492 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9868d6280 con 0x55b986852b90 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d31ffb640 1 -- 172.21.15.5:0/2992637797 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b9867bea70 con 0x55b986852b90 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d31ffb640 1 -- 172.21.15.5:0/2992637797 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d380a90b0 con 0x55b986852b90 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 10 client.18492 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 20 client.18492 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9868d6a00 0x55b9868d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 4 client.18492 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b98673a590 con 0x55b9868d6a00 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.163+0000 7f5d4e182640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9868d6a00 0x55b9868d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d4e182640 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9868d6a00 0x55b9868d8e20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68929fa9424ae0c server_cookie=cf2f256803c86d12 in_seq=0 out_seq=0 2024-04-22T13:35:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d31ffb640 10 client.18492 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d31ffb640 1 -- 172.21.15.5:0/2992637797 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b98673a590 con 0x55b9868d6a00 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d31ffb640 10 client.18492 handle_command_reply: tid=0 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 client.18492 shutdown 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 msgr2=0x55b986859940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b986859940 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5d380bb260 tx=0x7f5d380035f0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 shutdown_connections 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b9868d6a00 0x55b9868d8e20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b986852b90 0x55b986859940 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b98685fe90 0x55b986859ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d31ffb640 0 client.18492 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 --2- 172.21.15.5:0/2992637797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b986852270 0x55b986868520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 >> 172.21.15.5:0/2992637797 conn(0x55b98684ccb0 msgr2=0x55b98685d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 shutdown_connections 2024-04-22T13:35:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 1 -- 172.21.15.5:0/2992637797 wait complete. 2024-04-22T13:35:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.164+0000 7f5d52fd4740 20 client.18492 trim_cache size 0 max 16384 2024-04-22T13:35:26.168 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:35:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:35:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:35:26.168 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d340a3da0 msgr2=0x7f5d340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d340a3da0 0x7f5d340a6260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d40005fd0 tx=0x7f5d400095a0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 msgr2=0x7f5d4819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 0x7f5d4819ea70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000c00 tx=0x7f5d380045e0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 shutdown_connections 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d48107250 0x7f5d4819ea70 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f5d340a3da0 0x7f5d340a6260 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d48198680 0x7f5d48198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 --2- 172.21.15.5:0/4181306551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d48102180 0x7f5d4819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 >> 172.21.15.5:0/4181306551 conn(0x7f5d480fda00 msgr2=0x7f5d480fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 shutdown_connections 2024-04-22T13:35:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.165+0000 7f5d503e6640 1 -- 172.21.15.5:0/4181306551 wait complete. 2024-04-22T13:35:26.172 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:35:26.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 -- 172.21.15.5:0/2866673490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac10c590 msgr2=0x7fa5ac10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 --2- 172.21.15.5:0/2866673490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac10c590 0x7fa5ac10e9c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009ca0 tx=0x7fa5a402c180 comp rx=0 tx=0).stop 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 -- 172.21.15.5:0/2866673490 shutdown_connections 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 --2- 172.21.15.5:0/2866673490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac10c590 0x7fa5ac10e9c0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 --2- 172.21.15.5:0/2866673490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac103ae0 0x7fa5ac10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 --2- 172.21.15.5:0/2866673490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 -- 172.21.15.5:0/2866673490 >> 172.21.15.5:0/2866673490 conn(0x7fa5ac077f70 msgr2=0x7fa5ac078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 -- 172.21.15.5:0/2866673490 shutdown_connections 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.278+0000 7fa5b2426640 1 -- 172.21.15.5:0/2866673490 wait complete. 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 Processor -- start 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 -- start start 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac103ae0 0x7fa5ac198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac10c590 0x7fa5ac19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1121b0 con 0x7fa5ac1031c0 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112030 con 0x7fa5ac103ae0 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5b2426640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112330 con 0x7fa5ac10c590 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58776/0 (socket says 172.21.15.5:58776) 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.279+0000 7fa5abfff640 1 -- 172.21.15.5:0/1514685787 learned_addr learned my addr 172.21.15.5:0/1514685787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5b09c3640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac10c590 0x7fa5ac19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5ab7fe640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac103ae0 0x7fa5ac198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5abfff640 1 -- 172.21.15.5:0/1514685787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac10c590 msgr2=0x7fa5ac19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5abfff640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac10c590 0x7fa5ac19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5abfff640 1 -- 172.21.15.5:0/1514685787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac103ae0 msgr2=0x7fa5ac198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5abfff640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac103ae0 0x7fa5ac198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5abfff640 1 -- 172.21.15.5:0/1514685787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac19d6e0 con 0x7fa5ac1031c0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5b09c3640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac10c590 0x7fa5ac19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5abfff640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1986f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9f0 tx=0x7fa59400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400cd80 con 0x7fa5ac1031c0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5940108e0 con 0x7fa5ac1031c0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac19d9d0 con 0x7fa5ac1031c0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594018980 con 0x7fa5ac1031c0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.280+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1a5260 con 0x7fa5ac1031c0 2024-04-22T13:35:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.281+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac069250 con 0x7fa5ac1031c0 2024-04-22T13:35:26.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.282+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa594010430 con 0x7fa5ac1031c0 2024-04-22T13:35:26.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.283+0000 7fa5a97fa640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa5980a3da0 0x7fa5980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.283+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fa5940d1070 con 0x7fa5ac1031c0 2024-04-22T13:35:26.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.283+0000 7fa5ab7fe640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa5980a3da0 0x7fa5980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.283+0000 7fa5ab7fe640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa5980a3da0 0x7fa5980a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa59c005fd0 tx=0x7fa59c00b510 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.287+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa59409d710 con 0x7fa5ac1031c0 2024-04-22T13:35:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.513+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5ac1076e0 con 0x7fa5ac1031c0 2024-04-22T13:35:26.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.514+0000 7fa5a97fa640 1 -- 172.21.15.5:0/1514685787 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7fa59409d010 con 0x7fa5ac1031c0 2024-04-22T13:35:26.510 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:35:26.510 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:35:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:35:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa5980a3da0 msgr2=0x7fa5980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa5980a3da0 0x7fa5980a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa59c005fd0 tx=0x7fa59c00b510 comp rx=0 tx=0).stop 2024-04-22T13:35:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 msgr2=0x7fa5ac1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1986f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9f0 tx=0x7fa59400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 shutdown_connections 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fa5980a3da0 0x7fa5980a6260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5ac10c590 0x7fa5ac19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5ac103ae0 0x7fa5ac198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 --2- 172.21.15.5:0/1514685787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ac1031c0 0x7fa5ac1986f0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 >> 172.21.15.5:0/1514685787 conn(0x7fa5ac077f70 msgr2=0x7fa5ac0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 shutdown_connections 2024-04-22T13:35:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.516+0000 7fa5b2426640 1 -- 172.21.15.5:0/1514685787 wait complete. 2024-04-22T13:35:26.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:35:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/1090257770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94101090 msgr2=0x7f2d94101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/1090257770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94101090 0x7f2d94101470 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009900 tx=0x7f2d8c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/1090257770 shutdown_connections 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/1090257770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94106160 0x7f2d9410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/1090257770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94101a40 0x7f2d94105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/1090257770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94101090 0x7f2d94101470 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/1090257770 >> 172.21.15.5:0/1090257770 conn(0x7f2d94078000 msgr2=0x7f2d94078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/1090257770 shutdown_connections 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/1090257770 wait complete. 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 Processor -- start 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 -- start start 2024-04-22T13:35:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.631+0000 7f2d9ba3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101090 0x7f2d941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94106160 0x7f2d94198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 0x7f2d9419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941121e0 con 0x7f2d9419cfa0 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94112060 con 0x7f2d94101090 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94112360 con 0x7f2d94106160 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 0x7f2d9419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 0x7f2d9419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 -- 172.21.15.5:0/520568516 learned_addr learned my addr 172.21.15.5:0/520568516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d997d6640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101090 0x7f2d941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94106160 0x7f2d94198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 -- 172.21.15.5:0/520568516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94106160 msgr2=0x7f2d94198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94106160 0x7f2d94198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 -- 172.21.15.5:0/520568516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101090 msgr2=0x7f2d941986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101090 0x7f2d941986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 -- 172.21.15.5:0/520568516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9419da90 con 0x7f2d9419cfa0 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94106160 0x7f2d94198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d99fd7640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 0x7f2d9419d450 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000da40 tx=0x7f2d9000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d8a7fc640 1 -- 172.21.15.5:0/520568516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90017070 con 0x7f2d9419cfa0 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d8a7fc640 1 -- 172.21.15.5:0/520568516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d90003e20 con 0x7f2d9419cfa0 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9419dd80 con 0x7f2d9419cfa0 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d8a7fc640 1 -- 172.21.15.5:0/520568516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90010460 con 0x7f2d9419cfa0 2024-04-22T13:35:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.632+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941a5520 con 0x7f2d9419cfa0 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.634+0000 7f2d8a7fc640 1 -- 172.21.15.5:0/520568516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2d90004020 con 0x7f2d9419cfa0 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 Processor -- start 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 -- start start 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93e7e80 0x5620c93f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 0x5620c93f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620c934c1f0 con 0x5620c93f0a00 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5620c92e94b0 con 0x5620c93e74d0 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d9e621740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620c93481b0 con 0x5620c93e7e80 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d8a7fc640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d800a3d20 0x7f2d800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d98fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 0x5620c93f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d98fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 0x5620c93f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58800/0 (socket says 172.21.15.5:58800) 2024-04-22T13:35:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d98fd5640 1 -- 172.21.15.5:0/664497997 learned_addr learned my addr 172.21.15.5:0/664497997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d99fd7640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d800a3d20 0x7f2d800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d8a7fc640 1 -- 172.21.15.5:0/520568516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f2d900d0500 con 0x7f2d9419cfa0 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.635+0000 7f2d997d6640 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93e7e80 0x5620c93f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d98fd5640 1 -- 172.21.15.5:0/664497997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93e7e80 msgr2=0x5620c93f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93e7e80 0x5620c93f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d98fd5640 1 -- 172.21.15.5:0/664497997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 msgr2=0x5620c93e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d98fd5640 1 -- 172.21.15.5:0/664497997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c92db1b0 con 0x5620c93f0a00 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 0x5620c93f2e50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2d84005fd0 tx=0x7f2d8400d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209e4fb6708dd15d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d99fd7640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d800a3d20 0x7f2d800a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2d900cdd00 tx=0x7f2d90091930 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d7f7fe640 1 -- 172.21.15.5:0/664497997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400aa60 con 0x5620c93f0a00 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d7f7fe640 1 -- 172.21.15.5:0/664497997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d840054d0 con 0x5620c93f0a00 2024-04-22T13:35:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.636+0000 7f2d7f7fe640 1 -- 172.21.15.5:0/664497997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84004a90 con 0x5620c93f0a00 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 -- 172.21.15.5:0/664497997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 msgr2=0x5620c93f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 0x5620c93f2e50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2d84005fd0 tx=0x7f2d8400d420 comp rx=0 tx=0).stop 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 -- 172.21.15.5:0/664497997 shutdown_connections 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93f0a00 0x5620c93f2e50 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93e7e80 0x5620c93f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 --2- 172.21.15.5:0/664497997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 -- 172.21.15.5:0/664497997 >> 172.21.15.5:0/664497997 conn(0x5620c93e1ba0 msgr2=0x5620c93e1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 -- 172.21.15.5:0/664497997 shutdown_connections 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 -- 172.21.15.5:0/664497997 wait complete. 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 -- start start 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.637+0000 7f2d9e621740 1 Processor -- start 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c93edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620c93f3970 con 0x5620c93e74d0 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5620c9305500 con 0x5620c93e7e80 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620c93f3af0 con 0x5620c93f0a00 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d9e621740 10 client.0 _ll_register_callbacks cb 0x5620c92d3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58810/0 (socket says 172.21.15.5:58810) 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 -- 172.21.15.5:0/3284164351 learned_addr learned my addr 172.21.15.5:0/3284164351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d997d6640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c93edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 -- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 msgr2=0x5620c93edc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c93edc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 -- 172.21.15.5:0/3284164351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 msgr2=0x5620c93ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 -- 172.21.15.5:0/3284164351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c905e3f0 con 0x5620c93e74d0 2024-04-22T13:35:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f0300 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004770 tx=0x7f2d84005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fe7d8d0bd33f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.638+0000 7f2d7cff9640 1 -- 172.21.15.5:0/3284164351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400dbb0 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d7cff9640 1 -- 172.21.15.5:0/3284164351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f2d8400a860 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d840054d0 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d7cff9640 1 -- 172.21.15.5:0/3284164351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d8400aa60 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 10 client.18510 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620c91287d0 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d7cff9640 1 -- 172.21.15.5:0/3284164351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d840054d0 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d7cff9640 1 -- 172.21.15.5:0/3284164351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2d84027020 con 0x5620c93e74d0 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 10 client.18510 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 10 client.18510 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 20 client.18510 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9400b90 0x5620c9402f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 4 client.18510 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.639+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620c92eb930 con 0x5620c9400b90 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.640+0000 7f2d997d6640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9400b90 0x5620c9402f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.640+0000 7f2d997d6640 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9400b90 0x5620c9402f80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c727eac4feb008b8 server_cookie=bf8e57b9360c19cd in_seq=0 out_seq=0 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.640+0000 7f2d7cff9640 10 client.18510 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d7cff9640 1 -- 172.21.15.5:0/3284164351 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620c92eb930 con 0x5620c9400b90 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d7cff9640 10 client.18510 handle_command_reply: tid=0 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 client.18510 shutdown 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 msgr2=0x5620c93f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f0300 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004770 tx=0x7f2d84005c30 comp rx=0 tx=0).stop 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 shutdown_connections 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c93edc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9400b90 0x5620c9402f80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3284164351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f0300 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d7cff9640 0 client.18510 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 >> 172.21.15.5:0/3284164351 conn(0x5620c93e1ba0 msgr2=0x5620c93e38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 shutdown_connections 2024-04-22T13:35:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 1 -- 172.21.15.5:0/3284164351 wait complete. 2024-04-22T13:35:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.641+0000 7f2d9e621740 20 client.18510 trim_cache size 0 max 16384 2024-04-22T13:35:26.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 Processor -- start 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 -- start start 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 0x5620c93f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c9408670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941adb20 con 0x5620c93e7e80 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941af660 con 0x5620c93e74d0 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.654+0000 7f2d9e621740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941ae1f0 con 0x5620c93f0a00 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 0x5620c93f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 0x5620c93f0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58814/0 (socket says 172.21.15.5:58814) 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 -- 172.21.15.5:0/3342291815 learned_addr learned my addr 172.21.15.5:0/3342291815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c9408670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 -- 172.21.15.5:0/3342291815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 msgr2=0x5620c9408670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c9408670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 -- 172.21.15.5:0/3342291815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 msgr2=0x5620c93efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93efdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 -- 172.21.15.5:0/3342291815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c93e6e90 con 0x5620c93e7e80 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c9408670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d997d6640 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 0x5620c93f0300 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02c170 tx=0x7f2d8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d976d76bac99f8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d7effd640 1 -- 172.21.15.5:0/3342291815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c048050 con 0x5620c93e7e80 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d7effd640 1 -- 172.21.15.5:0/3342291815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c02cc90 con 0x5620c93e7e80 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.655+0000 7f2d7effd640 1 -- 172.21.15.5:0/3342291815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c005410 con 0x5620c93e7e80 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 -- 172.21.15.5:0/3342291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 msgr2=0x5620c93f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 0x5620c93f0300 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02c170 tx=0x7f2d8c009260 comp rx=0 tx=0).stop 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 -- 172.21.15.5:0/3342291815 shutdown_connections 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e7e80 0x5620c93f0300 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c9408670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 --2- 172.21.15.5:0/3342291815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e74d0 0x5620c93efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 -- 172.21.15.5:0/3342291815 >> 172.21.15.5:0/3342291815 conn(0x5620c93e1ba0 msgr2=0x5620c93e8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 -- 172.21.15.5:0/3342291815 shutdown_connections 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 -- 172.21.15.5:0/3342291815 wait complete. 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 -- start start 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 Processor -- start 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.656+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c945eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941aef90 con 0x5620c93e74d0 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941ae8c0 con 0x5620c93e7e80 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941abb20 con 0x5620c93f0a00 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 10 client.0 _ll_register_callbacks cb 0x7f2d84012780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58828/0 (socket says 172.21.15.5:58828) 2024-04-22T13:35:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 -- 172.21.15.5:0/2548728455 learned_addr learned my addr 172.21.15.5:0/2548728455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d997d6640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c945eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 -- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 msgr2=0x5620c945eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c945eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 -- 172.21.15.5:0/2548728455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 msgr2=0x5620c93f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 -- 172.21.15.5:0/2548728455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c945f460 con 0x5620c93e74d0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d997d6640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d98fd5640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f3f90 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2d84029550 tx=0x7f2d84005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4a599f235f947 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d7cff9640 1 -- 172.21.15.5:0/2548728455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84036040 con 0x5620c93e74d0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c02cc90 con 0x5620c93e74d0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d7cff9640 1 -- 172.21.15.5:0/2548728455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f2d8400ac00 con 0x5620c93e74d0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d7cff9640 1 -- 172.21.15.5:0/2548728455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d8400dac0 con 0x5620c93e74d0 2024-04-22T13:35:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 10 client.18516 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.657+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620c9466780 con 0x5620c93e74d0 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d7cff9640 1 -- 172.21.15.5:0/2548728455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c02cc90 con 0x5620c93e74d0 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d7cff9640 1 -- 172.21.15.5:0/2548728455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2d84027040 con 0x5620c93e74d0 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d9e621740 10 client.18516 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d9e621740 10 client.18516 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d9e621740 20 client.18516 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d9e621740 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9466f00 0x5620c9469320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d9e621740 4 client.18516 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620c9359c10 con 0x5620c9466f00 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.658+0000 7f2d997d6640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9466f00 0x5620c9469320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d997d6640 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9466f00 0x5620c9469320 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f494b9e7ce234214 server_cookie=9bcfd399635d223d in_seq=0 out_seq=0 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d7cff9640 10 client.18516 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d7cff9640 1 -- 172.21.15.5:0/2548728455 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620c9359c10 con 0x5620c9466f00 2024-04-22T13:35:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d7cff9640 10 client.18516 handle_command_reply: tid=0 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 client.18516 shutdown 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 msgr2=0x5620c93f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f3f90 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2d84029550 tx=0x7f2d84005f50 comp rx=0 tx=0).stop 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 shutdown_connections 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c93f0a00 0x5620c945eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5620c93e7e80 0x5620c93f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x5620c9466f00 0x5620c9469320 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 --2- 172.21.15.5:0/2548728455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c93e74d0 0x5620c93f3f90 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d7cff9640 0 client.18516 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 >> 172.21.15.5:0/2548728455 conn(0x5620c93e1ba0 msgr2=0x5620c93e2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 shutdown_connections 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 1 -- 172.21.15.5:0/2548728455 wait complete. 2024-04-22T13:35:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.659+0000 7f2d9e621740 20 client.18516 trim_cache size 0 max 16384 2024-04-22T13:35:26.663 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d800a3d20 msgr2=0x7f2d800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d800a3d20 0x7f2d800a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2d900cdd00 tx=0x7f2d90091930 comp rx=0 tx=0).stop 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 msgr2=0x7f2d9419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 0x7f2d9419d450 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000da40 tx=0x7f2d9000df00 comp rx=0 tx=0).stop 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 shutdown_connections 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9419cfa0 0x7f2d9419d450 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2d800a3d20 0x7f2d800a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94106160 0x7f2d94198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 --2- 172.21.15.5:0/520568516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101090 0x7f2d941986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 >> 172.21.15.5:0/520568516 conn(0x7f2d94078000 msgr2=0x7f2d940ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 shutdown_connections 2024-04-22T13:35:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.660+0000 7f2d9ba3a640 1 -- 172.21.15.5:0/520568516 wait complete. 2024-04-22T13:35:26.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 -- 172.21.15.5:0/2786017397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 msgr2=0x7f08241035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 --2- 172.21.15.5:0/2786017397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f08241035a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f0814009900 tx=0x7f081402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 -- 172.21.15.5:0/2786017397 shutdown_connections 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 --2- 172.21.15.5:0/2786017397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082410c500 0x7f082410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 --2- 172.21.15.5:0/2786017397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0824103ae0 0x7f082410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 --2- 172.21.15.5:0/2786017397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f08241035a0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 -- 172.21.15.5:0/2786017397 >> 172.21.15.5:0/2786017397 conn(0x7f0824078000 msgr2=0x7f0824078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 -- 172.21.15.5:0/2786017397 shutdown_connections 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.775+0000 7f0829567640 1 -- 172.21.15.5:0/2786017397 wait complete. 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 Processor -- start 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 -- start start 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f0824198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0824103ae0 0x7f0824198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082410c500 0x7f082419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0824112190 con 0x7f08241031c0 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0824112010 con 0x7f0824103ae0 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0829567640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0824112310 con 0x7f082410c500 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0822ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f0824198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0822ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f0824198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58842/0 (socket says 172.21.15.5:58842) 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f0822ffd640 1 -- 172.21.15.5:0/1314078628 learned_addr learned my addr 172.21.15.5:0/1314078628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.776+0000 7f08227fc640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0824103ae0 0x7f0824198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f08237fe640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082410c500 0x7f082419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0822ffd640 1 -- 172.21.15.5:0/1314078628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082410c500 msgr2=0x7f082419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0822ffd640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082410c500 0x7f082419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0822ffd640 1 -- 172.21.15.5:0/1314078628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0824103ae0 msgr2=0x7f0824198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0822ffd640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0824103ae0 0x7f0824198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0822ffd640 1 -- 172.21.15.5:0/1314078628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082419d660 con 0x7f08241031c0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0822ffd640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f0824198670 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0814005940 tx=0x7f08140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f081bfff640 1 -- 172.21.15.5:0/1314078628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814043020 con 0x7f08241031c0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f081bfff640 1 -- 172.21.15.5:0/1314078628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f081402dba0 con 0x7f08241031c0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082419d8f0 con 0x7f08241031c0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f081bfff640 1 -- 172.21.15.5:0/1314078628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814003850 con 0x7f08241031c0 2024-04-22T13:35:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.777+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08241a51e0 con 0x7f08241031c0 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.779+0000 7f081bfff640 1 -- 172.21.15.5:0/1314078628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0814046020 con 0x7f08241031c0 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 Processor -- start 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 -- start start 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84998b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654f83fe1f0 con 0x5654f84a2a00 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5654f839b4b0 con 0x5654f8499e80 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f082c23a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654f83fa1b0 con 0x5654f84994d0 2024-04-22T13:35:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f08227fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f08227fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58850/0 (socket says 172.21.15.5:58850) 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f08227fc640 1 -- 172.21.15.5:0/1643542495 learned_addr learned my addr 172.21.15.5:0/1643542495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f08237fe640 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f081bfff640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08100a3d20 0x7f08100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.780+0000 7f0822ffd640 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08237fe640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08100a3d20 0x7f08100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08227fc640 1 -- 172.21.15.5:0/1643542495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 msgr2=0x5654f84998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08227fc640 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08227fc640 1 -- 172.21.15.5:0/1643542495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 msgr2=0x5654f84a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08227fc640 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08227fc640 1 -- 172.21.15.5:0/1643542495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f838d1b0 con 0x5654f84a2a00 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f081bfff640 1 -- 172.21.15.5:0/1314078628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f08140f2770 con 0x7f08241031c0 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08227fc640 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a4e50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f080c00aca0 tx=0x7f080c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bbcd499658acfd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08197fa640 1 -- 172.21.15.5:0/1643542495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c01d070 con 0x5654f84a2a00 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08237fe640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08100a3d20 0x7f08100a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f081c005fd0 tx=0x7f081c004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08197fa640 1 -- 172.21.15.5:0/1643542495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080c018af0 con 0x5654f84a2a00 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.781+0000 7f08197fa640 1 -- 172.21.15.5:0/1643542495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c010b80 con 0x5654f84a2a00 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 -- 172.21.15.5:0/1643542495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 msgr2=0x5654f84a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a4e50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f080c00aca0 tx=0x7f080c00f790 comp rx=0 tx=0).stop 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 -- 172.21.15.5:0/1643542495 shutdown_connections 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a4e50 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 --2- 172.21.15.5:0/1643542495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 -- 172.21.15.5:0/1643542495 >> 172.21.15.5:0/1643542495 conn(0x5654f8493ba0 msgr2=0x5654f8493f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 -- 172.21.15.5:0/1643542495 shutdown_connections 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 -- 172.21.15.5:0/1643542495 wait complete. 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 -- start start 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.782+0000 7f082c23a740 1 Processor -- start 2024-04-22T13:35:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 0x5654f849f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654f84a5970 con 0x5654f8499e80 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5654f83b7500 con 0x5654f84a2a00 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654f84a5af0 con 0x5654f84994d0 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 10 client.0 _ll_register_callbacks cb 0x5654f8385b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 0x5654f849f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 0x5654f849f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58864/0 (socket says 172.21.15.5:58864) 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 -- 172.21.15.5:0/538694894 learned_addr learned my addr 172.21.15.5:0/538694894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f0822ffd640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 -- 172.21.15.5:0/538694894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 msgr2=0x5654f84a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08237fe640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 -- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 msgr2=0x5654f849fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 -- 172.21.15.5:0/538694894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f81103f0 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08237fe640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f08227fc640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 0x5654f849f680 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f080c01b040 tx=0x7f080c00a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61773ed831dc94e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f080a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f080a7fc640 1 -- 172.21.15.5:0/538694894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c01d050 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f080a7fc640 1 -- 172.21.15.5:0/538694894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f080c0108c0 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.783+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f080c018af0 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f080a7fc640 1 -- 172.21.15.5:0/538694894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f080c032050 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 10 client.18528 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f81da7d0 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f080a7fc640 1 -- 172.21.15.5:0/538694894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080c018af0 con 0x5654f8499e80 2024-04-22T13:35:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f080a7fc640 1 -- 172.21.15.5:0/538694894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f080c0046c0 con 0x5654f8499e80 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 10 client.18528 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 10 client.18528 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 20 client.18528 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f84b2b30 0x5654f84b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 4 client.18528 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f839d930 con 0x5654f84b2b30 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.784+0000 7f0822ffd640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f84b2b30 0x5654f84b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.785+0000 7f0822ffd640 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f84b2b30 0x5654f84b4f20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e40c0a1a14c5e94 server_cookie=fb565ffbda1986ca in_seq=0 out_seq=0 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.785+0000 7f080a7fc640 10 client.18528 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f080a7fc640 1 -- 172.21.15.5:0/538694894 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654f839d930 con 0x5654f84b2b30 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f080a7fc640 10 client.18528 handle_command_reply: tid=0 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 client.18528 shutdown 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 msgr2=0x5654f849f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 0x5654f849f680 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f080c01b040 tx=0x7f080c00a970 comp rx=0 tx=0).stop 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 shutdown_connections 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f84b2b30 0x5654f84b4f20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f8499e80 0x5654f849f680 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f080a7fc640 0 client.18528 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 --2- 172.21.15.5:0/538694894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 >> 172.21.15.5:0/538694894 conn(0x5654f8493ba0 msgr2=0x5654f84958f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 shutdown_connections 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 1 -- 172.21.15.5:0/538694894 wait complete. 2024-04-22T13:35:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.786+0000 7f082c23a740 20 client.18528 trim_cache size 0 max 16384 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.799+0000 7f082c23a740 1 Processor -- start 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.799+0000 7f082c23a740 1 -- start start 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f082c23a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f082c23a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f082c23a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f082c23a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08241adaa0 con 0x5654f84a2a00 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f082c23a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08241af5e0 con 0x5654f8499e80 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f082c23a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08241ae170 con 0x5654f84994d0 2024-04-22T13:35:26.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58872/0 (socket says 172.21.15.5:58872) 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 -- 172.21.15.5:0/3909676843 learned_addr learned my addr 172.21.15.5:0/3909676843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f0822ffd640 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 -- 172.21.15.5:0/3909676843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 msgr2=0x5654f84ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08227fc640 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 -- 172.21.15.5:0/3909676843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 msgr2=0x5654f84956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f08237fe640 1 -- 172.21.15.5:0/3909676843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f8494dd0 con 0x5654f84a2a00 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.800+0000 7f0822ffd640 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f08237fe640 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a1db0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f081c009eb0 tx=0x7f081c009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5966ea3c819e0d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f0818ff9640 1 -- 172.21.15.5:0/3909676843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c011470 con 0x5654f84a2a00 2024-04-22T13:35:26.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f0818ff9640 1 -- 172.21.15.5:0/3909676843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f081c028070 con 0x5654f84a2a00 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f0818ff9640 1 -- 172.21.15.5:0/3909676843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c023900 con 0x5654f84a2a00 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 -- 172.21.15.5:0/3909676843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 msgr2=0x5654f84a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a1db0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f081c009eb0 tx=0x7f081c009a10 comp rx=0 tx=0).stop 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 -- 172.21.15.5:0/3909676843 shutdown_connections 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84a2a00 0x5654f84a1db0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f8499e80 0x5654f84956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 --2- 172.21.15.5:0/3909676843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f84994d0 0x5654f84ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 -- 172.21.15.5:0/3909676843 >> 172.21.15.5:0/3909676843 conn(0x5654f8493ba0 msgr2=0x5654f8494fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 -- 172.21.15.5:0/3909676843 shutdown_connections 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.801+0000 7f082c23a740 1 -- 172.21.15.5:0/3909676843 wait complete. 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 -- start start 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 Processor -- start 2024-04-22T13:35:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 0x5654f849c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f8499e80 0x5654f849c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08241aef10 con 0x5654f84994d0 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f08241ae840 con 0x5654f84a2a00 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08241abaa0 con 0x5654f8499e80 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f082c23a740 10 client.0 _ll_register_callbacks cb 0x7f080c00ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 0x5654f849c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 0x5654f849c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58876/0 (socket says 172.21.15.5:58876) 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 -- 172.21.15.5:0/642946953 learned_addr learned my addr 172.21.15.5:0/642946953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f08227fc640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f8499e80 0x5654f849c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 -- 172.21.15.5:0/642946953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f8499e80 msgr2=0x5654f849c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f8499e80 0x5654f849c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 -- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 msgr2=0x5654f849cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f0822ffd640 1 -- 172.21.15.5:0/642946953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f8517df0 con 0x5654f84994d0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.802+0000 7f08227fc640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f8499e80 0x5654f849c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f0822ffd640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 0x5654f849c190 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f08140be7b0 tx=0x7f08140f4c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e0d6d66aef9fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f080a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f080a7fc640 1 -- 172.21.15.5:0/642946953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140c7490 con 0x5654f84994d0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f080a7fc640 1 -- 172.21.15.5:0/642946953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f081402dd50 con 0x5654f84994d0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f081c028070 con 0x5654f84994d0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f080a7fc640 1 -- 172.21.15.5:0/642946953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0814050d90 con 0x5654f84994d0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 10 client.18534 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f8518f80 con 0x5654f84994d0 2024-04-22T13:35:26.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f080a7fc640 1 -- 172.21.15.5:0/642946953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f081c028070 con 0x5654f84994d0 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f080a7fc640 1 -- 172.21.15.5:0/642946953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f08140bae90 con 0x5654f84994d0 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 10 client.18534 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 10 client.18534 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 20 client.18534 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f8519650 0x5654f851baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 4 client.18534 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654f840bc10 con 0x5654f8519650 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.803+0000 7f08227fc640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f8519650 0x5654f851baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f080a7fc640 10 client.18534 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f08227fc640 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f8519650 0x5654f851baa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc59502825d9ba31 server_cookie=c3b4542559329aed in_seq=0 out_seq=0 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f080a7fc640 1 -- 172.21.15.5:0/642946953 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654f840bc10 con 0x5654f8519650 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f080a7fc640 10 client.18534 handle_command_reply: tid=0 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 client.18534 shutdown 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 msgr2=0x5654f849c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 0x5654f849c190 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f08140be7b0 tx=0x7f08140f4c80 comp rx=0 tx=0).stop 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 shutdown_connections 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5654f8519650 0x5654f851baa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654f84a2a00 0x5654f849cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f080a7fc640 0 client.18534 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654f8499e80 0x5654f849c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 --2- 172.21.15.5:0/642946953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654f84994d0 0x5654f849c190 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 >> 172.21.15.5:0/642946953 conn(0x5654f8493ba0 msgr2=0x5654f8495850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 shutdown_connections 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 1 -- 172.21.15.5:0/642946953 wait complete. 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.804+0000 7f082c23a740 20 client.18534 trim_cache size 0 max 16384 2024-04-22T13:35:26.808 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08100a3d20 msgr2=0x7f08100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08100a3d20 0x7f08100a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f081c005fd0 tx=0x7f081c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 msgr2=0x7f0824198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f0824198670 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0814005940 tx=0x7f08140045e0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 shutdown_connections 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f08100a3d20 0x7f08100a61e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082410c500 0x7f082419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0824103ae0 0x7f0824198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 --2- 172.21.15.5:0/1314078628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08241031c0 0x7f0824198670 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 >> 172.21.15.5:0/1314078628 conn(0x7f0824078000 msgr2=0x7f08240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 shutdown_connections 2024-04-22T13:35:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.805+0000 7f0829567640 1 -- 172.21.15.5:0/1314078628 wait complete. 2024-04-22T13:35:26.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:26 smithi005 ceph-mon[28006]: pgmap v256: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:35:26.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:26 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1514685787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:26.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 -- 172.21.15.5:0/3690072383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 msgr2=0x7fed84101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 --2- 172.21.15.5:0/3690072383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84101470 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fed74009900 tx=0x7fed7402bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 -- 172.21.15.5:0/3690072383 shutdown_connections 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 --2- 172.21.15.5:0/3690072383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 0x7fed8410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 --2- 172.21.15.5:0/3690072383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed84101a40 0x7fed84105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 --2- 172.21.15.5:0/3690072383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84101470 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 -- 172.21.15.5:0/3690072383 >> 172.21.15.5:0/3690072383 conn(0x7fed84078000 msgr2=0x7fed84078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 -- 172.21.15.5:0/3690072383 shutdown_connections 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.921+0000 7fed89a31640 1 -- 172.21.15.5:0/3690072383 wait complete. 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 Processor -- start 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 -- start start 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 0x7fed84197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed8419bf00 0x7fed8419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed841110a0 con 0x7fed84101090 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed84110f20 con 0x7fed8419bf00 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed89a31640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111220 con 0x7fed84106160 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58890/0 (socket says 172.21.15.5:58890) 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed82ffd640 1 -- 172.21.15.5:0/646896255 learned_addr learned my addr 172.21.15.5:0/646896255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.922+0000 7fed827fc640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 0x7fed84197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed82ffd640 1 -- 172.21.15.5:0/646896255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 msgr2=0x7fed84197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed82ffd640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 0x7fed84197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed82ffd640 1 -- 172.21.15.5:0/646896255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed8419bf00 msgr2=0x7fed8419c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed82ffd640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed8419bf00 0x7fed8419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed82ffd640 1 -- 172.21.15.5:0/646896255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8419c9f0 con 0x7fed84101090 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed827fc640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 0x7fed84197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed82ffd640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84197640 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fed74005940 tx=0x7fed7402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed88a2f640 1 -- 172.21.15.5:0/646896255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74043020 con 0x7fed84101090 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed88a2f640 1 -- 172.21.15.5:0/646896255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7402ca70 con 0x7fed84101090 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8419cc80 con 0x7fed84101090 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed88a2f640 1 -- 172.21.15.5:0/646896255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740039f0 con 0x7fed84101090 2024-04-22T13:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.923+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed841a44a0 con 0x7fed84101090 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 Processor -- start 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed88a2f640 1 -- 172.21.15.5:0/646896255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fed74046020 con 0x7fed84101090 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 -- start start 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 0x55c398f56370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f63a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f63f70 0x55c398f66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c398ebb1f0 con 0x55c398f55f90 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c398e584b0 con 0x55c398f56940 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.926+0000 7fed8c629740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c398eb71b0 con 0x55c398f63f70 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 0x55c398f56370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 0x55c398f56370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58894/0 (socket says 172.21.15.5:58894) 2024-04-22T13:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 -- 172.21.15.5:0/2021313429 learned_addr learned my addr 172.21.15.5:0/2021313429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed82ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f63a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed827fc640 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f63f70 0x55c398f66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 -- 172.21.15.5:0/2021313429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f63f70 msgr2=0x55c398f66360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f63f70 0x55c398f66360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 -- 172.21.15.5:0/2021313429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 msgr2=0x55c398f63a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f63a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 -- 172.21.15.5:0/2021313429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c398e4a1b0 con 0x55c398f55f90 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed88a2f640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed837fe640 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 0x55c398f56370 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009e10 tx=0x7fed7c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b41f39800b98c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed82ffd640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed88a2f640 1 -- 172.21.15.5:0/646896255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fed740f2910 con 0x7fed84101090 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed79ffb640 1 -- 172.21.15.5:0/2021313429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c004d20 con 0x55c398f55f90 2024-04-22T13:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed79ffb640 1 -- 172.21.15.5:0/2021313429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c00ee70 con 0x55c398f55f90 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.927+0000 7fed79ffb640 1 -- 172.21.15.5:0/2021313429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c010760 con 0x55c398f55f90 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 -- 172.21.15.5:0/2021313429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 msgr2=0x55c398f56370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed82ffd640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed700a3d20 0x7fed700a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55c398f57380 tx=0x7fed740df040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 0x55c398f56370 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009e10 tx=0x7fed7c00e720 comp rx=0 tx=0).stop 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 -- 172.21.15.5:0/2021313429 shutdown_connections 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f63f70 0x55c398f66360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f63a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 --2- 172.21.15.5:0/2021313429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f55f90 0x55c398f56370 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 -- 172.21.15.5:0/2021313429 >> 172.21.15.5:0/2021313429 conn(0x55c398f50ba0 msgr2=0x55c398f50f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 -- 172.21.15.5:0/2021313429 shutdown_connections 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 -- 172.21.15.5:0/2021313429 wait complete. 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.928+0000 7fed8c629740 1 -- start start 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 Processor -- start 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f60bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f61100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c398f66d60 con 0x55c398f63f70 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c398e74500 con 0x55c398f56940 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c398f66ee0 con 0x55c398f55f90 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed8c629740 10 client.0 _ll_register_callbacks cb 0x55c398e42b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f61100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f61100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58902/0 (socket says 172.21.15.5:58902) 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 -- 172.21.15.5:0/396804477 learned_addr learned my addr 172.21.15.5:0/396804477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed827fc640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f60bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 -- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 msgr2=0x55c398f63830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f63830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 -- 172.21.15.5:0/396804477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 msgr2=0x55c398f60bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f60bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.929+0000 7fed837fe640 1 -- 172.21.15.5:0/396804477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c398bcd3f0 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed837fe640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f61100 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005b40 tx=0x7fed7c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ccef2103cbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed62ffd640 1 -- 172.21.15.5:0/396804477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c02b060 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed62ffd640 1 -- 172.21.15.5:0/396804477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fed7c003970 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7c00ee70 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed62ffd640 1 -- 172.21.15.5:0/396804477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed7c011e20 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 10 client.18546 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c398c977d0 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed62ffd640 1 -- 172.21.15.5:0/396804477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c00ee70 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed62ffd640 1 -- 172.21.15.5:0/396804477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed7c0110c0 con 0x55c398f63f70 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 10 client.18546 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 10 client.18546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 20 client.18546 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398f73f90 0x55c398f76380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 4 client.18546 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.930+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c398e5a930 con 0x55c398f73f90 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.931+0000 7fed827fc640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398f73f90 0x55c398f76380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.931+0000 7fed62ffd640 10 client.18546 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.931+0000 7fed827fc640 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398f73f90 0x55c398f76380 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=814f5fbcfef1b3d1 server_cookie=837d3de098e016d0 in_seq=0 out_seq=0 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed62ffd640 1 -- 172.21.15.5:0/396804477 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c398e5a930 con 0x55c398f73f90 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed62ffd640 10 client.18546 handle_command_reply: tid=0 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 client.18546 shutdown 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 msgr2=0x55c398f61100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f61100 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005b40 tx=0x7fed7c005a40 comp rx=0 tx=0).stop 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 shutdown_connections 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398f73f90 0x55c398f76380 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f61100 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f56940 0x55c398f60bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed62ffd640 0 client.18546 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 --2- 172.21.15.5:0/396804477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 >> 172.21.15.5:0/396804477 conn(0x55c398f50ba0 msgr2=0x55c398f528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 shutdown_connections 2024-04-22T13:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 1 -- 172.21.15.5:0/396804477 wait complete. 2024-04-22T13:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.932+0000 7fed8c629740 20 client.18546 trim_cache size 0 max 16384 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 Processor -- start 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 -- start start 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f55f90 0x55c398f6fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f56940 0x55c398f5fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed841ab940 con 0x55c398f63f70 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed841ad480 con 0x55c398f55f90 2024-04-22T13:35:26.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed8c629740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed841ac010 con 0x55c398f56940 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58916/0 (socket says 172.21.15.5:58916) 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 -- 172.21.15.5:0/4062112970 learned_addr learned my addr 172.21.15.5:0/4062112970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 -- 172.21.15.5:0/4062112970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f56940 msgr2=0x55c398f5fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f56940 0x55c398f5fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 -- 172.21.15.5:0/4062112970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f55f90 msgr2=0x55c398f6fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f55f90 0x55c398f6fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 -- 172.21.15.5:0/4062112970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c398f51770 con 0x55c398f63f70 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed837fe640 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f632f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009ac0 tx=0x7fed7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60cf3dec08c6364 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed797fa640 1 -- 172.21.15.5:0/4062112970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c02b070 con 0x55c398f63f70 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed797fa640 1 -- 172.21.15.5:0/4062112970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c00ee70 con 0x55c398f63f70 2024-04-22T13:35:26.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.946+0000 7fed797fa640 1 -- 172.21.15.5:0/4062112970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c005050 con 0x55c398f63f70 2024-04-22T13:35:26.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 -- 172.21.15.5:0/4062112970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 msgr2=0x55c398f632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f632f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009ac0 tx=0x7fed7c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 -- 172.21.15.5:0/4062112970 shutdown_connections 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f63f70 0x55c398f632f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f56940 0x55c398f5fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 --2- 172.21.15.5:0/4062112970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f55f90 0x55c398f6fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 -- 172.21.15.5:0/4062112970 >> 172.21.15.5:0/4062112970 conn(0x55c398f50ba0 msgr2=0x55c398f5f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 -- 172.21.15.5:0/4062112970 shutdown_connections 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 -- 172.21.15.5:0/4062112970 wait complete. 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.947+0000 7fed8c629740 1 -- start start 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 Processor -- start 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 0x55c398f6c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f63f70 0x55c398f583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed841acdb0 con 0x55c398f56940 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed841ac6e0 con 0x55c398f63f70 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed841a9940 con 0x55c398f55f90 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed8c629740 10 client.0 _ll_register_callbacks cb 0x7fed7c004e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 0x55c398f6c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 0x55c398f6c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58920/0 (socket says 172.21.15.5:58920) 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 -- 172.21.15.5:0/954837304 learned_addr learned my addr 172.21.15.5:0/954837304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed837fe640 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 -- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 msgr2=0x55c398f6c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f6c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 -- 172.21.15.5:0/954837304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f63f70 msgr2=0x55c398f583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f63f70 0x55c398f583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 -- 172.21.15.5:0/954837304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c398f58b00 con 0x55c398f56940 2024-04-22T13:35:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed827fc640 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 0x55c398f6c6e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x55c398f51d30 tx=0x7fed6c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660050f7a5aba5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.948+0000 7fed62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed62ffd640 1 -- 172.21.15.5:0/954837304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c012e60 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed62ffd640 1 -- 172.21.15.5:0/954837304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fed6c004230 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7c00ee70 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed62ffd640 1 -- 172.21.15.5:0/954837304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed6c02e050 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 10 client.18552 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c398f62ee0 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed62ffd640 1 -- 172.21.15.5:0/954837304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c00ee70 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed62ffd640 1 -- 172.21.15.5:0/954837304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed6c0039d0 con 0x55c398f56940 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 10 client.18552 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 10 client.18552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 20 client.18552 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398fda7f0 0x55c398fdcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 4 client.18552 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.949+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c398f61490 con 0x55c398fda7f0 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed837fe640 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398fda7f0 0x55c398fdcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed837fe640 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398fda7f0 0x55c398fdcc10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd126b7214527c53 server_cookie=7b89051a5339d01f in_seq=0 out_seq=0 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed62ffd640 10 client.18552 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed62ffd640 1 -- 172.21.15.5:0/954837304 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c398f61490 con 0x55c398fda7f0 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed62ffd640 10 client.18552 handle_command_reply: tid=0 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 client.18552 shutdown 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 msgr2=0x55c398f6c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 0x55c398f6c6e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x55c398f51d30 tx=0x7fed6c012820 comp rx=0 tx=0).stop 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 shutdown_connections 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55c398fda7f0 0x55c398fdcc10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c398f56940 0x55c398f6c6e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c398f63f70 0x55c398f583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed62ffd640 0 client.18552 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 --2- 172.21.15.5:0/954837304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c398f55f90 0x55c398f6c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 >> 172.21.15.5:0/954837304 conn(0x55c398f50ba0 msgr2=0x55c398f51ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 shutdown_connections 2024-04-22T13:35:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.950+0000 7fed8c629740 1 -- 172.21.15.5:0/954837304 wait complete. 2024-04-22T13:35:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed8c629740 20 client.18552 trim_cache size 0 max 16384 2024-04-22T13:35:26.953 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed700a3d20 msgr2=0x7fed700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed700a3d20 0x7fed700a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55c398f57380 tx=0x7fed740df040 comp rx=0 tx=0).stop 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 msgr2=0x7fed84197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84197640 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fed74005940 tx=0x7fed7402c5f0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 shutdown_connections 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed8419bf00 0x7fed8419c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed84106160 0x7fed84197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 --2- 172.21.15.5:0/646896255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed84101090 0x7fed84197640 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 >> 172.21.15.5:0/646896255 conn(0x7fed84078000 msgr2=0x7fed840ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 shutdown_connections 2024-04-22T13:35:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:26.951+0000 7fed89a31640 1 -- 172.21.15.5:0/646896255 wait complete. 2024-04-22T13:35:26.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:35:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:26 smithi049 ceph-mon[31379]: pgmap v256: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:35:27.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:26 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1514685787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:27.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:26 smithi107 ceph-mon[31859]: pgmap v256: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:35:27.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:26 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1514685787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- 172.21.15.5:0/3455645759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 msgr2=0x7fcea410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- 172.21.15.5:0/3455645759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea410bfc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fce94009900 tx=0x7fce9402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- 172.21.15.5:0/3455645759 shutdown_connections 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- 172.21.15.5:0/3455645759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea410bfc0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- 172.21.15.5:0/3455645759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 0x7fcea410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- 172.21.15.5:0/3455645759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea41031c0 0x7fcea41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- 172.21.15.5:0/3455645759 >> 172.21.15.5:0/3455645759 conn(0x7fcea4078000 msgr2=0x7fcea4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- 172.21.15.5:0/3455645759 shutdown_connections 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- 172.21.15.5:0/3455645759 wait complete. 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 Processor -- start 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- start start 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea41031c0 0x7fcea41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 0x7fcea419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4112170 con 0x7fcea4103ae0 2024-04-22T13:35:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4111ff0 con 0x7fcea41031c0 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.066+0000 7fceabc3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41122f0 con 0x7fcea410c500 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea4198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58946/0 (socket says 172.21.15.5:58946) 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 -- 172.21.15.5:0/2311234785 learned_addr learned my addr 172.21.15.5:0/2311234785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea99d6640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea41031c0 0x7fcea41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 0x7fcea419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 -- 172.21.15.5:0/2311234785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 msgr2=0x7fcea419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 0x7fcea419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 -- 172.21.15.5:0/2311234785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea41031c0 msgr2=0x7fcea41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea41031c0 0x7fcea41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 -- 172.21.15.5:0/2311234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea419d6c0 con 0x7fcea4103ae0 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 0x7fcea419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fcea91d5640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea4198c10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fce94005940 tx=0x7fce9402d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fce9affd640 1 -- 172.21.15.5:0/2311234785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94043020 con 0x7fcea4103ae0 2024-04-22T13:35:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fce9affd640 1 -- 172.21.15.5:0/2311234785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9402dc90 con 0x7fcea4103ae0 2024-04-22T13:35:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea419d950 con 0x7fcea4103ae0 2024-04-22T13:35:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fce9affd640 1 -- 172.21.15.5:0/2311234785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94003ab0 con 0x7fcea4103ae0 2024-04-22T13:35:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.067+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41a5240 con 0x7fcea4103ae0 2024-04-22T13:35:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.069+0000 7fce9affd640 1 -- 172.21.15.5:0/2311234785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fce94046020 con 0x7fcea4103ae0 2024-04-22T13:35:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 Processor -- start 2024-04-22T13:35:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 -- start start 2024-04-22T13:35:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ab2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118abbe60 0x557118abc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557118a171f0 con 0x557118ab2d60 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571189b44b0 con 0x557118ab23b0 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fceae822740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557118a131b0 con 0x557118abbe60 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fcea99d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fce9affd640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fce900a3da0 0x7fce900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fcea99d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58948/0 (socket says 172.21.15.5:58948) 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fcea99d6640 1 -- 172.21.15.5:0/1022245170 learned_addr learned my addr 172.21.15.5:0/1022245170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.070+0000 7fcea91d5640 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118abbe60 0x557118abc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ab2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fce900a3da0 0x7fce900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fce9affd640 1 -- 172.21.15.5:0/2311234785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fce940f27e0 con 0x7fcea4103ae0 2024-04-22T13:35:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fcea99d6640 1 -- 172.21.15.5:0/1022245170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118abbe60 msgr2=0x557118abc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fcea99d6640 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118abbe60 0x557118abc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fcea99d6640 1 -- 172.21.15.5:0/1022245170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 msgr2=0x557118ab2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fcea99d6640 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ab2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fcea99d6640 1 -- 172.21.15.5:0/1022245170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571189a61b0 con 0x557118ab2d60 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fcea99d6640 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abb890 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fcea000bd10 tx=0x7fcea00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22ecf7575fae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fce900a3da0 0x7fce900a6260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007310 tx=0x7fce9c0072a0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fce8ffff640 1 -- 172.21.15.5:0/1022245170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0004030 con 0x557118ab2d60 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fce8ffff640 1 -- 172.21.15.5:0/1022245170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea00035f0 con 0x557118ab2d60 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.071+0000 7fce8ffff640 1 -- 172.21.15.5:0/1022245170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0011770 con 0x557118ab2d60 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 -- 172.21.15.5:0/1022245170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 msgr2=0x557118abb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abb890 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fcea000bd10 tx=0x7fcea00107a0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 -- 172.21.15.5:0/1022245170 shutdown_connections 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abb890 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118abbe60 0x557118abc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 --2- 172.21.15.5:0/1022245170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ab2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 -- 172.21.15.5:0/1022245170 >> 172.21.15.5:0/1022245170 conn(0x557118aacba0 msgr2=0x557118aacf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 -- 172.21.15.5:0/1022245170 shutdown_connections 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 -- 172.21.15.5:0/1022245170 wait complete. 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 -- start start 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.072+0000 7fceae822740 1 Processor -- start 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab23b0 0x557118ac2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118abfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557118ac2da0 con 0x557118ab2d60 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5571189d0500 con 0x557118abbe60 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557118ac2f20 con 0x557118ab23b0 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceae822740 10 client.0 _ll_register_callbacks cb 0x55711899eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58956/0 (socket says 172.21.15.5:58956) 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 -- 172.21.15.5:0/3138959682 learned_addr learned my addr 172.21.15.5:0/3138959682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea99d6640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab23b0 0x557118ac2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 -- 172.21.15.5:0/3138959682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab23b0 msgr2=0x557118ac2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab23b0 0x557118ac2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118abfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 -- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 msgr2=0x557118abfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118abfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 -- 172.21.15.5:0/3138959682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571187293f0 con 0x557118ab2d60 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.073+0000 7fcea91d5640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abf5e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fce940bb260 tx=0x7fce94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a37afea531f9ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fce8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fce8d7fa640 1 -- 172.21.15.5:0/3138959682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce940c2c40 con 0x557118ab2d60 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fce8d7fa640 1 -- 172.21.15.5:0/3138959682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fce940a90b0 con 0x557118ab2d60 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea00035f0 con 0x557118ab2d60 2024-04-22T13:35:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fce8d7fa640 1 -- 172.21.15.5:0/3138959682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce940c7500 con 0x557118ab2d60 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 10 client.18564 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571187f37d0 con 0x557118ab2d60 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fce8d7fa640 1 -- 172.21.15.5:0/3138959682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce940dc070 con 0x557118ab2d60 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fce8d7fa640 1 -- 172.21.15.5:0/3138959682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fce940c7100 con 0x557118ab2d60 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 10 client.18564 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 10 client.18564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 20 client.18564 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118ad0080 0x557118ad2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 4 client.18564 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.074+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571189b6930 con 0x557118ad0080 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.075+0000 7fcea99d6640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118ad0080 0x557118ad2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.075+0000 7fce8d7fa640 10 client.18564 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.075+0000 7fcea99d6640 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118ad0080 0x557118ad2470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2544cca2fad5b18 server_cookie=d7cbe5e737995943 in_seq=0 out_seq=0 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fce8d7fa640 1 -- 172.21.15.5:0/3138959682 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571189b6930 con 0x557118ad0080 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fce8d7fa640 10 client.18564 handle_command_reply: tid=0 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 client.18564 shutdown 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 msgr2=0x557118abf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abf5e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fce940bb260 tx=0x7fce94000bc0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 shutdown_connections 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118ad0080 0x557118ad2470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab2d60 0x557118abf5e0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118abfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 --2- 172.21.15.5:0/3138959682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab23b0 0x557118ac2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fce8d7fa640 0 client.18564 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 >> 172.21.15.5:0/3138959682 conn(0x557118aacba0 msgr2=0x557118aae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 shutdown_connections 2024-04-22T13:35:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 1 -- 172.21.15.5:0/3138959682 wait complete. 2024-04-22T13:35:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.076+0000 7fceae822740 20 client.18564 trim_cache size 0 max 16384 2024-04-22T13:35:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 Processor -- start 2024-04-22T13:35:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 -- start start 2024-04-22T13:35:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 0x557118acbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118abe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118aaba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ab850 con 0x557118ab23b0 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ad390 con 0x557118abbe60 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceae822740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41abf20 con 0x557118ab2d60 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea99d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 0x557118acbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea99d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 0x557118acbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58964/0 (socket says 172.21.15.5:58964) 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea99d6640 1 -- 172.21.15.5:0/3405032432 learned_addr learned my addr 172.21.15.5:0/3405032432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118aaba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea91d5640 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118abe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea99d6640 1 -- 172.21.15.5:0/3405032432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 msgr2=0x557118abe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea99d6640 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118abe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.090+0000 7fcea99d6640 1 -- 172.21.15.5:0/3405032432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 msgr2=0x557118aaba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fcea99d6640 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118aaba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fcea99d6640 1 -- 172.21.15.5:0/3405032432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557118ac2c80 con 0x557118ab23b0 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fcea91d5640 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118abe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fcea99d6640 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 0x557118acbd00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fcea000bd10 tx=0x7fcea001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f31200b079b429a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fce8f7fe640 1 -- 172.21.15.5:0/3405032432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea002f070 con 0x557118ab23b0 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fce8f7fe640 1 -- 172.21.15.5:0/3405032432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea0003c50 con 0x557118ab23b0 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fce8f7fe640 1 -- 172.21.15.5:0/3405032432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00292b0 con 0x557118ab23b0 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 -- 172.21.15.5:0/3405032432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 msgr2=0x557118acbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 0x557118acbd00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fcea000bd10 tx=0x7fcea001d040 comp rx=0 tx=0).stop 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 -- 172.21.15.5:0/3405032432 shutdown_connections 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118abbe60 0x557118aaba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118abe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 --2- 172.21.15.5:0/3405032432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118ab23b0 0x557118acbd00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.091+0000 7fceae822740 1 -- 172.21.15.5:0/3405032432 >> 172.21.15.5:0/3405032432 conn(0x557118aacba0 msgr2=0x557118aab370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 -- 172.21.15.5:0/3405032432 shutdown_connections 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 -- 172.21.15.5:0/3405032432 wait complete. 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 -- start start 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 Processor -- start 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ac8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118ac0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 0x557118ac1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41accc0 con 0x557118abbe60 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ac5f0 con 0x557118ab23b0 2024-04-22T13:35:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41a9850 con 0x557118ab2d60 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceae822740 10 client.0 _ll_register_callbacks cb 0x7fce94009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceaa1d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 0x557118ac1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceaa1d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 0x557118ac1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58980/0 (socket says 172.21.15.5:58980) 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fceaa1d7640 1 -- 172.21.15.5:0/2909168023 learned_addr learned my addr 172.21.15.5:0/2909168023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.092+0000 7fcea99d6640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ac8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fcea91d5640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118ac0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceaa1d7640 1 -- 172.21.15.5:0/2909168023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 msgr2=0x557118ac0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118ac0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceaa1d7640 1 -- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 msgr2=0x557118ac8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ac8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceaa1d7640 1 -- 172.21.15.5:0/2909168023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557118ac1980 con 0x557118abbe60 2024-04-22T13:35:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fcea99d6640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ac8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceaa1d7640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 0x557118ac1310 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0134d0 tx=0x7fce9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8ddb078e19985d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fce8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fce8d7fa640 1 -- 172.21.15.5:0/2909168023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004d20 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fce8d7fa640 1 -- 172.21.15.5:0/2909168023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fce9c0128d0 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea0003c50 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fce8d7fa640 1 -- 172.21.15.5:0/2909168023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce9c012ad0 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceae822740 10 client.18570 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557118b36450 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.093+0000 7fce8d7fa640 1 -- 172.21.15.5:0/2909168023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea0003c50 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fce8d7fa640 1 -- 172.21.15.5:0/2909168023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fce9c005850 con 0x557118abbe60 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 10 client.18570 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 10 client.18570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:35:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 20 client.18570 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118b36bb0 0x557118b38fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 4 client.18570 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557118a24c10 con 0x557118b36bb0 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fcea99d6640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118b36bb0 0x557118b38fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fce8d7fa640 10 client.18570 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fcea99d6640 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118b36bb0 0x557118b38fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1decdbfc5b0a2cd6 server_cookie=d96ee0cf171a585b in_seq=0 out_seq=0 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fce8d7fa640 1 -- 172.21.15.5:0/2909168023 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557118a24c10 con 0x557118b36bb0 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fce8d7fa640 10 client.18570 handle_command_reply: tid=0 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.094+0000 7fceae822740 1 client.18570 shutdown 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 msgr2=0x557118ac1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 0x557118ac1310 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0134d0 tx=0x7fce9c004a40 comp rx=0 tx=0).stop 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 shutdown_connections 2024-04-22T13:35:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x557118b36bb0 0x557118b38fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557118abbe60 0x557118ac1310 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557118ab2d60 0x557118ac0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 --2- 172.21.15.5:0/2909168023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557118ab23b0 0x557118ac8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fce8d7fa640 0 client.18570 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 >> 172.21.15.5:0/2909168023 conn(0x557118aacba0 msgr2=0x557118aabc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 shutdown_connections 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 1 -- 172.21.15.5:0/2909168023 wait complete. 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceae822740 20 client.18570 trim_cache size 0 max 16384 2024-04-22T13:35:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fce900a3da0 msgr2=0x7fce900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fce900a3da0 0x7fce900a6260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007310 tx=0x7fce9c0072a0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 msgr2=0x7fcea4198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea4198c10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fce94005940 tx=0x7fce9402d890 comp rx=0 tx=0).stop 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 shutdown_connections 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcea4103ae0 0x7fcea4198c10 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fce900a3da0 0x7fce900a6260 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcea410c500 0x7fcea419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 --2- 172.21.15.5:0/2311234785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea41031c0 0x7fcea41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 >> 172.21.15.5:0/2311234785 conn(0x7fcea4078000 msgr2=0x7fcea40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 shutdown_connections 2024-04-22T13:35:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.095+0000 7fceabc3a640 1 -- 172.21.15.5:0/2311234785 wait complete. 2024-04-22T13:35:27.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:35:27.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 -- 172.21.15.5:0/456430135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e410c6d0 msgr2=0x7f18e410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 --2- 172.21.15.5:0/456430135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e410c6d0 0x7f18e410eb60 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ca0 tx=0x7f18dc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 -- 172.21.15.5:0/456430135 shutdown_connections 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 --2- 172.21.15.5:0/456430135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e410c6d0 0x7f18e410eb60 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 --2- 172.21.15.5:0/456430135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18e4103b70 0x7f18e410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 --2- 172.21.15.5:0/456430135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18e41031c0 0x7f18e41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 -- 172.21.15.5:0/456430135 >> 172.21.15.5:0/456430135 conn(0x7f18e4077f70 msgr2=0x7f18e4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 -- 172.21.15.5:0/456430135 shutdown_connections 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.210+0000 7f18e9723640 1 -- 172.21.15.5:0/456430135 wait complete. 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 Processor -- start 2024-04-22T13:35:27.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 -- start start 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18e41031c0 0x7f18e419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18e410c6d0 0x7f18e419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 0x7f18e4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4112380 con 0x7f18e4198740 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4112200 con 0x7f18e41031c0 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.211+0000 7f18e9723640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4112500 con 0x7f18e410c6d0 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 0x7f18e4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 0x7f18e4198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58996/0 (socket says 172.21.15.5:58996) 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 -- 172.21.15.5:0/935684107 learned_addr learned my addr 172.21.15.5:0/935684107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18e41031c0 0x7f18e419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18da7fc640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18e410c6d0 0x7f18e419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 -- 172.21.15.5:0/935684107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18e410c6d0 msgr2=0x7f18e419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18e410c6d0 0x7f18e419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 -- 172.21.15.5:0/935684107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18e41031c0 msgr2=0x7f18e419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18e41031c0 0x7f18e419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 -- 172.21.15.5:0/935684107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e4199340 con 0x7f18e4198740 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e37fe640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 0x7f18e4198bc0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005e30 tx=0x7f18dc034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e0ff9640 1 -- 172.21.15.5:0/935684107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc039040 con 0x7f18e4198740 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e0ff9640 1 -- 172.21.15.5:0/935684107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18dc044070 con 0x7f18e4198740 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e41995d0 con 0x7f18e4198740 2024-04-22T13:35:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e0ff9640 1 -- 172.21.15.5:0/935684107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc003e60 con 0x7f18e4198740 2024-04-22T13:35:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.212+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41a56c0 con 0x7f18e4198740 2024-04-22T13:35:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.214+0000 7f18e0ff9640 1 -- 172.21.15.5:0/935684107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f18dc0324c0 con 0x7f18e4198740 2024-04-22T13:35:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 Processor -- start 2024-04-22T13:35:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 -- start start 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5638a543c1f0 con 0x5638a54d74d0 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5638a53d94b0 con 0x5638a54e0a00 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638a54381b0 con 0x5638a54d7e80 2024-04-22T13:35:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54d78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59002/0 (socket says 172.21.15.5:59002) 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 -- 172.21.15.5:0/2727179642 learned_addr learned my addr 172.21.15.5:0/2727179642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18e0ff9640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f18d00a3d20 0x7f18d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18e37fe640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f18d00a3d20 0x7f18d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 -- 172.21.15.5:0/2727179642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 msgr2=0x5638a54e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 -- 172.21.15.5:0/2727179642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 msgr2=0x5638a54e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.215+0000 7f18da7fc640 1 -- 172.21.15.5:0/2727179642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a53cb1b0 con 0x5638a54d74d0 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.216+0000 7f18e0ff9640 1 -- 172.21.15.5:0/935684107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f18dc0f2960 con 0x7f18e4198740 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.216+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54d78b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00bc50 tx=0x7f18cc0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19524e6d6fed0fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.216+0000 7f18d97fa640 1 -- 172.21.15.5:0/2727179642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0120d0 con 0x5638a54d74d0 2024-04-22T13:35:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.216+0000 7f18e37fe640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f18d00a3d20 0x7f18d00a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0c70f0 tx=0x7f18dc02c0f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.216+0000 7f18d97fa640 1 -- 172.21.15.5:0/2727179642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc016070 con 0x5638a54d74d0 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.216+0000 7f18d97fa640 1 -- 172.21.15.5:0/2727179642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc004a00 con 0x5638a54d74d0 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2727179642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 msgr2=0x5638a54d78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54d78b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00bc50 tx=0x7f18cc0127c0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2727179642 shutdown_connections 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2727179642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54d78b0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2727179642 >> 172.21.15.5:0/2727179642 conn(0x5638a54d1ba0 msgr2=0x5638a54d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2727179642 shutdown_connections 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2727179642 wait complete. 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- start start 2024-04-22T13:35:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 Processor -- start 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54dd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5638a54e3970 con 0x5638a54d74d0 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5638a53f5500 con 0x5638a54e0a00 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638a54e3af0 con 0x5638a54d7e80 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18ec3ed740 10 client.0 _ll_register_callbacks cb 0x5638a53c3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.217+0000 7f18da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54e0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59010/0 (socket says 172.21.15.5:59010) 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 -- 172.21.15.5:0/3362819232 learned_addr learned my addr 172.21.15.5:0/3362819232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54dd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 -- 172.21.15.5:0/3362819232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 msgr2=0x5638a54dd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54dd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 -- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 msgr2=0x5638a54ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54ddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 -- 172.21.15.5:0/3362819232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a514e3f0 con 0x5638a54d74d0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18da7fc640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54e0390 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f18cc015040 tx=0x7f18cc005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2024b0b211315f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ca7fc640 1 -- 172.21.15.5:0/3362819232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0046c0 con 0x5638a54d74d0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ca7fc640 1 -- 172.21.15.5:0/3362819232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f18cc004860 con 0x5638a54d74d0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc016070 con 0x5638a54d74d0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ca7fc640 1 -- 172.21.15.5:0/3362819232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18cc004a60 con 0x5638a54d74d0 2024-04-22T13:35:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ec3ed740 10 client.18582 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.218+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638a52187d0 con 0x5638a54d74d0 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ca7fc640 1 -- 172.21.15.5:0/3362819232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc016070 con 0x5638a54d74d0 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ca7fc640 1 -- 172.21.15.5:0/3362819232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f18cc032b90 con 0x5638a54d74d0 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ec3ed740 10 client.18582 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ec3ed740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ec3ed740 20 client.18582 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a54f0b20 0x5638a54f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ec3ed740 4 client.18582 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638a53db930 con 0x5638a54f0b20 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.219+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a54f0b20 0x5638a54f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.220+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a54f0b20 0x5638a54f2f10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3191b8a3e63976 server_cookie=c9939e9fcedad7dc in_seq=0 out_seq=0 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.220+0000 7f18ca7fc640 10 client.18582 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.220+0000 7f18ca7fc640 1 -- 172.21.15.5:0/3362819232 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638a53db930 con 0x5638a54f0b20 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.220+0000 7f18ca7fc640 10 client.18582 handle_command_reply: tid=0 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.220+0000 7f18ec3ed740 1 client.18582 shutdown 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 msgr2=0x5638a54e0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54e0390 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f18cc015040 tx=0x7f18cc005570 comp rx=0 tx=0).stop 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 shutdown_connections 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a54f0b20 0x5638a54f2f10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ca7fc640 0 client.18582 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/3362819232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54e0390 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 >> 172.21.15.5:0/3362819232 conn(0x5638a54d1ba0 msgr2=0x5638a54d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 shutdown_connections 2024-04-22T13:35:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 1 -- 172.21.15.5:0/3362819232 wait complete. 2024-04-22T13:35:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.221+0000 7f18ec3ed740 20 client.18582 trim_cache size 0 max 16384 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.234+0000 7f18ec3ed740 1 Processor -- start 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.234+0000 7f18ec3ed740 1 -- start start 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.234+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.234+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41adc10 con 0x5638a54d74d0 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41af750 con 0x5638a54e0a00 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41ae2e0 con 0x5638a54d7e80 2024-04-22T13:35:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54ec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59024/0 (socket says 172.21.15.5:59024) 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 -- 172.21.15.5:0/2029231096 learned_addr learned my addr 172.21.15.5:0/2029231096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 -- 172.21.15.5:0/2029231096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 msgr2=0x5638a54d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 -- 172.21.15.5:0/2029231096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 msgr2=0x5638a54dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54dfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 -- 172.21.15.5:0/2029231096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a54de0a0 con 0x5638a54d74d0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18da7fc640 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54ec830 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f18cc001eb0 tx=0x7f18cc005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1309728056532fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18d8ff9640 1 -- 172.21.15.5:0/2029231096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc02b280 con 0x5638a54d74d0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18d8ff9640 1 -- 172.21.15.5:0/2029231096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc016070 con 0x5638a54d74d0 2024-04-22T13:35:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.235+0000 7f18d8ff9640 1 -- 172.21.15.5:0/2029231096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc004e60 con 0x5638a54d74d0 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2029231096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 msgr2=0x5638a54ec830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54ec830 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f18cc001eb0 tx=0x7f18cc005600 comp rx=0 tx=0).stop 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2029231096 shutdown_connections 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d7e80 0x5638a54d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/2029231096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d74d0 0x5638a54ec830 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2029231096 >> 172.21.15.5:0/2029231096 conn(0x5638a54d1ba0 msgr2=0x5638a54dc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2029231096 shutdown_connections 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 -- 172.21.15.5:0/2029231096 wait complete. 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.236+0000 7f18ec3ed740 1 -- start start 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 Processor -- start 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d74d0 0x5638a54e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 0x5638a54da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54da610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41af080 con 0x5638a54d7e80 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41ae9b0 con 0x5638a54e0a00 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41abc10 con 0x5638a54d74d0 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 10 client.0 _ll_register_callbacks cb 0x7f18cc00bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 0x5638a54da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 0x5638a54da0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59038/0 (socket says 172.21.15.5:59038) 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 -- 172.21.15.5:0/1122674160 learned_addr learned my addr 172.21.15.5:0/1122674160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18da7fc640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d74d0 0x5638a54e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18da7fc640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 -- 172.21.15.5:0/1122674160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d74d0 msgr2=0x5638a54e90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d74d0 0x5638a54e90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 -- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 msgr2=0x5638a54da610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54da610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 -- 172.21.15.5:0/1122674160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a54dadf0 con 0x5638a54d7e80 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18da7fc640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54da610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18e2ffd640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 0x5638a54da0d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000c00 tx=0x7f18d401f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd3f65bec590910 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ca7fc640 1 -- 172.21.15.5:0/1122674160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4004220 con 0x5638a54d7e80 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ca7fc640 1 -- 172.21.15.5:0/1122674160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f18d40043c0 con 0x5638a54d7e80 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.237+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc016070 con 0x5638a54d7e80 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ca7fc640 1 -- 172.21.15.5:0/1122674160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18d4010960 con 0x5638a54d7e80 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 10 client.18588 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638a5556d60 con 0x5638a54d7e80 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ca7fc640 1 -- 172.21.15.5:0/1122674160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc016070 con 0x5638a54d7e80 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ca7fc640 1 -- 172.21.15.5:0/1122674160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f18d4004830 con 0x5638a54d7e80 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 10 client.18588 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 20 client.18588 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a5557510 0x5638a5559930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 4 client.18588 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638a54d1220 con 0x5638a5557510 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.238+0000 7f18da7fc640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a5557510 0x5638a5559930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18da7fc640 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a5557510 0x5638a5559930 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848a226bff8d8ab7 server_cookie=1e3f5ed0abd6bc18 in_seq=0 out_seq=0 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ca7fc640 10 client.18588 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ca7fc640 1 -- 172.21.15.5:0/1122674160 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638a54d1220 con 0x5638a5557510 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ca7fc640 10 client.18588 handle_command_reply: tid=0 2024-04-22T13:35:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 client.18588 shutdown 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 msgr2=0x5638a54da0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 0x5638a54da0d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000c00 tx=0x7f18d401f710 comp rx=0 tx=0).stop 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 shutdown_connections 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5638a5557510 0x5638a5559930 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638a54d7e80 0x5638a54da0d0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ca7fc640 0 client.18588 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5638a54e0a00 0x5638a54da610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 --2- 172.21.15.5:0/1122674160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638a54d74d0 0x5638a54e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 >> 172.21.15.5:0/1122674160 conn(0x5638a54d1ba0 msgr2=0x5638a54d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 shutdown_connections 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 1 -- 172.21.15.5:0/1122674160 wait complete. 2024-04-22T13:35:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.239+0000 7f18ec3ed740 20 client.18588 trim_cache size 0 max 16384 2024-04-22T13:35:27.243 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f18d00a3d20 msgr2=0x7f18d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f18d00a3d20 0x7f18d00a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0c70f0 tx=0x7f18dc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 msgr2=0x7f18e4198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 0x7f18e4198bc0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005e30 tx=0x7f18dc034760 comp rx=0 tx=0).stop 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 shutdown_connections 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18e4198740 0x7f18e4198bc0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f18d00a3d20 0x7f18d00a61e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18e410c6d0 0x7f18e419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 --2- 172.21.15.5:0/935684107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18e41031c0 0x7f18e419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 >> 172.21.15.5:0/935684107 conn(0x7f18e4077f70 msgr2=0x7f18e40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 shutdown_connections 2024-04-22T13:35:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:27.240+0000 7f18e9723640 1 -- 172.21.15.5:0/935684107 wait complete. 2024-04-22T13:35:28.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:35:28.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:35:28.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 -- 172.21.15.5:0/2866148171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 msgr2=0x7f46cc110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/2866148171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc110ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009ca0 tx=0x7f46c402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 -- 172.21.15.5:0/2866148171 shutdown_connections 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/2866148171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc110ce0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/2866148171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 0x7f46cc10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/2866148171 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 0x7f46cc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 -- 172.21.15.5:0/2866148171 >> 172.21.15.5:0/2866148171 conn(0x7f46cc078000 msgr2=0x7f46cc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 -- 172.21.15.5:0/2866148171 shutdown_connections 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 -- 172.21.15.5:0/2866148171 wait complete. 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 Processor -- start 2024-04-22T13:35:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.371+0000 7f46d3b6d640 1 -- start start 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d3b6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 0x7f46cc1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d3b6d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 0x7f46cc1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d3b6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d3b6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc114390 con 0x7f46cc10e8f0 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d3b6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc114210 con 0x7f46cc100d40 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d3b6d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc114510 con 0x7f46cc101660 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59058/0 (socket says 172.21.15.5:59058) 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 -- 172.21.15.5:0/1138956370 learned_addr learned my addr 172.21.15.5:0/1138956370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d1108640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 0x7f46cc1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d1909640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 0x7f46cc1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 -- 172.21.15.5:0/1138956370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 msgr2=0x7f46cc1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 0x7f46cc1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 -- 172.21.15.5:0/1138956370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 msgr2=0x7f46cc1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 0x7f46cc1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 -- 172.21.15.5:0/1138956370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc19af90 con 0x7f46cc10e8f0 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d1108640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 0x7f46cc1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d1909640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 0x7f46cc1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.372+0000 7f46d210a640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc19a6d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f46c402c700 tx=0x7f46c4034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.373+0000 7f46c2ffd640 1 -- 172.21.15.5:0/1138956370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4041020 con 0x7f46cc10e8f0 2024-04-22T13:35:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.373+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc19b220 con 0x7f46cc10e8f0 2024-04-22T13:35:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.373+0000 7f46c2ffd640 1 -- 172.21.15.5:0/1138956370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c4034ca0 con 0x7f46cc10e8f0 2024-04-22T13:35:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.373+0000 7f46c2ffd640 1 -- 172.21.15.5:0/1138956370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4039040 con 0x7f46cc10e8f0 2024-04-22T13:35:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.373+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a7540 con 0x7f46cc10e8f0 2024-04-22T13:35:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.375+0000 7f46c2ffd640 1 -- 172.21.15.5:0/1138956370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f46c4044020 con 0x7f46cc10e8f0 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 Processor -- start 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 -- start start 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b858984cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858990260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8588e8f00 con 0x55b85898de10 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b858886590 con 0x55b8589848e0 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d6834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b8588e4ec0 con 0x55b858985290 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46c2ffd640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46b80a3d20 0x7f46b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.376+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858990260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858990260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59072/0 (socket says 172.21.15.5:59072) 2024-04-22T13:35:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 -- 172.21.15.5:0/2671430367 learned_addr learned my addr 172.21.15.5:0/2671430367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1909640 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d210a640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46b80a3d20 0x7f46b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b858984cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46c2ffd640 1 -- 172.21.15.5:0/1138956370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f46c40f25c0 con 0x7f46cc10e8f0 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 -- 172.21.15.5:0/2671430367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 msgr2=0x55b85898d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 -- 172.21.15.5:0/2671430367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 msgr2=0x55b858984cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b858984cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 -- 172.21.15.5:0/2671430367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8586c37e0 con 0x55b85898de10 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1909640 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d1108640 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858990260 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0139d0 tx=0x7f46bc00c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1fc46cb6b45820 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46d210a640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46b80a3d20 0x7f46b80a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f46c40be310 tx=0x7f46c40be3e0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46b7fff640 1 -- 172.21.15.5:0/2671430367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc004220 con 0x55b85898de10 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46b7fff640 1 -- 172.21.15.5:0/2671430367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46bc00c9d0 con 0x55b85898de10 2024-04-22T13:35:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.377+0000 7f46b7fff640 1 -- 172.21.15.5:0/2671430367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0039e0 con 0x55b85898de10 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 -- 172.21.15.5:0/2671430367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 msgr2=0x55b858990260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858990260 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0139d0 tx=0x7f46bc00c510 comp rx=0 tx=0).stop 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 -- 172.21.15.5:0/2671430367 shutdown_connections 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858990260 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 --2- 172.21.15.5:0/2671430367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b858984cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 -- 172.21.15.5:0/2671430367 >> 172.21.15.5:0/2671430367 conn(0x55b85897efb0 msgr2=0x55b85897f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 -- 172.21.15.5:0/2671430367 shutdown_connections 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 -- 172.21.15.5:0/2671430367 wait complete. 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.378+0000 7f46d6834740 1 -- start start 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 Processor -- start 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b85898ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b85898b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b858990d80 con 0x55b85898de10 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8588a2580 con 0x55b8589848e0 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b858990f00 con 0x55b858985290 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d6834740 10 client.0 _ll_register_callbacks cb 0x55b8588a8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b85898b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b85898b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59084/0 (socket says 172.21.15.5:59084) 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1108640 1 -- 172.21.15.5:0/933078217 learned_addr learned my addr 172.21.15.5:0/933078217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1909640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1108640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b85898ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1108640 1 -- 172.21.15.5:0/933078217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 msgr2=0x55b85898b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.379+0000 7f46d1108640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d1108640 1 -- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 msgr2=0x55b85898ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d1108640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b85898ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d1108640 1 -- 172.21.15.5:0/933078217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b858723f70 con 0x55b85898de10 2024-04-22T13:35:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d1108640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b85898b860 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f46bc004490 tx=0x7f46bc0036c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c958ca20a34ded server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46b57fa640 1 -- 172.21.15.5:0/933078217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc023070 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46b57fa640 1 -- 172.21.15.5:0/933078217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f46bc00fce0 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46bc00c9d0 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46b57fa640 1 -- 172.21.15.5:0/933078217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46bc032050 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d6834740 10 client.18600 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8585bb170 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46b57fa640 1 -- 172.21.15.5:0/933078217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46bc00c9d0 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46b57fa640 1 -- 172.21.15.5:0/933078217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f46bc02c020 con 0x55b85898de10 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d6834740 10 client.18600 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.380+0000 7f46d6834740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46d6834740 20 client.18600 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46d6834740 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b85899e200 0x55b8589a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46d6834740 4 client.18600 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:28.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8588f6920 con 0x55b85899e200 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46d1909640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b85899e200 0x55b8589a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46d1909640 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b85899e200 0x55b8589a05f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631807c7f0ef040e server_cookie=3093fe6f45fb4fdc in_seq=0 out_seq=0 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.381+0000 7f46b57fa640 10 client.18600 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46b57fa640 1 -- 172.21.15.5:0/933078217 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8588f6920 con 0x55b85899e200 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46b57fa640 10 client.18600 handle_command_reply: tid=0 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 client.18600 shutdown 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 msgr2=0x55b85898b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b85898b860 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f46bc004490 tx=0x7f46bc0036c0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 shutdown_connections 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b85899e200 0x55b8589a05f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b85898b860 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b858985290 0x55b85898b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 --2- 172.21.15.5:0/933078217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8589848e0 0x55b85898ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46b57fa640 0 client.18600 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 >> 172.21.15.5:0/933078217 conn(0x55b85897efb0 msgr2=0x55b858980d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 shutdown_connections 2024-04-22T13:35:28.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 1 -- 172.21.15.5:0/933078217 wait complete. 2024-04-22T13:35:28.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.382+0000 7f46d6834740 20 client.18600 trim_cache size 0 max 16384 2024-04-22T13:35:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 Processor -- start 2024-04-22T13:35:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 -- start start 2024-04-22T13:35:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 0x55b85898d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858999c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b85898de10 0x55b858989cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1afd20 con 0x55b8589848e0 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1b1860 con 0x55b858985290 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.400+0000 7f46d6834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1b03f0 con 0x55b85898de10 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 0x55b85898d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 0x55b85898d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59088/0 (socket says 172.21.15.5:59088) 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 -- 172.21.15.5:0/24536124 learned_addr learned my addr 172.21.15.5:0/24536124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 -- 172.21.15.5:0/24536124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b85898de10 msgr2=0x55b858989cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b85898de10 0x55b858989cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 -- 172.21.15.5:0/24536124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 msgr2=0x55b858999c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858999c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 -- 172.21.15.5:0/24536124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b85897f7f0 con 0x55b8589848e0 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46d1108640 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 0x55b85898d510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f46bc003690 tx=0x7f46bc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67d0e841e96761f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46b77fe640 1 -- 172.21.15.5:0/24536124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc03b070 con 0x55b8589848e0 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46b77fe640 1 -- 172.21.15.5:0/24536124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46bc00cdd0 con 0x55b8589848e0 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.401+0000 7f46b77fe640 1 -- 172.21.15.5:0/24536124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc023070 con 0x55b8589848e0 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 -- 172.21.15.5:0/24536124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 msgr2=0x55b85898d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 0x55b85898d510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f46bc003690 tx=0x7f46bc005850 comp rx=0 tx=0).stop 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 -- 172.21.15.5:0/24536124 shutdown_connections 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b85898de10 0x55b858989cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858999c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 --2- 172.21.15.5:0/24536124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8589848e0 0x55b85898d510 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 -- 172.21.15.5:0/24536124 >> 172.21.15.5:0/24536124 conn(0x55b85897efb0 msgr2=0x55b8589895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 -- 172.21.15.5:0/24536124 shutdown_connections 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 -- 172.21.15.5:0/24536124 wait complete. 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 -- start start 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.402+0000 7f46d6834740 1 Processor -- start 2024-04-22T13:35:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8589848e0 0x55b858987e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858988390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858a03690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1b1190 con 0x55b85898de10 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1b0ac0 con 0x55b858985290 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1add20 con 0x55b8589848e0 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 10 client.0 _ll_register_callbacks cb 0x7f46bc013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858a03690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858a03690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59104/0 (socket says 172.21.15.5:59104) 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 -- 172.21.15.5:0/171775916 learned_addr learned my addr 172.21.15.5:0/171775916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1909640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858988390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8589848e0 0x55b858987e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 -- 172.21.15.5:0/171775916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8589848e0 msgr2=0x55b858987e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8589848e0 0x55b858987e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 -- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 msgr2=0x55b858988390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858988390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 -- 172.21.15.5:0/171775916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b858988c60 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d1108640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858a03690 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00faa0 tx=0x7f46bc00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb76fd48143d4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46b57fa640 1 -- 172.21.15.5:0/171775916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc03b070 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46b57fa640 1 -- 172.21.15.5:0/171775916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f46bc0134e0 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.403+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46bc00cdd0 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46b57fa640 1 -- 172.21.15.5:0/171775916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46bc023030 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 10 client.18606 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b858a04d10 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46b57fa640 1 -- 172.21.15.5:0/171775916 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46bc00cdd0 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46b57fa640 1 -- 172.21.15.5:0/171775916 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f46bc02c070 con 0x55b85898de10 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 10 client.18606 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 10 client.18606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 20 client.18606 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b858a052d0 0x55b858a076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 4 client.18606 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:35:28.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b8588f0790 con 0x55b858a052d0 2024-04-22T13:35:28.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.404+0000 7f46d1909640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b858a052d0 0x55b858a076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.405+0000 7f46b57fa640 10 client.18606 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.405+0000 7f46d1909640 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b858a052d0 0x55b858a076c0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e08a3727a56f3b server_cookie=1386775f73f4b5c9 in_seq=0 out_seq=0 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46b57fa640 1 -- 172.21.15.5:0/171775916 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b8588f0790 con 0x55b858a052d0 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46b57fa640 10 client.18606 handle_command_reply: tid=0 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 client.18606 shutdown 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 msgr2=0x55b858a03690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858a03690 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00faa0 tx=0x7f46bc00fa30 comp rx=0 tx=0).stop 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 shutdown_connections 2024-04-22T13:35:28.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b858a052d0 0x55b858a076c0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b85898de10 0x55b858a03690 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b858985290 0x55b858988390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 --2- 172.21.15.5:0/171775916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8589848e0 0x55b858987e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 >> 172.21.15.5:0/171775916 conn(0x55b85897efb0 msgr2=0x55b858989e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46b57fa640 0 client.18606 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 shutdown_connections 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 1 -- 172.21.15.5:0/171775916 wait complete. 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.406+0000 7f46d6834740 20 client.18606 trim_cache size 0 max 16384 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "bc515903-3be4-40d9-b1b3-7d5ac0512ed9", 2024-04-22T13:35:28.408 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:35:28.409 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:35:28.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46b80a3d20 msgr2=0x7f46b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46b80a3d20 0x7f46b80a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f46c40be310 tx=0x7f46c40be3e0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 msgr2=0x7f46cc19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc19a6d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f46c402c700 tx=0x7f46c4034670 comp rx=0 tx=0).stop 2024-04-22T13:35:28.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 shutdown_connections 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46cc10e8f0 0x7f46cc19a6d0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f46b80a3d20 0x7f46b80a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46cc101660 0x7f46cc1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 --2- 172.21.15.5:0/1138956370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46cc100d40 0x7f46cc1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 >> 172.21.15.5:0/1138956370 conn(0x7f46cc078000 msgr2=0x7f46cc10fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 shutdown_connections 2024-04-22T13:35:28.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.407+0000 7f46d3b6d640 1 -- 172.21.15.5:0/1138956370 wait complete. 2024-04-22T13:35:28.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:35:28.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 -- 172.21.15.5:0/353784308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd970103ae0 msgr2=0x7fd97010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 --2- 172.21.15.5:0/353784308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd970103ae0 0x7fd97010bfc0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd964009900 tx=0x7fd96402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 -- 172.21.15.5:0/353784308 shutdown_connections 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 --2- 172.21.15.5:0/353784308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd970103ae0 0x7fd97010bfc0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 --2- 172.21.15.5:0/353784308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd97010c590 0x7fd97010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 --2- 172.21.15.5:0/353784308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9701031c0 0x7fd9701035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 -- 172.21.15.5:0/353784308 >> 172.21.15.5:0/353784308 conn(0x7fd970077f40 msgr2=0x7fd970078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 -- 172.21.15.5:0/353784308 shutdown_connections 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.522+0000 7fd975fab640 1 -- 172.21.15.5:0/353784308 wait complete. 2024-04-22T13:35:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 Processor -- start 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 -- start start 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9701031c0 0x7fd970198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd970103ae0 0x7fd970198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 0x7fd97019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701121d0 con 0x7fd97010c590 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd970112050 con 0x7fd970103ae0 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd975fab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd970112350 con 0x7fd9701031c0 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd96ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 0x7fd97019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd96ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 0x7fd97019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59122/0 (socket says 172.21.15.5:59122) 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.523+0000 7fd96ffff640 1 -- 172.21.15.5:0/1026139954 learned_addr learned my addr 172.21.15.5:0/1026139954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96effd640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd970103ae0 0x7fd970198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9701031c0 0x7fd970198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96ffff640 1 -- 172.21.15.5:0/1026139954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9701031c0 msgr2=0x7fd970198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96ffff640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9701031c0 0x7fd970198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96ffff640 1 -- 172.21.15.5:0/1026139954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd970103ae0 msgr2=0x7fd970198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96ffff640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd970103ae0 0x7fd970198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96ffff640 1 -- 172.21.15.5:0/1026139954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97019d680 con 0x7fd97010c590 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96ffff640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 0x7fd97019cf50 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd96800eb40 tx=0x7fd96800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96cff9640 1 -- 172.21.15.5:0/1026139954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968004b20 con 0x7fd97010c590 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96cff9640 1 -- 172.21.15.5:0/1026139954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd968004cc0 con 0x7fd97010c590 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97019d970 con 0x7fd97010c590 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd96cff9640 1 -- 172.21.15.5:0/1026139954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968010720 con 0x7fd97010c590 2024-04-22T13:35:28.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.524+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9701a5200 con 0x7fd97010c590 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.526+0000 7fd96cff9640 1 -- 172.21.15.5:0/1026139954 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd968005560 con 0x7fd97010c590 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 Processor -- start 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 -- start start 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a87960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a90540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568a79ec1e0 con 0x5568a7a87ea0 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5568a7989390 con 0x5568a7a87580 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd978b9d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568a79e81a0 con 0x5568a7a90a80 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a90540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96cff9640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd9580a3da0 0x7fd9580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a90540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59128/0 (socket says 172.21.15.5:59128) 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 -- 172.21.15.5:0/2949556320 learned_addr learned my addr 172.21.15.5:0/2949556320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96effd640 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a87960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96ffff640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd9580a3da0 0x7fd9580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96effd640 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 -- 172.21.15.5:0/2949556320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 msgr2=0x5568a7a92e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96cff9640 1 -- 172.21.15.5:0/1026139954 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fd9680d0fc0 con 0x7fd97010c590 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a92e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 -- 172.21.15.5:0/2949556320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 msgr2=0x5568a7a87960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a87960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96f7fe640 1 -- 172.21.15.5:0/2949556320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a77c87d0 con 0x5568a7a87ea0 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.527+0000 7fd96effd640 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a90540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd96000cc60 tx=0x7fd96000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d199041de37d55b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd96ffff640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd9580a3da0 0x7fd9580a6260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd96800eb10 tx=0x7fd96805f040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd955ffb640 1 -- 172.21.15.5:0/2949556320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960014070 con 0x5568a7a87ea0 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd955ffb640 1 -- 172.21.15.5:0/2949556320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9600047a0 con 0x5568a7a87ea0 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd955ffb640 1 -- 172.21.15.5:0/2949556320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960010720 con 0x5568a7a87ea0 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 -- 172.21.15.5:0/2949556320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 msgr2=0x5568a7a90540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a90540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd96000cc60 tx=0x7fd96000e720 comp rx=0 tx=0).stop 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 -- 172.21.15.5:0/2949556320 shutdown_connections 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a90540 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a92e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 --2- 172.21.15.5:0/2949556320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 -- 172.21.15.5:0/2949556320 >> 172.21.15.5:0/2949556320 conn(0x5568a7a81cb0 msgr2=0x5568a7a82090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 -- 172.21.15.5:0/2949556320 shutdown_connections 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.528+0000 7fd978b9d740 1 -- 172.21.15.5:0/2949556320 wait complete. 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 -- start start 2024-04-22T13:35:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 Processor -- start 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a8da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a8dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a8e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568a7a93870 con 0x5568a7a87ea0 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5568a79a52f0 con 0x5568a7a87580 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568a7a939f0 con 0x5568a7a90a80 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd978b9d740 10 client.0 _ll_register_callbacks cb 0x5568a79b2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a8dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a8dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59134/0 (socket says 172.21.15.5:59134) 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.529+0000 7fd96f7fe640 1 -- 172.21.15.5:0/1991838375 learned_addr learned my addr 172.21.15.5:0/1991838375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96effd640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a8e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96effd640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a8da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96f7fe640 1 -- 172.21.15.5:0/1991838375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 msgr2=0x5568a7a8e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a8e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96f7fe640 1 -- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 msgr2=0x5568a7a8da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a8da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96f7fe640 1 -- 172.21.15.5:0/1991838375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a76fe3f0 con 0x5568a7a87ea0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96effd640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a8e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a8dfd0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd960010d00 tx=0x7fd960010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5a0b397bc608d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd94effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd94effd640 1 -- 172.21.15.5:0/1991838375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960027060 con 0x5568a7a87ea0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd94effd640 1 -- 172.21.15.5:0/1991838375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fd960018820 con 0x5568a7a87ea0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9600047a0 con 0x5568a7a87ea0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd94effd640 1 -- 172.21.15.5:0/1991838375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd960014070 con 0x5568a7a87ea0 2024-04-22T13:35:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd978b9d740 10 client.18618 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568a76be0f0 con 0x5568a7a87ea0 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.530+0000 7fd94effd640 1 -- 172.21.15.5:0/1991838375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9600047a0 con 0x5568a7a87ea0 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd94effd640 1 -- 172.21.15.5:0/1991838375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd960022800 con 0x5568a7a87ea0 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd978b9d740 10 client.18618 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd978b9d740 10 client.18618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd978b9d740 20 client.18618 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd978b9d740 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7aa0a20 0x5568a7aa2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd978b9d740 4 client.18618 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a79f9c00 con 0x5568a7aa0a20 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd96effd640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7aa0a20 0x5568a7aa2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd94effd640 10 client.18618 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.531+0000 7fd96effd640 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7aa0a20 0x5568a7aa2e10 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9adb7e03165202 server_cookie=89908e803cb709a6 in_seq=0 out_seq=0 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd94effd640 1 -- 172.21.15.5:0/1991838375 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568a79f9c00 con 0x5568a7aa0a20 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd94effd640 10 client.18618 handle_command_reply: tid=0 2024-04-22T13:35:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 client.18618 shutdown 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 msgr2=0x5568a7a8dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a8dfd0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd960010d00 tx=0x7fd960010d30 comp rx=0 tx=0).stop 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 shutdown_connections 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7aa0a20 0x5568a7aa2e10 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a8dfd0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a90a80 0x5568a7a8e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 --2- 172.21.15.5:0/1991838375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7a8da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd94effd640 0 client.18618 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 >> 172.21.15.5:0/1991838375 conn(0x5568a7a81cb0 msgr2=0x5568a7a83a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 shutdown_connections 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.532+0000 7fd978b9d740 1 -- 172.21.15.5:0/1991838375 wait complete. 2024-04-22T13:35:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.533+0000 7fd978b9d740 20 client.18618 trim_cache size 0 max 16384 2024-04-22T13:35:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 Processor -- start 2024-04-22T13:35:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 -- start start 2024-04-22T13:35:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87580 0x5568a7a901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a90a80 0x5568a7a99050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701adb00 con 0x5568a7a87ea0 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701af640 con 0x5568a7a90a80 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd978b9d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701ae1d0 con 0x5568a7a87580 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a9c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59142/0 (socket says 172.21.15.5:59142) 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.550+0000 7fd96f7fe640 1 -- 172.21.15.5:0/3845387030 learned_addr learned my addr 172.21.15.5:0/3845387030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96effd640 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a90a80 0x5568a7a99050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96f7fe640 1 -- 172.21.15.5:0/3845387030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87580 msgr2=0x5568a7a901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87580 0x5568a7a901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96f7fe640 1 -- 172.21.15.5:0/3845387030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a90a80 msgr2=0x5568a7a99050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a90a80 0x5568a7a99050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96f7fe640 1 -- 172.21.15.5:0/3845387030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a7a89670 con 0x5568a7a87ea0 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96effd640 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a90a80 0x5568a7a99050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a9c760 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd960022c20 tx=0x7fd96000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4152b80238d018 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd9557fa640 1 -- 172.21.15.5:0/3845387030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960027040 con 0x5568a7a87ea0 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd9557fa640 1 -- 172.21.15.5:0/3845387030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9600213c0 con 0x5568a7a87ea0 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd9557fa640 1 -- 172.21.15.5:0/3845387030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9600054d0 con 0x5568a7a87ea0 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd978b9d740 1 -- 172.21.15.5:0/3845387030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 msgr2=0x5568a7a9c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a9c760 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd960022c20 tx=0x7fd96000e720 comp rx=0 tx=0).stop 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd978b9d740 1 -- 172.21.15.5:0/3845387030 shutdown_connections 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.551+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a87ea0 0x5568a7a9c760 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a90a80 0x5568a7a99050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3845387030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87580 0x5568a7a901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- 172.21.15.5:0/3845387030 >> 172.21.15.5:0/3845387030 conn(0x5568a7a81cb0 msgr2=0x5568a7a830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- 172.21.15.5:0/3845387030 shutdown_connections 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- 172.21.15.5:0/3845387030 wait complete. 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- start start 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 Processor -- start 2024-04-22T13:35:28.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7afec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87ea0 0x5568a7aff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 0x5568a7aff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701aef70 con 0x5568a7a90a80 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701ae8a0 con 0x5568a7a87580 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701abb00 con 0x5568a7a87ea0 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd978b9d740 10 client.0 _ll_register_callbacks cb 0x7fd96000cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd96effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 0x5568a7aff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd96effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 0x5568a7aff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59148/0 (socket says 172.21.15.5:59148) 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.552+0000 7fd96effd640 1 -- 172.21.15.5:0/3175086037 learned_addr learned my addr 172.21.15.5:0/3175086037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7afec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87ea0 0x5568a7aff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 -- 172.21.15.5:0/3175086037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87ea0 msgr2=0x5568a7aff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87ea0 0x5568a7aff150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 -- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 msgr2=0x5568a7afec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7afec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 -- 172.21.15.5:0/3175086037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a7b09df0 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd96effd640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 0x5568a7aff690 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd964005940 tx=0x7fd964009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d518767d5b131 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd94effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd94effd640 1 -- 172.21.15.5:0/3175086037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96402ca80 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568a79f3a70 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd94effd640 1 -- 172.21.15.5:0/3175086037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fd96402cc20 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd94effd640 1 -- 172.21.15.5:0/3175086037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd964059050 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd978b9d740 10 client.18624 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568a7b0b100 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.553+0000 7fd94effd640 1 -- 172.21.15.5:0/3175086037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5568a79f3a70 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd94effd640 1 -- 172.21.15.5:0/3175086037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd964040040 con 0x5568a7a90a80 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd978b9d740 10 client.18624 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd978b9d740 10 client.18624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd978b9d740 20 client.18624 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7b0b810 0x5568a7b0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd978b9d740 4 client.18624 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568a796f590 con 0x5568a7b0b810 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7b0b810 0x5568a7b0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd96f7fe640 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7b0b810 0x5568a7b0dc30 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7430cbca8e9e357 server_cookie=d33b78c37e005c2c in_seq=0 out_seq=0 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.554+0000 7fd94effd640 10 client.18624 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd94effd640 1 -- 172.21.15.5:0/3175086037 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5568a796f590 con 0x5568a7b0b810 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd94effd640 10 client.18624 handle_command_reply: tid=0 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 client.18624 shutdown 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 msgr2=0x5568a7aff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 0x5568a7aff690 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd964005940 tx=0x7fd964009260 comp rx=0 tx=0).stop 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 shutdown_connections 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5568a7b0b810 0x5568a7b0dc30 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568a7a90a80 0x5568a7aff690 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7a87ea0 0x5568a7aff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 --2- 172.21.15.5:0/3175086037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5568a7a87580 0x5568a7afec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd94effd640 0 client.18624 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 >> 172.21.15.5:0/3175086037 conn(0x5568a7a81cb0 msgr2=0x5568a7a80d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 shutdown_connections 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 1 -- 172.21.15.5:0/3175086037 wait complete. 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd978b9d740 20 client.18624 trim_cache size 0 max 16384 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:35:28.558 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout: "bc515903-3be4-40d9-b1b3-7d5ac0512ed9": { 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "bc515903-3be4-40d9-b1b3-7d5ac0512ed9", 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:35:28.559 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd9580a3da0 msgr2=0x7fd9580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd975fab640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd9580a3da0 0x7fd9580a6260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd96800eb10 tx=0x7fd96805f040 comp rx=0 tx=0).stop 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 msgr2=0x7fd97019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.555+0000 7fd975fab640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 0x7fd97019cf50 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd96800eb40 tx=0x7fd96800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 shutdown_connections 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd97010c590 0x7fd97019cf50 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd9580a3da0 0x7fd9580a6260 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd970103ae0 0x7fd970198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 --2- 172.21.15.5:0/1026139954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9701031c0 0x7fd970198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:28.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 >> 172.21.15.5:0/1026139954 conn(0x7fd970077f40 msgr2=0x7fd9700fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:28.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 shutdown_connections 2024-04-22T13:35:28.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:28.556+0000 7fd975fab640 1 -- 172.21.15.5:0/1026139954 wait complete. 2024-04-22T13:35:28.563 INFO:tasks.cephfs.filesystem:scrub status for tag:bc515903-3be4-40d9-b1b3-7d5ac0512ed9 - {'path': '/', 'tag': 'bc515903-3be4-40d9-b1b3-7d5ac0512ed9', 'options': 'recursive,force'} 2024-04-22T13:35:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:28 smithi049 ceph-mon[31379]: pgmap v257: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:28 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:35:29.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:28 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:35:29.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:28 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:35:29.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:28 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:35:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:28 smithi107 ceph-mon[31859]: pgmap v257: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:28 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:35:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:28 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:35:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:28 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:35:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:28 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:35:29.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:28 smithi005 ceph-mon[28006]: pgmap v257: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:29.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:28 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:35:29.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:28 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:35:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:28 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:35:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:28 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:35:31.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:30 smithi049 ceph-mon[31379]: pgmap v258: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:31.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:30 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:35:31.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:30 smithi107 ceph-mon[31859]: pgmap v258: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:31.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:30 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:35:31.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:30 smithi005 ceph-mon[28006]: pgmap v258: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:31.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:30 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:35:33.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:32 smithi049 ceph-mon[31379]: pgmap v259: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:33.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:32 smithi107 ceph-mon[31859]: pgmap v259: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:33.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:32 smithi005 ceph-mon[28006]: pgmap v259: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:35.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:34 smithi049 ceph-mon[31379]: pgmap v260: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:35.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:34 smithi107 ceph-mon[31859]: pgmap v260: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:35.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:34 smithi005 ceph-mon[28006]: pgmap v260: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:37.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:36 smithi049 ceph-mon[31379]: pgmap v261: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:37.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:36 smithi107 ceph-mon[31859]: pgmap v261: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:37.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:36 smithi005 ceph-mon[28006]: pgmap v261: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:39.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:38 smithi049 ceph-mon[31379]: pgmap v262: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:38 smithi107 ceph-mon[31859]: pgmap v262: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:39.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:38 smithi005 ceph-mon[28006]: pgmap v262: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:41.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:40 smithi049 ceph-mon[31379]: pgmap v263: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:41.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:40 smithi107 ceph-mon[31859]: pgmap v263: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:41.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:40 smithi005 ceph-mon[28006]: pgmap v263: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:35:43.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:42 smithi049 ceph-mon[31379]: pgmap v264: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:43.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:42 smithi107 ceph-mon[31859]: pgmap v264: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:43.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:42 smithi005 ceph-mon[28006]: pgmap v264: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:35:45.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:44 smithi049 ceph-mon[31379]: pgmap v265: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:45.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:44 smithi107 ceph-mon[31859]: pgmap v265: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:44 smithi005 ceph-mon[28006]: pgmap v265: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:47.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:46 smithi049 ceph-mon[31379]: pgmap v266: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:47.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:46 smithi107 ceph-mon[31859]: pgmap v266: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:47.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:46 smithi005 ceph-mon[28006]: pgmap v266: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:49.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:48 smithi049 ceph-mon[31379]: pgmap v267: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:49.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:48 smithi107 ceph-mon[31859]: pgmap v267: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:48 smithi005 ceph-mon[28006]: pgmap v267: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:51.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:50 smithi049 ceph-mon[31379]: pgmap v268: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:51.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:50 smithi107 ceph-mon[31859]: pgmap v268: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:51.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:50 smithi005 ceph-mon[28006]: pgmap v268: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:53.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:52 smithi049 ceph-mon[31379]: pgmap v269: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:53.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:52 smithi107 ceph-mon[31859]: pgmap v269: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:52 smithi005 ceph-mon[28006]: pgmap v269: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:54 smithi049 ceph-mon[31379]: pgmap v270: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:55.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:54 smithi107 ceph-mon[31859]: pgmap v270: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:55.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:54 smithi005 ceph-mon[28006]: pgmap v270: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:57.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:56 smithi049 ceph-mon[31379]: pgmap v271: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:57.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:56 smithi107 ceph-mon[31859]: pgmap v271: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:57.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:56 smithi005 ceph-mon[28006]: pgmap v271: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:58.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:35:58.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:35:58.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.682+0000 7faf76185640 1 -- 172.21.15.5:0/771154411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70101a00 msgr2=0x7faf70105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.682+0000 7faf76185640 1 --2- 172.21.15.5:0/771154411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70101a00 0x7faf70105a40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7faf64009900 tx=0x7faf6402bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 -- 172.21.15.5:0/771154411 shutdown_connections 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 --2- 172.21.15.5:0/771154411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70101a00 0x7faf70105a40 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 --2- 172.21.15.5:0/771154411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf70106120 0x7faf7010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 --2- 172.21.15.5:0/771154411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf70101050 0x7faf70101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 -- 172.21.15.5:0/771154411 >> 172.21.15.5:0/771154411 conn(0x7faf70077fa0 msgr2=0x7faf700783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 -- 172.21.15.5:0/771154411 shutdown_connections 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 -- 172.21.15.5:0/771154411 wait complete. 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 Processor -- start 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.683+0000 7faf76185640 1 -- start start 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf76185640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf70101050 0x7faf70198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf76185640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 0x7faf70198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf76185640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf7019cfe0 0x7faf7019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf76185640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf701121a0 con 0x7faf70106120 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf76185640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf70112020 con 0x7faf70101050 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf76185640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf70112320 con 0x7faf7019cfe0 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 0x7faf70198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 0x7faf70198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45856/0 (socket says 172.21.15.5:45856) 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 -- 172.21.15.5:0/1877153119 learned_addr learned my addr 172.21.15.5:0/1877153119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf70101050 0x7faf70198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6ffff640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf7019cfe0 0x7faf7019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 -- 172.21.15.5:0/1877153119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf7019cfe0 msgr2=0x7faf7019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf7019cfe0 0x7faf7019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 -- 172.21.15.5:0/1877153119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf70101050 msgr2=0x7faf70198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf70101050 0x7faf70198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 -- 172.21.15.5:0/1877153119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7019dad0 con 0x7faf70106120 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6effd640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 0x7faf70198c60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7faf64036040 tx=0x7faf640045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.684+0000 7faf6cff9640 1 -- 172.21.15.5:0/1877153119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64043020 con 0x7faf70106120 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.685+0000 7faf6cff9640 1 -- 172.21.15.5:0/1877153119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf6402ca70 con 0x7faf70106120 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.685+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7019dd60 con 0x7faf70106120 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.685+0000 7faf6cff9640 1 -- 172.21.15.5:0/1877153119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64003ac0 con 0x7faf70106120 2024-04-22T13:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.685+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf701a5580 con 0x7faf70106120 2024-04-22T13:35:58.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf6cff9640 1 -- 172.21.15.5:0/1877153119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7faf64046020 con 0x7faf70106120 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 Processor -- start 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 -- start start 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03cea5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceb0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b03ce0a1e0 con 0x55b03cea5580 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b03cda7390 con 0x55b03cea5ea0 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf78d73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b03ce061a0 con 0x55b03ceaea80 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03cea5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf6cff9640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7faf580a3d20 0x7faf580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03cea5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45868/0 (socket says 172.21.15.5:45868) 2024-04-22T13:35:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.687+0000 7faf6ffff640 1 -- 172.21.15.5:0/973875904 learned_addr learned my addr 172.21.15.5:0/973875904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7faf580a3d20 0x7faf580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6cff9640 1 -- 172.21.15.5:0/1877153119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7faf640f2820 con 0x7faf70106120 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6effd640 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceb0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 -- 172.21.15.5:0/973875904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 msgr2=0x55b03ceb0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceb0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 -- 172.21.15.5:0/973875904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 msgr2=0x55b03ceae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 -- 172.21.15.5:0/973875904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b03cbe67d0 con 0x55b03cea5580 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf6ffff640 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03cea5960 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7faf680129f0 tx=0x7faf68012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb680f3acbe31a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf55ffb640 1 -- 172.21.15.5:0/973875904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68023070 con 0x55b03cea5580 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf55ffb640 1 -- 172.21.15.5:0/973875904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf680046e0 con 0x55b03cea5580 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.688+0000 7faf55ffb640 1 -- 172.21.15.5:0/973875904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6801fa00 con 0x55b03cea5580 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 -- 172.21.15.5:0/973875904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 msgr2=0x55b03cea5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf6ffff640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7faf580a3d20 0x7faf580a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faf68022000 tx=0x7faf68021000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03cea5960 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7faf680129f0 tx=0x7faf68012eb0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 -- 172.21.15.5:0/973875904 shutdown_connections 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceb0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 --2- 172.21.15.5:0/973875904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03cea5960 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 -- 172.21.15.5:0/973875904 >> 172.21.15.5:0/973875904 conn(0x55b03ce9fcb0 msgr2=0x55b03cea0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 -- 172.21.15.5:0/973875904 shutdown_connections 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 -- 172.21.15.5:0/973875904 wait complete. 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.689+0000 7faf78d73740 1 -- start start 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 Processor -- start 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03cea5580 0x55b03ceabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 0x55b03ceac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b03ceb1870 con 0x55b03ceaea80 2024-04-22T13:35:58.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b03cdc32f0 con 0x55b03cea5ea0 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b03ceb19f0 con 0x55b03cea5580 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf78d73740 10 client.0 _ll_register_callbacks cb 0x55b03cdd0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 0x55b03ceac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03cea5580 0x55b03ceabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 0x55b03ceac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45874/0 (socket says 172.21.15.5:45874) 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 -- 172.21.15.5:0/1521897190 learned_addr learned my addr 172.21.15.5:0/1521897190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6effd640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 -- 172.21.15.5:0/1521897190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03cea5580 msgr2=0x55b03ceabb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03cea5580 0x55b03ceabb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 -- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 msgr2=0x55b03ceac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6ffff640 1 -- 172.21.15.5:0/1521897190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b03cb1c3f0 con 0x55b03ceaea80 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.690+0000 7faf6effd640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf6ffff640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 0x55b03ceac580 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7faf68005c40 tx=0x7faf68009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5553c89189dbccae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf4effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf4effd640 1 -- 172.21.15.5:0/1521897190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68023070 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf4effd640 1 -- 172.21.15.5:0/1521897190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7faf680042d0 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf680046e0 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf4effd640 1 -- 172.21.15.5:0/1521897190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf680044f0 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 10 client.18636 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b03cadc0f0 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf4effd640 1 -- 172.21.15.5:0/1521897190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf680046e0 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf4effd640 1 -- 172.21.15.5:0/1521897190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faf6802e070 con 0x55b03ceaea80 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 10 client.18636 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 10 client.18636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 20 client.18636 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cebe8c0 0x55b03cec0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 4 client.18636 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.691+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b03ce17c00 con 0x55b03cebe8c0 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.692+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cebe8c0 0x55b03cec0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.692+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cebe8c0 0x55b03cec0ce0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b543a1b88150eff4 server_cookie=e97f4ec9a421a464 in_seq=0 out_seq=0 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.692+0000 7faf4effd640 10 client.18636 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf4effd640 1 -- 172.21.15.5:0/1521897190 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b03ce17c00 con 0x55b03cebe8c0 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf4effd640 10 client.18636 handle_command_reply: tid=0 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 client.18636 shutdown 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 msgr2=0x55b03ceac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 0x55b03ceac580 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7faf68005c40 tx=0x7faf68009240 comp rx=0 tx=0).stop 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 shutdown_connections 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cebe8c0 0x55b03cec0ce0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03ceaea80 0x55b03ceac580 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ceac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 --2- 172.21.15.5:0/1521897190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03cea5580 0x55b03ceabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf4effd640 0 client.18636 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 >> 172.21.15.5:0/1521897190 conn(0x55b03ce9fcb0 msgr2=0x55b03cea1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 shutdown_connections 2024-04-22T13:35:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.693+0000 7faf78d73740 1 -- 172.21.15.5:0/1521897190 wait complete. 2024-04-22T13:35:58.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.694+0000 7faf78d73740 20 client.18636 trim_cache size 0 max 16384 2024-04-22T13:35:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 Processor -- start 2024-04-22T13:35:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 -- start start 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03ceba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ce9ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf701ada50 con 0x55b03cea5580 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf701af590 con 0x55b03cea5ea0 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.711+0000 7faf78d73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf701ae120 con 0x55b03ceaea80 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03ceba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03ceba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45876/0 (socket says 172.21.15.5:45876) 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 -- 172.21.15.5:0/185660832 learned_addr learned my addr 172.21.15.5:0/185660832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6effd640 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ce9ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 -- 172.21.15.5:0/185660832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 msgr2=0x55b03ceac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6ffff640 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 -- 172.21.15.5:0/185660832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 msgr2=0x55b03ce9ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ce9ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 -- 172.21.15.5:0/185660832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b03cea5cd0 con 0x55b03cea5580 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6effd640 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ce9ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03ceba730 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7faf60009d10 tx=0x7faf60005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac60a30f479e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf557fa640 1 -- 172.21.15.5:0/185660832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60024070 con 0x55b03cea5580 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf557fa640 1 -- 172.21.15.5:0/185660832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf60010e50 con 0x55b03cea5580 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.712+0000 7faf557fa640 1 -- 172.21.15.5:0/185660832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60003850 con 0x55b03cea5580 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 -- 172.21.15.5:0/185660832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 msgr2=0x55b03ceba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03ceba730 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7faf60009d10 tx=0x7faf60005f90 comp rx=0 tx=0).stop 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 -- 172.21.15.5:0/185660832 shutdown_connections 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03ceac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5ea0 0x55b03ce9ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 --2- 172.21.15.5:0/185660832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5580 0x55b03ceba730 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 -- 172.21.15.5:0/185660832 >> 172.21.15.5:0/185660832 conn(0x55b03ce9fcb0 msgr2=0x55b03ceaaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 -- 172.21.15.5:0/185660832 shutdown_connections 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 -- 172.21.15.5:0/185660832 wait complete. 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 -- start start 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.713+0000 7faf78d73740 1 Processor -- start 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5580 0x55b03ceb6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 0x55b03ceb7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03cea7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf701aeec0 con 0x55b03cea5ea0 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf701ae7f0 con 0x55b03cea5580 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf701aba50 con 0x55b03ceaea80 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf78d73740 10 client.0 _ll_register_callbacks cb 0x7faf60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 0x55b03ceb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 0x55b03ceb7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45892/0 (socket says 172.21.15.5:45892) 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 -- 172.21.15.5:0/2060486395 learned_addr learned my addr 172.21.15.5:0/2060486395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6ffff640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03cea7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5580 0x55b03ceb6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 -- 172.21.15.5:0/2060486395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 msgr2=0x55b03cea7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03cea7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 -- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5580 msgr2=0x55b03ceb6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5580 0x55b03ceb6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 -- 172.21.15.5:0/2060486395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b03cec1050 con 0x55b03cea5ea0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf6effd640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 0x55b03ceb7180 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faf640f00e0 tx=0x7faf640df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9052f61e2beede41 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf4effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.714+0000 7faf4effd640 1 -- 172.21.15.5:0/2060486395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf640be8d0 con 0x55b03cea5ea0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf4effd640 1 -- 172.21.15.5:0/2060486395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7faf64003c60 con 0x55b03cea5ea0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf60010e50 con 0x55b03cea5ea0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf4effd640 1 -- 172.21.15.5:0/2060486395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf64003e60 con 0x55b03cea5ea0 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 10 client.18642 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b03cf24620 con 0x55b03cea5ea0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf4effd640 1 -- 172.21.15.5:0/2060486395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf60010e50 con 0x55b03cea5ea0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf4effd640 1 -- 172.21.15.5:0/2060486395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faf640c70b0 con 0x55b03cea5ea0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 10 client.18642 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 10 client.18642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 20 client.18642 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cf24da0 0x55b03cf271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 4 client.18642 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b03ce11a70 con 0x55b03cf24da0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.715+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cf24da0 0x55b03cf271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf6f7fe640 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cf24da0 0x55b03cf271c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4776036c0bde911e server_cookie=5c446be22efd7c37 in_seq=0 out_seq=0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf4effd640 10 client.18642 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf4effd640 1 -- 172.21.15.5:0/2060486395 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b03ce11a70 con 0x55b03cf24da0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf4effd640 10 client.18642 handle_command_reply: tid=0 2024-04-22T13:35:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 client.18642 shutdown 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 msgr2=0x55b03ceb7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 0x55b03ceb7180 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faf640f00e0 tx=0x7faf640df040 comp rx=0 tx=0).stop 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 shutdown_connections 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55b03cf24da0 0x55b03cf271c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b03cea5ea0 0x55b03ceb7180 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b03ceaea80 0x55b03cea7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 --2- 172.21.15.5:0/2060486395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b03cea5580 0x55b03ceb6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf4effd640 0 client.18642 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 >> 172.21.15.5:0/2060486395 conn(0x55b03ce9fcb0 msgr2=0x55b03cea1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 shutdown_connections 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 1 -- 172.21.15.5:0/2060486395 wait complete. 2024-04-22T13:35:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.716+0000 7faf78d73740 20 client.18642 trim_cache size 0 max 16384 2024-04-22T13:35:58.719 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:35:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:35:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:35:58.719 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7faf580a3d20 msgr2=0x7faf580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7faf580a3d20 0x7faf580a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faf68022000 tx=0x7faf68021000 comp rx=0 tx=0).stop 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 msgr2=0x7faf70198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 0x7faf70198c60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7faf64036040 tx=0x7faf640045e0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 shutdown_connections 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf70106120 0x7faf70198c60 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7faf580a3d20 0x7faf580a61e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf7019cfe0 0x7faf7019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 --2- 172.21.15.5:0/1877153119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf70101050 0x7faf70198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 >> 172.21.15.5:0/1877153119 conn(0x7faf70077fa0 msgr2=0x7faf700ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 shutdown_connections 2024-04-22T13:35:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.717+0000 7faf76185640 1 -- 172.21.15.5:0/1877153119 wait complete. 2024-04-22T13:35:58.723 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:35:58.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:35:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 -- 172.21.15.5:0/2257261751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd47006a930 msgr2=0x7fd47006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 --2- 172.21.15.5:0/2257261751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd47006a930 0x7fd47006ad10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009ca0 tx=0x7fd46c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 -- 172.21.15.5:0/2257261751 shutdown_connections 2024-04-22T13:35:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 --2- 172.21.15.5:0/2257261751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd470107310 0x7fd47010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 --2- 172.21.15.5:0/2257261751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd47006b2e0 0x7fd470106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 --2- 172.21.15.5:0/2257261751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd47006a930 0x7fd47006ad10 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 -- 172.21.15.5:0/2257261751 >> 172.21.15.5:0/2257261751 conn(0x7fd470077c20 msgr2=0x7fd470078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 -- 172.21.15.5:0/2257261751 shutdown_connections 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 -- 172.21.15.5:0/2257261751 wait complete. 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 Processor -- start 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.830+0000 7fd4781db640 1 -- start start 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd47006a930 0x7fd470101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 0x7fd4701016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd470101bf0 0x7fd4701a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701121e0 con 0x7fd470107310 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112060 con 0x7fd470101bf0 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112360 con 0x7fd47006a930 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 0x7fd4701016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 0x7fd4701016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37022/0 (socket says 172.21.15.5:37022) 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 -- 172.21.15.5:0/663143031 learned_addr learned my addr 172.21.15.5:0/663143031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:58.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd476778640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd470101bf0 0x7fd4701a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475f77640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd47006a930 0x7fd470101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 -- 172.21.15.5:0/663143031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd47006a930 msgr2=0x7fd470101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd47006a930 0x7fd470101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 -- 172.21.15.5:0/663143031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd470101bf0 msgr2=0x7fd4701a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd470101bf0 0x7fd4701a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 -- 172.21.15.5:0/663143031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701a6f50 con 0x7fd470107310 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd475776640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 0x7fd4701016b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd460005fd0 tx=0x7fd46000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460017070 con 0x7fd470107310 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd460009c30 con 0x7fd470107310 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701a7240 con 0x7fd470107310 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460003c40 con 0x7fd470107310 2024-04-22T13:35:58.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.831+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a77e0 con 0x7fd470107310 2024-04-22T13:35:58.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.832+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd470069250 con 0x7fd470107310 2024-04-22T13:35:58.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.833+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd460003de0 con 0x7fd470107310 2024-04-22T13:35:58.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.834+0000 7fd466ffd640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd45c0a3da0 0x7fd45c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:58.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.834+0000 7fd475f77640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd45c0a3da0 0x7fd45c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:58.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.834+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fd4600d0050 con 0x7fd470107310 2024-04-22T13:35:58.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.834+0000 7fd475f77640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd45c0a3da0 0x7fd45c0a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000f80 tx=0x7fd46c005c30 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:58.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:58.838+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd46009c770 con 0x7fd470107310 2024-04-22T13:35:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:58 smithi107 ceph-mon[31859]: pgmap v272: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:59.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:58 smithi005 ceph-mon[28006]: pgmap v272: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:59.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.066+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd470102650 con 0x7fd470107310 2024-04-22T13:35:59.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.067+0000 7fd466ffd640 1 -- 172.21.15.5:0/663143031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7fd470102650 con 0x7fd470107310 2024-04-22T13:35:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:35:59.063 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:35:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd45c0a3da0 msgr2=0x7fd45c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd45c0a3da0 0x7fd45c0a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000f80 tx=0x7fd46c005c30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 msgr2=0x7fd4701016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 0x7fd4701016b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd460005fd0 tx=0x7fd46000dde0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 shutdown_connections 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470107310 0x7fd4701016b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd45c0a3da0 0x7fd45c0a6260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd470101bf0 0x7fd4701a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 --2- 172.21.15.5:0/663143031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd47006a930 0x7fd470101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 >> 172.21.15.5:0/663143031 conn(0x7fd470077c20 msgr2=0x7fd4700ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 shutdown_connections 2024-04-22T13:35:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.069+0000 7fd4781db640 1 -- 172.21.15.5:0/663143031 wait complete. 2024-04-22T13:35:59.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 -- 172.21.15.5:0/625875994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 msgr2=0x7fe558103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/625875994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558103310 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe554009900 tx=0x7fe55402bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 -- 172.21.15.5:0/625875994 shutdown_connections 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/625875994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe55810c3a0 0x7fe55810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/625875994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe558103850 0x7fe55810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/625875994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558103310 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 -- 172.21.15.5:0/625875994 >> 172.21.15.5:0/625875994 conn(0x7fe5580fc750 msgr2=0x7fe5580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 -- 172.21.15.5:0/625875994 shutdown_connections 2024-04-22T13:35:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.186+0000 7fe55ed9f640 1 -- 172.21.15.5:0/625875994 wait complete. 2024-04-22T13:35:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 Processor -- start 2024-04-22T13:35:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 -- start start 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe558103850 0x7fe558198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe55810c3a0 0x7fe55819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe558112030 con 0x7fe558102f30 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe558111eb0 con 0x7fe55810c3a0 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55ed9f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581121b0 con 0x7fe558103850 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37050/0 (socket says 172.21.15.5:37050) 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 -- 172.21.15.5:0/3923955302 learned_addr learned my addr 172.21.15.5:0/3923955302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55d59c640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe558103850 0x7fe558198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55e59e640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe55810c3a0 0x7fe55819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 -- 172.21.15.5:0/3923955302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe558103850 msgr2=0x7fe558198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe558103850 0x7fe558198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 -- 172.21.15.5:0/3923955302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe55810c3a0 msgr2=0x7fe55819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe55810c3a0 0x7fe55819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55dd9d640 1 -- 172.21.15.5:0/3923955302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55819d580 con 0x7fe558102f30 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.187+0000 7fe55e59e640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe55810c3a0 0x7fe55819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.188+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558198590 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe554000c00 tx=0x7fe55402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.188+0000 7fe54effd640 1 -- 172.21.15.5:0/3923955302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554043020 con 0x7fe558102f30 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.188+0000 7fe54effd640 1 -- 172.21.15.5:0/3923955302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55402cbb0 con 0x7fe558102f30 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.188+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55819d810 con 0x7fe558102f30 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.188+0000 7fe54effd640 1 -- 172.21.15.5:0/3923955302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554003ac0 con 0x7fe558102f30 2024-04-22T13:35:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.188+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a5100 con 0x7fe558102f30 2024-04-22T13:35:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.190+0000 7fe54effd640 1 -- 172.21.15.5:0/3923955302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe554046020 con 0x7fe558102f30 2024-04-22T13:35:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.190+0000 7fe562bf1740 1 Processor -- start 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 -- start start 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd39648ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39649080 0x55bd39651630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 0x55bd39654050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd395ad1f0 con 0x55bd39651c00 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bd3954a4b0 con 0x55bd396486d0 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe562bf1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bd395a91b0 con 0x55bd39649080 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe54effd640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe5440a3d20 0x7fe5440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55d59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 0x55bd39654050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55d59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 0x55bd39654050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37062/0 (socket says 172.21.15.5:37062) 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55d59c640 1 -- 172.21.15.5:0/1818916194 learned_addr learned my addr 172.21.15.5:0/1818916194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55dd9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39649080 0x55bd39651630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55e59e640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe5440a3d20 0x7fe5440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55e59e640 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd39648ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe54effd640 1 -- 172.21.15.5:0/3923955302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe5540f28d0 con 0x7fe558102f30 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55d59c640 1 -- 172.21.15.5:0/1818916194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39649080 msgr2=0x55bd39651630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.191+0000 7fe55d59c640 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39649080 0x55bd39651630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe55d59c640 1 -- 172.21.15.5:0/1818916194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 msgr2=0x55bd39648ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe55d59c640 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd39648ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe55d59c640 1 -- 172.21.15.5:0/1818916194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3953c1b0 con 0x55bd39651c00 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe55e59e640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe5440a3d20 0x7fe5440a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe550005c40 tx=0x7fe550009960 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe55d59c640 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 0x55bd39654050 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe548009c60 tx=0x7fe548011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1742d9bbf293f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe543fff640 1 -- 172.21.15.5:0/1818916194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548004220 con 0x55bd39651c00 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe543fff640 1 -- 172.21.15.5:0/1818916194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5480043c0 con 0x55bd39651c00 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.192+0000 7fe543fff640 1 -- 172.21.15.5:0/1818916194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548004d80 con 0x55bd39651c00 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 -- 172.21.15.5:0/1818916194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 msgr2=0x55bd39654050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 0x55bd39654050 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe548009c60 tx=0x7fe548011810 comp rx=0 tx=0).stop 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 -- 172.21.15.5:0/1818916194 shutdown_connections 2024-04-22T13:35:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39651c00 0x55bd39654050 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39649080 0x55bd39651630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 --2- 172.21.15.5:0/1818916194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd39648ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 -- 172.21.15.5:0/1818916194 >> 172.21.15.5:0/1818916194 conn(0x55bd39642da0 msgr2=0x55bd39643180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 -- 172.21.15.5:0/1818916194 shutdown_connections 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 -- 172.21.15.5:0/1818916194 wait complete. 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 -- start start 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.193+0000 7fe562bf1740 1 Processor -- start 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd396486d0 0x55bd39651500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd3964e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39651c00 0x55bd3964ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd39654b70 con 0x55bd39649080 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55bd39566500 con 0x55bd39651c00 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bd39654cf0 con 0x55bd396486d0 2024-04-22T13:35:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 10 client.0 _ll_register_callbacks cb 0x55bd39534b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd3964e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd3964e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37070/0 (socket says 172.21.15.5:37070) 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 -- 172.21.15.5:0/3272542408 learned_addr learned my addr 172.21.15.5:0/3272542408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd396486d0 0x55bd39651500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55e59e640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39651c00 0x55bd3964ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 -- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd396486d0 msgr2=0x55bd39651500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd396486d0 0x55bd39651500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 -- 172.21.15.5:0/3272542408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39651c00 msgr2=0x55bd3964ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39651c00 0x55bd3964ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 -- 172.21.15.5:0/3272542408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd392bf3f0 con 0x55bd39649080 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe55d59c640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd3964e890 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe548004880 tx=0x7fe5480051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c5463b9ff77713 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe5417fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe5417fa640 1 -- 172.21.15.5:0/3272542408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548027030 con 0x55bd39649080 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe5417fa640 1 -- 172.21.15.5:0/3272542408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe5480055f0 con 0x55bd39649080 2024-04-22T13:35:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.194+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5480043c0 con 0x55bd39649080 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe5417fa640 1 -- 172.21.15.5:0/3272542408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe54801d040 con 0x55bd39649080 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 10 client.18660 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd393897d0 con 0x55bd39649080 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe5417fa640 1 -- 172.21.15.5:0/3272542408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5480043c0 con 0x55bd39649080 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe5417fa640 1 -- 172.21.15.5:0/3272542408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe5480359c0 con 0x55bd39649080 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 10 client.18660 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 10 client.18660 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 20 client.18660 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd39661c80 0x55bd39664070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 4 client.18660 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd3954c930 con 0x55bd39661c80 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.195+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd39661c80 0x55bd39664070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.196+0000 7fe5417fa640 10 client.18660 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.196+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd39661c80 0x55bd39664070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a62b61723b70603 server_cookie=504e5050c9be0363 in_seq=0 out_seq=0 2024-04-22T13:35:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe5417fa640 1 -- 172.21.15.5:0/3272542408 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd3954c930 con 0x55bd39661c80 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe5417fa640 10 client.18660 handle_command_reply: tid=0 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 client.18660 shutdown 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 msgr2=0x55bd3964e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd3964e890 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe548004880 tx=0x7fe5480051b0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 shutdown_connections 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd39661c80 0x55bd39664070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd3964e890 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe5417fa640 0 client.18660 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39651c00 0x55bd3964ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 --2- 172.21.15.5:0/3272542408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd396486d0 0x55bd39651500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 >> 172.21.15.5:0/3272542408 conn(0x55bd39642da0 msgr2=0x55bd39644af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 shutdown_connections 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 1 -- 172.21.15.5:0/3272542408 wait complete. 2024-04-22T13:35:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.197+0000 7fe562bf1740 20 client.18660 trim_cache size 0 max 16384 2024-04-22T13:35:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.210+0000 7fe562bf1740 1 Processor -- start 2024-04-22T13:35:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.210+0000 7fe562bf1740 1 -- start start 2024-04-22T13:35:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd3965da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd39644ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39650fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe562bf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581ada00 con 0x55bd39649080 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe562bf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581af540 con 0x55bd396486d0 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe562bf1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581ae0d0 con 0x55bd39651c00 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd39644ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd39644ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37080/0 (socket says 172.21.15.5:37080) 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 -- 172.21.15.5:0/653935894 learned_addr learned my addr 172.21.15.5:0/653935894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd3965da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 -- 172.21.15.5:0/653935894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 msgr2=0x55bd39650fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39650fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 -- 172.21.15.5:0/653935894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 msgr2=0x55bd3965da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd3965da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 -- 172.21.15.5:0/653935894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd39648e20 con 0x55bd39649080 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe55d59c640 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd39644ca0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe5480048b0 tx=0x7fe548004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67743a046187cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe5437fe640 1 -- 172.21.15.5:0/653935894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548027040 con 0x55bd39649080 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe5437fe640 1 -- 172.21.15.5:0/653935894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5480097a0 con 0x55bd39649080 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.211+0000 7fe5437fe640 1 -- 172.21.15.5:0/653935894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54802e900 con 0x55bd39649080 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 -- 172.21.15.5:0/653935894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 msgr2=0x55bd39644ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd39644ca0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe5480048b0 tx=0x7fe548004db0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 -- 172.21.15.5:0/653935894 shutdown_connections 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd39649080 0x55bd39644ca0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39650fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 --2- 172.21.15.5:0/653935894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd396486d0 0x55bd3965da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 -- 172.21.15.5:0/653935894 >> 172.21.15.5:0/653935894 conn(0x55bd39642da0 msgr2=0x55bd3964d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 -- 172.21.15.5:0/653935894 shutdown_connections 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 -- 172.21.15.5:0/653935894 wait complete. 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 -- start start 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.212+0000 7fe562bf1740 1 Processor -- start 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 0x55bd3965a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39649080 0x55bd39655520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39655a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581aee70 con 0x55bd396486d0 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581ae7a0 con 0x55bd39649080 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581aba00 con 0x55bd39651c00 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 10 client.0 _ll_register_callbacks cb 0x7fe548004e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 0x55bd3965a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 0x55bd3965a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37090/0 (socket says 172.21.15.5:37090) 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 -- 172.21.15.5:0/2892950740 learned_addr learned my addr 172.21.15.5:0/2892950740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55d59c640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39649080 0x55bd39655520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55e59e640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39655a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 -- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 msgr2=0x55bd39655a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39655a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 -- 172.21.15.5:0/2892950740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39649080 msgr2=0x55bd39655520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39649080 0x55bd39655520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 -- 172.21.15.5:0/2892950740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd39656050 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe55dd9d640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 0x55bd3965a0c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fe55402c210 tx=0x7fe55408e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee8d05c71edee5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe5417fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe5417fa640 1 -- 172.21.15.5:0/2892950740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5540a7e60 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe5417fa640 1 -- 172.21.15.5:0/2892950740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe554003c60 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5480097a0 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe5417fa640 1 -- 172.21.15.5:0/2892950740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe554003e60 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 10 client.18666 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.213+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd396c7ee0 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe5417fa640 1 -- 172.21.15.5:0/2892950740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5480097a0 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe5417fa640 1 -- 172.21.15.5:0/2892950740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe554090040 con 0x55bd396486d0 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe562bf1740 10 client.18666 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe562bf1740 10 client.18666 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:35:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe562bf1740 20 client.18666 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe562bf1740 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd396c8590 0x55bd396ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe562bf1740 4 client.18666 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd395bac10 con 0x55bd396c8590 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.214+0000 7fe55d59c640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd396c8590 0x55bd396ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe5417fa640 10 client.18666 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe55d59c640 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd396c8590 0x55bd396ca9b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa851abfe72cb79a server_cookie=59dbeaec02ca0efd in_seq=0 out_seq=0 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe5417fa640 1 -- 172.21.15.5:0/2892950740 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd395bac10 con 0x55bd396c8590 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe5417fa640 10 client.18666 handle_command_reply: tid=0 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 client.18666 shutdown 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 msgr2=0x55bd3965a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 0x55bd3965a0c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fe55402c210 tx=0x7fe55408e200 comp rx=0 tx=0).stop 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 shutdown_connections 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd39651c00 0x55bd39655a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd39649080 0x55bd39655520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55bd396c8590 0x55bd396ca9b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 --2- 172.21.15.5:0/2892950740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd396486d0 0x55bd3965a0c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 >> 172.21.15.5:0/2892950740 conn(0x55bd39642da0 msgr2=0x55bd39641590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe5417fa640 0 client.18666 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 shutdown_connections 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 1 -- 172.21.15.5:0/2892950740 wait complete. 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.215+0000 7fe562bf1740 20 client.18666 trim_cache size 0 max 16384 2024-04-22T13:35:59.217 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe5440a3d20 msgr2=0x7fe5440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe5440a3d20 0x7fe5440a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe550005c40 tx=0x7fe550009960 comp rx=0 tx=0).stop 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 msgr2=0x7fe558198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558198590 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe554000c00 tx=0x7fe55402c6c0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 shutdown_connections 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe5440a3d20 0x7fe5440a61e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe55810c3a0 0x7fe55819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe558103850 0x7fe558198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 --2- 172.21.15.5:0/3923955302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe558102f30 0x7fe558198590 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 >> 172.21.15.5:0/3923955302 conn(0x7fe5580fc750 msgr2=0x7fe5580fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 shutdown_connections 2024-04-22T13:35:59.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.216+0000 7fe55ed9f640 1 -- 172.21.15.5:0/3923955302 wait complete. 2024-04-22T13:35:59.222 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:35:59.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:58 smithi049 ceph-mon[31379]: pgmap v272: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.330+0000 7fe7c12eb640 1 -- 172.21.15.5:0/1393078838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc101a40 msgr2=0x7fe7bc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.330+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/1393078838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc101a40 0x7fe7bc105a80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009900 tx=0x7fe7b002bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 -- 172.21.15.5:0/1393078838 shutdown_connections 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/1393078838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc101a40 0x7fe7bc105a80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/1393078838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7bc106160 0x7fe7bc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/1393078838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7bc101090 0x7fe7bc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 -- 172.21.15.5:0/1393078838 >> 172.21.15.5:0/1393078838 conn(0x7fe7bc078000 msgr2=0x7fe7bc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 -- 172.21.15.5:0/1393078838 shutdown_connections 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 -- 172.21.15.5:0/1393078838 wait complete. 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 Processor -- start 2024-04-22T13:35:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.331+0000 7fe7c12eb640 1 -- start start 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7bc101090 0x7fe7bc1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7bc106160 0x7fe7bc199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 0x7fe7bc1a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc111080 con 0x7fe7bc199fe0 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc110f00 con 0x7fe7bc101090 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc111200 con 0x7fe7bc106160 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 0x7fe7bc1a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 0x7fe7bc1a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37106/0 (socket says 172.21.15.5:37106) 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 -- 172.21.15.5:0/3691227725 learned_addr learned my addr 172.21.15.5:0/3691227725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7bc106160 0x7fe7bc199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7baffd640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7bc101090 0x7fe7bc1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 -- 172.21.15.5:0/3691227725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7bc106160 msgr2=0x7fe7bc199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7bc106160 0x7fe7bc199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 -- 172.21.15.5:0/3691227725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7bc101090 msgr2=0x7fe7bc1994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7bc101090 0x7fe7bc1994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 -- 172.21.15.5:0/3691227725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc1a4d60 con 0x7fe7bc199fe0 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7bb7fe640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 0x7fe7bc1a4540 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400cab0 tx=0x7fe7b400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7a3fff640 1 -- 172.21.15.5:0/3691227725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400ae20 con 0x7fe7bc199fe0 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc1a5050 con 0x7fe7bc199fe0 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7a3fff640 1 -- 172.21.15.5:0/3691227725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b400f900 con 0x7fe7bc199fe0 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7a3fff640 1 -- 172.21.15.5:0/3691227725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40189f0 con 0x7fe7bc199fe0 2024-04-22T13:35:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.332+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1a5520 con 0x7fe7bc199fe0 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7a3fff640 1 -- 172.21.15.5:0/3691227725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe7b400fab0 con 0x7fe7bc199fe0 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 Processor -- start 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 -- start start 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eece1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 0x5651eeceb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651eec461f0 con 0x5651eeceae60 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5651eebe34b0 con 0x5651eece13b0 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651eec421b0 con 0x5651eece1d60 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7a3fff640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 0x5651eeceb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 0x5651eeceb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37108/0 (socket says 172.21.15.5:37108) 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.335+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3200217888 learned_addr learned my addr 172.21.15.5:0/3200217888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7bb7fe640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7baffd640 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7a3fff640 1 -- 172.21.15.5:0/3691227725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe7b40d0e30 con 0x7fe7bc199fe0 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eece1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3200217888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 msgr2=0x5651eecea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3200217888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 msgr2=0x5651eece1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eece1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3200217888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651eebd51b0 con 0x5651eeceae60 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7baffd640 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 0x5651eeceb2c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc06b670 tx=0x7fe7b0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe41d9223129426a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7bb7fe640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40ce630 tx=0x7fe7b4067920 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7a17fa640 1 -- 172.21.15.5:0/3200217888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0047020 con 0x5651eeceae60 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7a17fa640 1 -- 172.21.15.5:0/3200217888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b002cc90 con 0x5651eeceae60 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.336+0000 7fe7a17fa640 1 -- 172.21.15.5:0/3200217888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0004920 con 0x5651eeceae60 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3200217888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 msgr2=0x5651eeceb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 0x5651eeceb2c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc06b670 tx=0x7fe7b0005e40 comp rx=0 tx=0).stop 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3200217888 shutdown_connections 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eeceae60 0x5651eeceb2c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3200217888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eece1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3200217888 >> 172.21.15.5:0/3200217888 conn(0x5651eecdbba0 msgr2=0x5651eecdbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3200217888 shutdown_connections 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3200217888 wait complete. 2024-04-22T13:35:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- start start 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 Processor -- start 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecf1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 0x5651eeceeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651eecf1d40 con 0x5651eece13b0 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5651eebff500 con 0x5651eeceae60 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.337+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651eecf1ec0 con 0x5651eece1d60 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7c3fbb740 10 client.0 _ll_register_callbacks cb 0x5651eebcdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecf1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecf1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37122/0 (socket says 172.21.15.5:37122) 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3914684087 learned_addr learned my addr 172.21.15.5:0/3914684087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7baffd640 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3914684087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 msgr2=0x5651eecee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 msgr2=0x5651eeceeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 0x5651eeceeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/3914684087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ee9583f0 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecf1230 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002c170 tx=0x7fe7b002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f9bc657b9c22d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe79eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe79eff5640 1 -- 172.21.15.5:0/3914684087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0051070 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe79eff5640 1 -- 172.21.15.5:0/3914684087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe7b0003a80 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b002cc90 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe79eff5640 1 -- 172.21.15.5:0/3914684087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7b0047030 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7c3fbb740 10 client.18678 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.338+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651eea227d0 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe79eff5640 1 -- 172.21.15.5:0/3914684087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b002cc90 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe79eff5640 1 -- 172.21.15.5:0/3914684087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b0043450 con 0x5651eece13b0 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7c3fbb740 10 client.18678 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7c3fbb740 10 client.18678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:35:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7c3fbb740 20 client.18678 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eecfee40 0x5651eed01230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7c3fbb740 4 client.18678 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651eebe5930 con 0x5651eecfee40 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.339+0000 7fe7baffd640 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eecfee40 0x5651eed01230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.340+0000 7fe7baffd640 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eecfee40 0x5651eed01230 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63041f46bb33cbb0 server_cookie=ca4da2cab4c20e09 in_seq=0 out_seq=0 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.340+0000 7fe79eff5640 10 client.18678 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.340+0000 7fe79eff5640 1 -- 172.21.15.5:0/3914684087 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651eebe5930 con 0x5651eecfee40 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.340+0000 7fe79eff5640 10 client.18678 handle_command_reply: tid=0 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 client.18678 shutdown 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 msgr2=0x5651eecf1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecf1230 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002c170 tx=0x7fe7b002bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 shutdown_connections 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eecfee40 0x5651eed01230 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 0x5651eeceeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe79eff5640 0 client.18678 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/3914684087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecf1230 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 >> 172.21.15.5:0/3914684087 conn(0x5651eecdbba0 msgr2=0x5651eecdd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 shutdown_connections 2024-04-22T13:35:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/3914684087 wait complete. 2024-04-22T13:35:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.341+0000 7fe7c3fbb740 20 client.18678 trim_cache size 0 max 16384 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 Processor -- start 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 -- start start 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecfac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecf0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 0x5651eecf1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1ab7f0 con 0x5651eece13b0 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1ad330 con 0x5651eeceae60 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1abec0 con 0x5651eece1d60 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecfac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecfac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37136/0 (socket says 172.21.15.5:37136) 2024-04-22T13:35:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.354+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/2965377163 learned_addr learned my addr 172.21.15.5:0/2965377163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7baffd640 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecf0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/2965377163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 msgr2=0x5651eecf0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecf0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/2965377163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 msgr2=0x5651eecf1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 0x5651eecf1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/2965377163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651eed015a0 con 0x5651eece13b0 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecfac70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00040b0 tx=0x7fe7b002caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda0ed0049cf270f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7a0ff9640 1 -- 172.21.15.5:0/2965377163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0066070 con 0x5651eece13b0 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7a0ff9640 1 -- 172.21.15.5:0/2965377163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0009520 con 0x5651eece13b0 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.355+0000 7fe7a0ff9640 1 -- 172.21.15.5:0/2965377163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0051040 con 0x5651eece13b0 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/2965377163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 msgr2=0x5651eecfac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecfac70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00040b0 tx=0x7fe7b002caf0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/2965377163 shutdown_connections 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eeceae60 0x5651eecf1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eece1d60 0x5651eecf0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/2965377163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece13b0 0x5651eecfac70 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/2965377163 >> 172.21.15.5:0/2965377163 conn(0x5651eecdbba0 msgr2=0x5651eeced5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/2965377163 shutdown_connections 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/2965377163 wait complete. 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- start start 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 Processor -- start 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eecf7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 0x5651eecf76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 0x5651eecefc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1acc60 con 0x5651eece1d60 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1ac590 con 0x5651eece13b0 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1a97f0 con 0x5651eeceae60 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7c3fbb740 10 client.0 _ll_register_callbacks cb 0x7fe7b002cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 0x5651eecf76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 0x5651eecefc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7ba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 0x5651eecefc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:40658/0 (socket says 172.21.15.5:40658) 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/359054811 learned_addr learned my addr 172.21.15.5:0/359054811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.356+0000 7fe7baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 0x5651eecf76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37142/0 (socket says 172.21.15.5:37142) 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eecf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 msgr2=0x5651eecf7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eecf7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/359054811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 msgr2=0x5651eecf76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 0x5651eecf76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 -- 172.21.15.5:0/359054811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651eecf04a0 con 0x5651eeceae60 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7baffd640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 0x5651eecf76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7ba7fc640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 0x5651eecefc70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0043f00 tx=0x7fe7b0043dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1ce6cbc8757c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe79eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe79eff5640 1 -- 172.21.15.5:0/359054811 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0066040 con 0x5651eeceae60 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0009520 con 0x5651eeceae60 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe79eff5640 1 -- 172.21.15.5:0/359054811 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe7b0003a80 con 0x5651eeceae60 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe79eff5640 1 -- 172.21.15.5:0/359054811 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7b0003ca0 con 0x5651eeceae60 2024-04-22T13:35:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7c3fbb740 10 client.27826 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.357+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651eed64cd0 con 0x5651eeceae60 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe79eff5640 1 -- 172.21.15.5:0/359054811 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0009520 con 0x5651eeceae60 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe79eff5640 1 -- 172.21.15.5:0/359054811 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b0041460 con 0x5651eeceae60 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7c3fbb740 10 client.27826 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7c3fbb740 10 client.27826 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7c3fbb740 20 client.27826 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eed651f0 0x5651eed675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7c3fbb740 4 client.27826 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651eec53c10 con 0x5651eed651f0 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.358+0000 7fe7baffd640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eed651f0 0x5651eed675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7baffd640 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eed651f0 0x5651eed675e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b40cf4c2ad1863a server_cookie=6e4e541b77b01bd7 in_seq=0 out_seq=0 2024-04-22T13:35:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe79eff5640 10 client.27826 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe79eff5640 1 -- 172.21.15.5:0/359054811 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651eec53c10 con 0x5651eed651f0 2024-04-22T13:35:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe79eff5640 10 client.27826 handle_command_reply: tid=0 2024-04-22T13:35:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 client.27826 shutdown 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 msgr2=0x5651eecefc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 0x5651eecefc70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0043f00 tx=0x7fe7b0043dc0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 shutdown_connections 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5651eed651f0 0x5651eed675e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651eece1d60 0x5651eecf76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651eeceae60 0x5651eecefc70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe79eff5640 0 client.27826 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 --2- 172.21.15.5:0/359054811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5651eece13b0 0x5651eecf7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 >> 172.21.15.5:0/359054811 conn(0x5651eecdbba0 msgr2=0x5651eece87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 shutdown_connections 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 1 -- 172.21.15.5:0/359054811 wait complete. 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c3fbb740 20 client.27826 trim_cache size 0 max 16384 2024-04-22T13:35:59.360 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe7a40a3d20 msgr2=0x7fe7a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40ce630 tx=0x7fe7b4067920 comp rx=0 tx=0).stop 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 msgr2=0x7fe7bc1a4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.359+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 0x7fe7bc1a4540 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400cab0 tx=0x7fe7b400cf70 comp rx=0 tx=0).stop 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 shutdown_connections 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc199fe0 0x7fe7bc1a4540 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7bc106160 0x7fe7bc199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 --2- 172.21.15.5:0/3691227725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7bc101090 0x7fe7bc1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 >> 172.21.15.5:0/3691227725 conn(0x7fe7bc078000 msgr2=0x7fe7bc107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 shutdown_connections 2024-04-22T13:35:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.360+0000 7fe7c12eb640 1 -- 172.21.15.5:0/3691227725 wait complete. 2024-04-22T13:35:59.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 -- 172.21.15.5:0/1191441984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc101090 msgr2=0x7f50bc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 --2- 172.21.15.5:0/1191441984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc101090 0x7f50bc101470 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009900 tx=0x7f50b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 -- 172.21.15.5:0/1191441984 shutdown_connections 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 --2- 172.21.15.5:0/1191441984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50bc106160 0x7f50bc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 --2- 172.21.15.5:0/1191441984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101a40 0x7f50bc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 --2- 172.21.15.5:0/1191441984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc101090 0x7f50bc101470 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 -- 172.21.15.5:0/1191441984 >> 172.21.15.5:0/1191441984 conn(0x7f50bc078000 msgr2=0x7f50bc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 -- 172.21.15.5:0/1191441984 shutdown_connections 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 -- 172.21.15.5:0/1191441984 wait complete. 2024-04-22T13:35:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 Processor -- start 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 -- start start 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.474+0000 7f50c4196640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101090 0x7f50bc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50bc106160 0x7f50bc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 0x7f50bc1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1110c0 con 0x7f50bc19a090 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc110f40 con 0x7f50bc106160 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc111240 con 0x7f50bc101090 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 0x7f50bc1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 0x7f50bc1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37160/0 (socket says 172.21.15.5:37160) 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 -- 172.21.15.5:0/947908524 learned_addr learned my addr 172.21.15.5:0/947908524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c1731640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50bc106160 0x7f50bc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c1f32640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101090 0x7f50bc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 -- 172.21.15.5:0/947908524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101090 msgr2=0x7f50bc1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101090 0x7f50bc1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 -- 172.21.15.5:0/947908524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50bc106160 msgr2=0x7f50bc199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50bc106160 0x7f50bc199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 -- 172.21.15.5:0/947908524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc1a4e40 con 0x7f50bc19a090 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c1f32640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101090 0x7f50bc1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c2733640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 0x7f50bc1a4640 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f50b400e960 tx=0x7f50b400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50b2ffd640 1 -- 172.21.15.5:0/947908524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4004d20 con 0x7f50bc19a090 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50b2ffd640 1 -- 172.21.15.5:0/947908524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b4005020 con 0x7f50bc19a090 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc1a5130 con 0x7f50bc19a090 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50b2ffd640 1 -- 172.21.15.5:0/947908524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b40116d0 con 0x7f50bc19a090 2024-04-22T13:35:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.475+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a5630 con 0x7f50bc19a090 2024-04-22T13:35:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50b2ffd640 1 -- 172.21.15.5:0/947908524 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f50b40051d0 con 0x7f50bc19a090 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 Processor -- start 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 -- start start 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5b4d0 0x558cebf5b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 0x558cebf64430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf66e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cebec01f0 con 0x558cebf5be80 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558cebe5d4b0 con 0x558cebf5b4d0 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.478+0000 7f50c6d86740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558cebebc1b0 con 0x558cebf64a00 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 0x558cebf64430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50b2ffd640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50a80a3d20 0x7f50a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 0x558cebf64430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37174/0 (socket says 172.21.15.5:37174) 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 -- 172.21.15.5:0/83551291 learned_addr learned my addr 172.21.15.5:0/83551291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1731640 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5b4d0 0x558cebf5b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c2733640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50a80a3d20 0x7f50a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1731640 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf66e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50b2ffd640 1 -- 172.21.15.5:0/947908524 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f50b40d0910 con 0x7f50bc19a090 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 -- 172.21.15.5:0/83551291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 msgr2=0x558cebf66e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf66e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 -- 172.21.15.5:0/83551291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5b4d0 msgr2=0x558cebf5b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5b4d0 0x558cebf5b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 -- 172.21.15.5:0/83551291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cebe4f1b0 con 0x558cebf5be80 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1731640 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5b4d0 0x558cebf5b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c2733640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50a80a3d20 0x7f50a80a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50b40ce110 tx=0x7f50b4087040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.479+0000 7f50c1f32640 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 0x558cebf64430 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50b8036040 tx=0x7f50b8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffa6fd3059e0081 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50a7fff640 1 -- 172.21.15.5:0/83551291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8043020 con 0x558cebf5be80 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50a7fff640 1 -- 172.21.15.5:0/83551291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b802c7e0 con 0x558cebf5be80 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50a7fff640 1 -- 172.21.15.5:0/83551291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8005540 con 0x558cebf5be80 2024-04-22T13:35:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 -- 172.21.15.5:0/83551291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 msgr2=0x558cebf64430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 0x558cebf64430 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50b8036040 tx=0x7f50b8005e50 comp rx=0 tx=0).stop 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 -- 172.21.15.5:0/83551291 shutdown_connections 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5be80 0x558cebf64430 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf66e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 --2- 172.21.15.5:0/83551291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5b4d0 0x558cebf5b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 -- 172.21.15.5:0/83551291 >> 172.21.15.5:0/83551291 conn(0x558cebf55ba0 msgr2=0x558cebf55f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 -- 172.21.15.5:0/83551291 shutdown_connections 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.480+0000 7f50c6d86740 1 -- 172.21.15.5:0/83551291 wait complete. 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 -- start start 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 Processor -- start 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf61c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cebf67910 con 0x558cebf5b4d0 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558cebe79500 con 0x558cebf5be80 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558cebf67a90 con 0x558cebf64a00 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c1731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c1731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf642f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37176/0 (socket says 172.21.15.5:37176) 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c1731640 1 -- 172.21.15.5:0/3435571751 learned_addr learned my addr 172.21.15.5:0/3435571751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.481+0000 7f50c6d86740 10 client.0 _ll_register_callbacks cb 0x558cebe47b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf61c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 -- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 msgr2=0x558cebf61c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf61c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 -- 172.21.15.5:0/3435571751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 msgr2=0x558cebf61680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf61680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 -- 172.21.15.5:0/3435571751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cebbd23f0 con 0x558cebf5b4d0 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c1731640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf642f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f50ac003850 tx=0x7f50ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba69f9b41d81a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50a57fa640 1 -- 172.21.15.5:0/3435571751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac012dc0 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50a57fa640 1 -- 172.21.15.5:0/3435571751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f50ac005020 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b802c7e0 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50a57fa640 1 -- 172.21.15.5:0/3435571751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50ac005220 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c6d86740 10 client.18696 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cebc9c7d0 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.482+0000 7f50a57fa640 1 -- 172.21.15.5:0/3435571751 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b802c7e0 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50a57fa640 1 -- 172.21.15.5:0/3435571751 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50ac01d030 con 0x558cebf5b4d0 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c6d86740 10 client.18696 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c6d86740 10 client.18696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c6d86740 20 client.18696 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebf74bd0 0x558cebf76fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c6d86740 4 client.18696 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cebe5f930 con 0x558cebf74bd0 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebf74bd0 0x558cebf76fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebf74bd0 0x558cebf76fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c337e7628993a2b5 server_cookie=5e9a19fe33da26b4 in_seq=0 out_seq=0 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.483+0000 7f50a57fa640 10 client.18696 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50a57fa640 1 -- 172.21.15.5:0/3435571751 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cebe5f930 con 0x558cebf74bd0 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50a57fa640 10 client.18696 handle_command_reply: tid=0 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 client.18696 shutdown 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 msgr2=0x558cebf642f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf642f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f50ac003850 tx=0x7f50ac004730 comp rx=0 tx=0).stop 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 shutdown_connections 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebf74bd0 0x558cebf76fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf61c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3435571751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf642f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50a57fa640 0 client.18696 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 >> 172.21.15.5:0/3435571751 conn(0x558cebf55ba0 msgr2=0x558cebf578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 shutdown_connections 2024-04-22T13:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 1 -- 172.21.15.5:0/3435571751 wait complete. 2024-04-22T13:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.484+0000 7f50c6d86740 20 client.18696 trim_cache size 0 max 16384 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 Processor -- start 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 -- start start 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf63db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1adca0 con 0x558cebf5b4d0 2024-04-22T13:35:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1af7e0 con 0x558cebf5be80 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c6d86740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ae370 con 0x558cebf64a00 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37178/0 (socket says 172.21.15.5:37178) 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 -- 172.21.15.5:0/3230738042 learned_addr learned my addr 172.21.15.5:0/3230738042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf63db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 -- 172.21.15.5:0/3230738042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 msgr2=0x558cebf642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 -- 172.21.15.5:0/3230738042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 msgr2=0x558cebf63db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf63db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 -- 172.21.15.5:0/3230738042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cebf5bc20 con 0x558cebf5b4d0 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.498+0000 7f50c1731640 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf707d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f50ac015040 tx=0x7f50ac005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0a5a41132f1f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50a77fe640 1 -- 172.21.15.5:0/3230738042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac012bc0 con 0x558cebf5b4d0 2024-04-22T13:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50a77fe640 1 -- 172.21.15.5:0/3230738042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac012d60 con 0x558cebf5b4d0 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50a77fe640 1 -- 172.21.15.5:0/3230738042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0297b0 con 0x558cebf5b4d0 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 -- 172.21.15.5:0/3230738042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 msgr2=0x558cebf707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf707d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f50ac015040 tx=0x7f50ac005d30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 -- 172.21.15.5:0/3230738042 shutdown_connections 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf64a00 0x558cebf642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf5be80 0x558cebf63db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3230738042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf707d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 -- 172.21.15.5:0/3230738042 >> 172.21.15.5:0/3230738042 conn(0x558cebf55ba0 msgr2=0x558cebf54c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 -- 172.21.15.5:0/3230738042 shutdown_connections 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.499+0000 7f50c6d86740 1 -- 172.21.15.5:0/3230738042 wait complete. 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 -- start start 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 Processor -- start 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf5be80 0x558cebf5df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf64a00 0x558cebf5e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1af110 con 0x558cebf5b4d0 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1aea40 con 0x558cebf64a00 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1abca0 con 0x558cebf5be80 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c6d86740 10 client.0 _ll_register_callbacks cb 0x7f50ac0036a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c1731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c1731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf6cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37180/0 (socket says 172.21.15.5:37180) 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c1731640 1 -- 172.21.15.5:0/3952312716 learned_addr learned my addr 172.21.15.5:0/3952312716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.500+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf5be80 0x558cebf5df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1731640 1 -- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf5be80 msgr2=0x558cebf5df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1731640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf5be80 0x558cebf5df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1731640 1 -- 172.21.15.5:0/3952312716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf64a00 msgr2=0x558cebf5e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1731640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf64a00 0x558cebf5e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1731640 1 -- 172.21.15.5:0/3952312716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cebf5eb70 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf5be80 0x558cebf5df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c1731640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf6cfd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009d50 tx=0x7f50ac012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71acc327c3cf9e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50a57fa640 1 -- 172.21.15.5:0/3952312716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac041060 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50a57fa640 1 -- 172.21.15.5:0/3952312716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f50ac005480 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ac012d60 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50a57fa640 1 -- 172.21.15.5:0/3952312716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50ac005680 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c6d86740 10 client.18702 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cebfdaff0 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.501+0000 7f50a57fa640 1 -- 172.21.15.5:0/3952312716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac012d60 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50a57fa640 1 -- 172.21.15.5:0/3952312716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50ac01d030 con 0x558cebf5b4d0 2024-04-22T13:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 10 client.18702 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 10 client.18702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 20 client.18702 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebfdb600 0x558cebfdd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 4 client.18702 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cebecdc10 con 0x558cebfdb600 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebfdb600 0x558cebfdd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50a57fa640 10 client.18702 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c1f32640 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebfdb600 0x558cebfdd9f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28b457738ce20ef0 server_cookie=7ef33fdb2c00991 in_seq=0 out_seq=0 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50a57fa640 1 -- 172.21.15.5:0/3952312716 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cebecdc10 con 0x558cebfdb600 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50a57fa640 10 client.18702 handle_command_reply: tid=0 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 1 client.18702 shutdown 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 msgr2=0x558cebf6cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.502+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf6cfd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009d50 tx=0x7f50ac012c50 comp rx=0 tx=0).stop 2024-04-22T13:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 shutdown_connections 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x558cebfdb600 0x558cebfdd9f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558cebf64a00 0x558cebf5e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cebf5be80 0x558cebf5df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50a57fa640 0 client.18702 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 --2- 172.21.15.5:0/3952312716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cebf5b4d0 0x558cebf6cfd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 >> 172.21.15.5:0/3952312716 conn(0x558cebf55ba0 msgr2=0x558cebf5c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 shutdown_connections 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 1 -- 172.21.15.5:0/3952312716 wait complete. 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c6d86740 20 client.18702 trim_cache size 0 max 16384 2024-04-22T13:35:59.505 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50a80a3d20 msgr2=0x7f50a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50a80a3d20 0x7f50a80a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50b40ce110 tx=0x7f50b4087040 comp rx=0 tx=0).stop 2024-04-22T13:35:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 msgr2=0x7f50bc1a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 0x7f50bc1a4640 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f50b400e960 tx=0x7f50b400ee20 comp rx=0 tx=0).stop 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 shutdown_connections 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc19a090 0x7f50bc1a4640 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50a80a3d20 0x7f50a80a61e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50bc106160 0x7f50bc199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 --2- 172.21.15.5:0/947908524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50bc101090 0x7f50bc1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 >> 172.21.15.5:0/947908524 conn(0x7f50bc078000 msgr2=0x7f50bc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 shutdown_connections 2024-04-22T13:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.503+0000 7f50c4196640 1 -- 172.21.15.5:0/947908524 wait complete. 2024-04-22T13:35:59.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:35:59.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 -- 172.21.15.5:0/1436723714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101a40 msgr2=0x7f1e40105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 --2- 172.21.15.5:0/1436723714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101a40 0x7f1e40105a80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009900 tx=0x7f1e2c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 -- 172.21.15.5:0/1436723714 shutdown_connections 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 --2- 172.21.15.5:0/1436723714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101a40 0x7f1e40105a80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 --2- 172.21.15.5:0/1436723714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e40106160 0x7f1e4010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 --2- 172.21.15.5:0/1436723714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40101090 0x7f1e40101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 -- 172.21.15.5:0/1436723714 >> 172.21.15.5:0/1436723714 conn(0x7f1e40078000 msgr2=0x7f1e40078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 -- 172.21.15.5:0/1436723714 shutdown_connections 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.619+0000 7f1e46f23640 1 -- 172.21.15.5:0/1436723714 wait complete. 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 Processor -- start 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 -- start start 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 0x7f1e401986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40106160 0x7f1e40198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4019cf70 0x7f1e4019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401110a0 con 0x7f1e40101090 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40110f20 con 0x7f1e4019cf70 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40111220 con 0x7f1e40106160 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 0x7f1e401986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 0x7f1e401986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37196/0 (socket says 172.21.15.5:37196) 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2719061228 learned_addr learned my addr 172.21.15.5:0/2719061228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40106160 0x7f1e40198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2719061228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40106160 msgr2=0x7f1e40198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40106160 0x7f1e40198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2719061228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4019cf70 msgr2=0x7f1e4019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4019cf70 0x7f1e4019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2719061228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4019da60 con 0x7f1e40101090 2024-04-22T13:35:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40106160 0x7f1e40198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:59.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 0x7f1e401986b0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400d770 tx=0x7f1e3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e3dffb640 1 -- 172.21.15.5:0/2719061228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34017070 con 0x7f1e40101090 2024-04-22T13:35:59.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e3dffb640 1 -- 172.21.15.5:0/2719061228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e340054d0 con 0x7f1e40101090 2024-04-22T13:35:59.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.620+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4019dd50 con 0x7f1e40101090 2024-04-22T13:35:59.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.621+0000 7f1e3dffb640 1 -- 172.21.15.5:0/2719061228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34004c30 con 0x7f1e40101090 2024-04-22T13:35:59.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.621+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e401a5510 con 0x7f1e40101090 2024-04-22T13:35:59.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e3dffb640 1 -- 172.21.15.5:0/2719061228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1e34004e10 con 0x7f1e40101090 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 Processor -- start 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 -- start start 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7e7b291f0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f7e7ac64b0 con 0x55f7e7bc4e80 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e49bef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7e7b251b0 con 0x55f7e7bcda00 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e454c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e454c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37204/0 (socket says 172.21.15.5:37204) 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.623+0000 7f1e3dffb640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e300a3da0 0x7f1e300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 -- 172.21.15.5:0/2416613984 learned_addr learned my addr 172.21.15.5:0/2416613984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e300a3da0 0x7f1e300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e3dffb640 1 -- 172.21.15.5:0/2719061228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f1e340d0360 con 0x7f1e40101090 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 -- 172.21.15.5:0/2416613984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bcda00 msgr2=0x55f7e7bcfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 -- 172.21.15.5:0/2416613984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bc4e80 msgr2=0x55f7e7bcd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 -- 172.21.15.5:0/2416613984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7e7ab81b0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e454c0640 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc48b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1e38013850 tx=0x7f1e38013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36563874d3d0ebdb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e2affd640 1 -- 172.21.15.5:0/2416613984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38010cc0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e2affd640 1 -- 172.21.15.5:0/2416613984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e380044e0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.624+0000 7f1e2affd640 1 -- 172.21.15.5:0/2416613984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3801c5f0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 -- 172.21.15.5:0/2416613984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 msgr2=0x55f7e7bc48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc48b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1e38013850 tx=0x7f1e38013d10 comp rx=0 tx=0).stop 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e454c0640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e300a3da0 0x7f1e300a6260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e3801d000 tx=0x7f1e38018000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 -- 172.21.15.5:0/2416613984 shutdown_connections 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2416613984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc48b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 -- 172.21.15.5:0/2416613984 >> 172.21.15.5:0/2416613984 conn(0x55f7e7bbeba0 msgr2=0x55f7e7bbef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 -- 172.21.15.5:0/2416613984 shutdown_connections 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 -- 172.21.15.5:0/2416613984 wait complete. 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 -- start start 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.625+0000 7f1e49bef740 1 Processor -- start 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bcd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7e7bd0970 con 0x55f7e7bc44d0 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f7e7ae2500 con 0x55f7e7bcda00 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7e7bd0af0 con 0x55f7e7bc4e80 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e49bef740 10 client.0 _ll_register_callbacks cb 0x55f7e7ab0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bcd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bcd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37214/0 (socket says 172.21.15.5:37214) 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 -- 172.21.15.5:0/1481085819 learned_addr learned my addr 172.21.15.5:0/1481085819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e454c0640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 -- 172.21.15.5:0/1481085819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 msgr2=0x55f7e7bca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 -- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 msgr2=0x55f7e7bcace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 -- 172.21.15.5:0/1481085819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7e783b3f0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bcd340 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1e3409c230 tx=0x7f1e3406ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d8a46a4ac4d115 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.626+0000 7f1e1ffff640 1 -- 172.21.15.5:0/1481085819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3409c2a0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e1ffff640 1 -- 172.21.15.5:0/1481085819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f1e3409a5a0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e380044e0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e1ffff640 1 -- 172.21.15.5:0/1481085819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e340a0650 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 10 client.18714 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7e79057d0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e1ffff640 1 -- 172.21.15.5:0/1481085819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e380044e0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e1ffff640 1 -- 172.21.15.5:0/1481085819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1e340a4a80 con 0x55f7e7bc44d0 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 10 client.18714 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 10 client.18714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 20 client.18714 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7bdff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 4 client.18714 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7e7ac8930 con 0x55f7e7bddb10 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.627+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7bdff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.628+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7bdff80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=124f27e0cc49fe79 server_cookie=a9e22e71d9287e67 in_seq=0 out_seq=0 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.628+0000 7f1e1ffff640 10 client.18714 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.628+0000 7f1e1ffff640 1 -- 172.21.15.5:0/1481085819 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7e7ac8930 con 0x55f7e7bddb10 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.628+0000 7f1e1ffff640 10 client.18714 handle_command_reply: tid=0 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 client.18714 shutdown 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 msgr2=0x55f7e7bcd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bcd340 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1e3409c230 tx=0x7f1e3406ac50 comp rx=0 tx=0).stop 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 shutdown_connections 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7bdff80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 --2- 172.21.15.5:0/1481085819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bcd340 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e1ffff640 0 client.18714 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:59.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 >> 172.21.15.5:0/1481085819 conn(0x55f7e7bbeba0 msgr2=0x55f7e7bc08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 shutdown_connections 2024-04-22T13:35:59.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 1 -- 172.21.15.5:0/1481085819 wait complete. 2024-04-22T13:35:59.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.629+0000 7f1e49bef740 20 client.18714 trim_cache size 0 max 16384 2024-04-22T13:35:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 Processor -- start 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 -- start start 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bd9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ada50 con 0x55f7e7bc44d0 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401af590 con 0x55f7e7bcda00 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e49bef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ae120 con 0x55f7e7bc4e80 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bd9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bd9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37222/0 (socket says 172.21.15.5:37222) 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.642+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2570910951 learned_addr learned my addr 172.21.15.5:0/2570910951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e454c0640 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2570910951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 msgr2=0x55f7e7bcce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2570910951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 msgr2=0x55f7e7bcd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e44cbf640 1 -- 172.21.15.5:0/2570910951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7e7bcb050 con 0x55f7e7bc44d0 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bd9830 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009fd0 tx=0x7f1e3409bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe56d95e765a07d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e2a7fc640 1 -- 172.21.15.5:0/2570910951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e340cdee0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e2a7fc640 1 -- 172.21.15.5:0/2570910951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e340a4f10 con 0x55f7e7bc44d0 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.643+0000 7f1e2a7fc640 1 -- 172.21.15.5:0/2570910951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3409c070 con 0x55f7e7bc44d0 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- 172.21.15.5:0/2570910951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 msgr2=0x55f7e7bd9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bd9830 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009fd0 tx=0x7f1e3409bc00 comp rx=0 tx=0).stop 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- 172.21.15.5:0/2570910951 shutdown_connections 2024-04-22T13:35:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bcd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bcce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- 172.21.15.5:0/2570910951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bd9830 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- 172.21.15.5:0/2570910951 >> 172.21.15.5:0/2570910951 conn(0x55f7e7bbeba0 msgr2=0x55f7e7bc0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- 172.21.15.5:0/2570910951 shutdown_connections 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- 172.21.15.5:0/2570910951 wait complete. 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- start start 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 Processor -- start 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bc7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401aeec0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ae7f0 con 0x55f7e7bcda00 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401aba50 con 0x55f7e7bc4e80 2024-04-22T13:35:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.644+0000 7f1e49bef740 10 client.0 _ll_register_callbacks cb 0x7f1e3409bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37230/0 (socket says 172.21.15.5:37230) 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 -- 172.21.15.5:0/329500742 learned_addr learned my addr 172.21.15.5:0/329500742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bc7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e454c0640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 -- 172.21.15.5:0/329500742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 msgr2=0x55f7e7bc7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bc7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 -- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 msgr2=0x55f7e7bc7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bc7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 -- 172.21.15.5:0/329500742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7e7c43290 con 0x55f7e7bc44d0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e454c0640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bc7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e44cbf640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc7560 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1e340a4e10 tx=0x7f1e340057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2edaa7d36f4fa0a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e1ffff640 1 -- 172.21.15.5:0/329500742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34098910 con 0x55f7e7bc44d0 2024-04-22T13:35:59.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e1ffff640 1 -- 172.21.15.5:0/329500742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f1e34097ff0 con 0x55f7e7bc44d0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e340a4f10 con 0x55f7e7bc44d0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e1ffff640 1 -- 172.21.15.5:0/329500742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e340cde90 con 0x55f7e7bc44d0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e49bef740 10 client.18720 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7e7c43290 con 0x55f7e7bc44d0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.645+0000 7f1e1ffff640 1 -- 172.21.15.5:0/329500742 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e340a4f10 con 0x55f7e7bc44d0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e1ffff640 1 -- 172.21.15.5:0/329500742 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1e340a1870 con 0x55f7e7bc44d0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e49bef740 10 client.18720 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e49bef740 10 client.18720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e49bef740 20 client.18720 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e49bef740 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7c468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e49bef740 4 client.18720 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7e7b36c10 con 0x55f7e7bddb10 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7c468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e1ffff640 10 client.18720 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.646+0000 7f1e3ffff640 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7c468b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc941a8f4c6af133 server_cookie=8b2a16bac3ce5bb3 in_seq=0 out_seq=0 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e1ffff640 1 -- 172.21.15.5:0/329500742 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7e7b36c10 con 0x55f7e7bddb10 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e1ffff640 10 client.18720 handle_command_reply: tid=0 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 client.18720 shutdown 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 msgr2=0x55f7e7bc7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc7560 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1e340a4e10 tx=0x7f1e340057c0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 shutdown_connections 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55f7e7bddb10 0x55f7e7c468b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f7e7bcda00 0x55f7e7bc7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7e7bc4e80 0x55f7e7bc7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e1ffff640 0 client.18720 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 --2- 172.21.15.5:0/329500742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7e7bc44d0 0x55f7e7bc7560 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 >> 172.21.15.5:0/329500742 conn(0x55f7e7bbeba0 msgr2=0x55f7e7bbd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 shutdown_connections 2024-04-22T13:35:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 1 -- 172.21.15.5:0/329500742 wait complete. 2024-04-22T13:35:59.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e49bef740 20 client.18720 trim_cache size 0 max 16384 2024-04-22T13:35:59.650 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:59.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e300a3da0 msgr2=0x7f1e300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e46f23640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e300a3da0 0x7f1e300a6260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e3801d000 tx=0x7f1e38018000 comp rx=0 tx=0).stop 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 msgr2=0x7f1e401986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.647+0000 7f1e46f23640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 0x7f1e401986b0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400d770 tx=0x7f1e3400dc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 shutdown_connections 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1e300a3da0 0x7f1e300a6260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e4019cf70 0x7f1e4019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e40106160 0x7f1e40198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 --2- 172.21.15.5:0/2719061228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e40101090 0x7f1e401986b0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 >> 172.21.15.5:0/2719061228 conn(0x7f1e40078000 msgr2=0x7f1e4006a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 shutdown_connections 2024-04-22T13:35:59.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.648+0000 7f1e46f23640 1 -- 172.21.15.5:0/2719061228 wait complete. 2024-04-22T13:35:59.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 -- 172.21.15.5:0/1891876463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8102270 msgr2=0x7f77d8102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 --2- 172.21.15.5:0/1891876463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8102270 0x7f77d8102650 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009930 tx=0x7f77c802bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 -- 172.21.15.5:0/1891876463 shutdown_connections 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 --2- 172.21.15.5:0/1891876463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d81044f0 0x7f77d810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 --2- 172.21.15.5:0/1891876463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d8103b50 0x7f77d8103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 --2- 172.21.15.5:0/1891876463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8102270 0x7f77d8102650 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 -- 172.21.15.5:0/1891876463 >> 172.21.15.5:0/1891876463 conn(0x7f77d80fda10 msgr2=0x7f77d80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 -- 172.21.15.5:0/1891876463 shutdown_connections 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.762+0000 7f77dde6d640 1 -- 172.21.15.5:0/1891876463 wait complete. 2024-04-22T13:35:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 Processor -- start 2024-04-22T13:35:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 -- start start 2024-04-22T13:35:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d8102270 0x7f77d819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d8103b50 0x7f77d819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 0x7f77d8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81122e0 con 0x7f77d81044f0 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8112160 con 0x7f77d8102270 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77dde6d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8112460 con 0x7f77d8103b50 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 0x7f77d8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 0x7f77d8198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37250/0 (socket says 172.21.15.5:37250) 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d7fff640 1 -- 172.21.15.5:0/143143350 learned_addr learned my addr 172.21.15.5:0/143143350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d8103b50 0x7f77d819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d77fe640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d8102270 0x7f77d819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d7fff640 1 -- 172.21.15.5:0/143143350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d8103b50 msgr2=0x7f77d819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.763+0000 7f77d7fff640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d8103b50 0x7f77d819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d7fff640 1 -- 172.21.15.5:0/143143350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d8102270 msgr2=0x7f77d819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d7fff640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d8102270 0x7f77d819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d7fff640 1 -- 172.21.15.5:0/143143350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d8198f00 con 0x7f77d81044f0 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d7fff640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 0x7f77d8198640 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f77d000cad0 tx=0x7f77d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d4ff9640 1 -- 172.21.15.5:0/143143350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d000ae20 con 0x7f77d81044f0 2024-04-22T13:35:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d4ff9640 1 -- 172.21.15.5:0/143143350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d0005020 con 0x7f77d81044f0 2024-04-22T13:35:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d81991f0 con 0x7f77d81044f0 2024-04-22T13:35:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77d4ff9640 1 -- 172.21.15.5:0/143143350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d000f6b0 con 0x7f77d81044f0 2024-04-22T13:35:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.764+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d81a5500 con 0x7f77d81044f0 2024-04-22T13:35:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77d4ff9640 1 -- 172.21.15.5:0/143143350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f77d00051d0 con 0x7f77d81044f0 2024-04-22T13:35:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 Processor -- start 2024-04-22T13:35:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 -- start start 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c05143b0 0x5593c0514790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c051d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c051de60 0x5593c051e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593c04791f0 con 0x5593c0514d60 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593c04164b0 con 0x5593c051de60 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77e0a66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5593c04751b0 con 0x5593c05143b0 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c051d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c051d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37256/0 (socket says 172.21.15.5:37256) 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77d77fe640 1 -- 172.21.15.5:0/3650809086 learned_addr learned my addr 172.21.15.5:0/3650809086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c05143b0 0x5593c0514790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.767+0000 7f77d4ff9640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f77c40a3d20 0x7f77c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c051de60 0x5593c051e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d77fe640 1 -- 172.21.15.5:0/3650809086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c05143b0 msgr2=0x5593c0514790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d77fe640 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c05143b0 0x5593c0514790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d77fe640 1 -- 172.21.15.5:0/3650809086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c051de60 msgr2=0x5593c051e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d77fe640 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c051de60 0x5593c051e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d77fe640 1 -- 172.21.15.5:0/3650809086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c04081b0 con 0x5593c0514d60 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c051de60 0x5593c051e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d7fff640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f77c40a3d20 0x7f77c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d4ff9640 1 -- 172.21.15.5:0/143143350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f77d00d0960 con 0x7f77d81044f0 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77d77fe640 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c051d890 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f77c802c240 tx=0x7f77c802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2161e754c08779 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77cdffb640 1 -- 172.21.15.5:0/3650809086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8044020 con 0x5593c0514d60 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77cdffb640 1 -- 172.21.15.5:0/3650809086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c802cc90 con 0x5593c0514d60 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.768+0000 7f77cdffb640 1 -- 172.21.15.5:0/3650809086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80046c0 con 0x5593c0514d60 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 -- 172.21.15.5:0/3650809086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 msgr2=0x5593c051d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77d7fff640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f77c40a3d20 0x7f77c40a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ce160 tx=0x7f77d00673a0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c051d890 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f77c802c240 tx=0x7f77c802c650 comp rx=0 tx=0).stop 2024-04-22T13:35:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 -- 172.21.15.5:0/3650809086 shutdown_connections 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c051d890 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c051de60 0x5593c051e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3650809086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c05143b0 0x5593c0514790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 -- 172.21.15.5:0/3650809086 >> 172.21.15.5:0/3650809086 conn(0x5593c050eba0 msgr2=0x5593c050ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 -- 172.21.15.5:0/3650809086 shutdown_connections 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 -- 172.21.15.5:0/3650809086 wait complete. 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 -- start start 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.769+0000 7f77e0a66740 1 Processor -- start 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c0524200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c0514d60 0x5593c0521590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 0x5593c0521ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593c0524d40 con 0x5593c051de60 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5593c0432500 con 0x5593c05143b0 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5593c0524ec0 con 0x5593c0514d60 2024-04-22T13:35:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 10 client.0 _ll_register_callbacks cb 0x5593c0400b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 0x5593c0521ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 0x5593c0521ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37264/0 (socket says 172.21.15.5:37264) 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 -- 172.21.15.5:0/833065232 learned_addr learned my addr 172.21.15.5:0/833065232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d77fe640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c0514d60 0x5593c0521590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c0524200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 -- 172.21.15.5:0/833065232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c0514d60 msgr2=0x5593c0521590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c0514d60 0x5593c0521590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 -- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 msgr2=0x5593c0524200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c0524200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 -- 172.21.15.5:0/833065232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c018b3f0 con 0x5593c051de60 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d77fe640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c0514d60 0x5593c0521590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 0x5593c0521ad0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f77c0013dd0 tx=0x7f77c000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee99be38a4cbacb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77b6ffd640 1 -- 172.21.15.5:0/833065232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0020410 con 0x5593c051de60 2024-04-22T13:35:59.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77b6ffd640 1 -- 172.21.15.5:0/833065232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f77c00205b0 con 0x5593c051de60 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c802cc90 con 0x5593c051de60 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.770+0000 7f77b6ffd640 1 -- 172.21.15.5:0/833065232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77c00207b0 con 0x5593c051de60 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 10 client.18732 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593c02557d0 con 0x5593c051de60 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77b6ffd640 1 -- 172.21.15.5:0/833065232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c802cc90 con 0x5593c051de60 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77b6ffd640 1 -- 172.21.15.5:0/833065232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f77c0029930 con 0x5593c051de60 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 10 client.18732 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 10 client.18732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 20 client.18732 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c0531e90 0x5593c05342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 4 client.18732 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593c0418930 con 0x5593c0531e90 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.771+0000 7f77d77fe640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c0531e90 0x5593c05342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.772+0000 7f77b6ffd640 10 client.18732 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.772+0000 7f77d77fe640 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c0531e90 0x5593c05342e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfb5476a6e27187 server_cookie=848c32028e200fba in_seq=0 out_seq=0 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.772+0000 7f77b6ffd640 1 -- 172.21.15.5:0/833065232 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593c0418930 con 0x5593c0531e90 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.772+0000 7f77b6ffd640 10 client.18732 handle_command_reply: tid=0 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.772+0000 7f77e0a66740 1 client.18732 shutdown 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 msgr2=0x5593c0521ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 0x5593c0521ad0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f77c0013dd0 tx=0x7f77c000c7e0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 shutdown_connections 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c0531e90 0x5593c05342e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c051de60 0x5593c0521ad0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c0514d60 0x5593c0521590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77b6ffd640 0 client.18732 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 --2- 172.21.15.5:0/833065232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c0524200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 >> 172.21.15.5:0/833065232 conn(0x5593c050eba0 msgr2=0x5593c05108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 shutdown_connections 2024-04-22T13:35:59.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 1 -- 172.21.15.5:0/833065232 wait complete. 2024-04-22T13:35:59.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.773+0000 7f77e0a66740 20 client.18732 trim_cache size 0 max 16384 2024-04-22T13:35:59.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 Processor -- start 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 -- start start 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 0x5593c051c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c0514d60 0x5593c0523cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81ab9a0 con 0x5593c05143b0 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81ad4e0 con 0x5593c0514d60 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.786+0000 7f77e0a66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81ac070 con 0x5593c050eba0 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 0x5593c051c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 0x5593c051c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37270/0 (socket says 172.21.15.5:37270) 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 -- 172.21.15.5:0/3223820009 learned_addr learned my addr 172.21.15.5:0/3223820009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c0514d60 0x5593c0523cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 -- 172.21.15.5:0/3223820009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 msgr2=0x5593c052dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 -- 172.21.15.5:0/3223820009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c0514d60 msgr2=0x5593c0523cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c0514d60 0x5593c0523cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 -- 172.21.15.5:0/3223820009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c0508fe0 con 0x5593c05143b0 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77d77fe640 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 0x5593c051c800 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f77c802c240 tx=0x7f77c802c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4a2dca22e34e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77cd7fa640 1 -- 172.21.15.5:0/3223820009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8050070 con 0x5593c05143b0 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77cd7fa640 1 -- 172.21.15.5:0/3223820009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c80035f0 con 0x5593c05143b0 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.787+0000 7f77cd7fa640 1 -- 172.21.15.5:0/3223820009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8044040 con 0x5593c05143b0 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 -- 172.21.15.5:0/3223820009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 msgr2=0x5593c051c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 0x5593c051c800 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f77c802c240 tx=0x7f77c802c740 comp rx=0 tx=0).stop 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 -- 172.21.15.5:0/3223820009 shutdown_connections 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c05143b0 0x5593c051c800 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c0514d60 0x5593c0523cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 --2- 172.21.15.5:0/3223820009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 -- 172.21.15.5:0/3223820009 >> 172.21.15.5:0/3223820009 conn(0x5593c0521c30 msgr2=0x5593c0510830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 -- 172.21.15.5:0/3223820009 shutdown_connections 2024-04-22T13:35:59.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 -- 172.21.15.5:0/3223820009 wait complete. 2024-04-22T13:35:59.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 -- start start 2024-04-22T13:35:59.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.788+0000 7f77e0a66740 1 Processor -- start 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c052a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c0527150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81ace10 con 0x5593c0514d60 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81ac740 con 0x5593c05143b0 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81a99a0 con 0x5593c050eba0 2024-04-22T13:35:59.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 10 client.0 _ll_register_callbacks cb 0x7f77c0013a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c0527150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c0527150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37272/0 (socket says 172.21.15.5:37272) 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 -- 172.21.15.5:0/92742725 learned_addr learned my addr 172.21.15.5:0/92742725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d77fe640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c052a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 -- 172.21.15.5:0/92742725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 msgr2=0x5593c052a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 -- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 msgr2=0x5593c052a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c052a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 -- 172.21.15.5:0/92742725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c0527890 con 0x5593c0514d60 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77d6ffd640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c0527150 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f77c0013b20 tx=0x7f77c000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3f35b6eec495a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77b6ffd640 1 -- 172.21.15.5:0/92742725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c002f050 con 0x5593c0514d60 2024-04-22T13:35:59.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77b6ffd640 1 -- 172.21.15.5:0/92742725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f77c0020950 con 0x5593c0514d60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.789+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c80035f0 con 0x5593c0514d60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77b6ffd640 1 -- 172.21.15.5:0/92742725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77c0020b50 con 0x5593c0514d60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 10 client.18738 fetch_fsmap learned FSMap version 28 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593c0598e30 con 0x5593c0514d60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77b6ffd640 1 -- 172.21.15.5:0/92742725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c80035f0 con 0x5593c0514d60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77b6ffd640 1 -- 172.21.15.5:0/92742725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f77c0029480 con 0x5593c0514d60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 10 client.18738 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 20 client.18738 populate_metadata read hostname 'smithi005' 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c051de60 0x5593c059b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 4 client.18738 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593c0486c10 con 0x5593c051de60 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.790+0000 7f77d77fe640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c051de60 0x5593c059b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77b6ffd640 10 client.18738 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77d77fe640 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c051de60 0x5593c059b560 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725f440e4f5ed541 server_cookie=72e3c8044533dce5 in_seq=0 out_seq=0 2024-04-22T13:35:59.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77b6ffd640 1 -- 172.21.15.5:0/92742725 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593c0486c10 con 0x5593c051de60 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77b6ffd640 10 client.18738 handle_command_reply: tid=0 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 client.18738 shutdown 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 msgr2=0x5593c0527150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c0527150 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f77c0013b20 tx=0x7f77c000c7e0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 shutdown_connections 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5593c051de60 0x5593c059b560 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593c0514d60 0x5593c0527150 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5593c05143b0 0x5593c052a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 --2- 172.21.15.5:0/92742725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5593c050eba0 0x5593c052a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77b6ffd640 0 client.18738 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 >> 172.21.15.5:0/92742725 conn(0x5593c0521c30 msgr2=0x5593c050d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 shutdown_connections 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 1 -- 172.21.15.5:0/92742725 wait complete. 2024-04-22T13:35:59.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.791+0000 7f77e0a66740 20 client.18738 trim_cache size 0 max 16384 2024-04-22T13:35:59.793 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:35:59.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f77c40a3d20 msgr2=0x7f77c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f77c40a3d20 0x7f77c40a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ce160 tx=0x7f77d00673a0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 msgr2=0x7f77d8198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 0x7f77d8198640 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f77d000cad0 tx=0x7f77d000cf90 comp rx=0 tx=0).stop 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 shutdown_connections 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d81044f0 0x7f77d8198640 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f77c40a3d20 0x7f77c40a61e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d8103b50 0x7f77d819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 --2- 172.21.15.5:0/143143350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d8102270 0x7f77d819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 >> 172.21.15.5:0/143143350 conn(0x7f77d80fda10 msgr2=0x7f77d80fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 shutdown_connections 2024-04-22T13:35:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:35:59.792+0000 7f77dde6d640 1 -- 172.21.15.5:0/143143350 wait complete. 2024-04-22T13:36:00.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:35:59 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/663143031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:36:00.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:35:59 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/663143031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:36:00.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:35:59 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/663143031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:36:00.799 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:36:00.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:36:00.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:36:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 -- 172.21.15.5:0/3033681462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f410e8f0 msgr2=0x7f47f4110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/3033681462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f410e8f0 0x7f47f4110ce0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009ca0 tx=0x7f47ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 -- 172.21.15.5:0/3033681462 shutdown_connections 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/3033681462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f410e8f0 0x7f47f4110ce0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/3033681462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47f4101780 0x7f47f410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/3033681462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4100dd0 0x7f47f41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 -- 172.21.15.5:0/3033681462 >> 172.21.15.5:0/3033681462 conn(0x7f47f4077f70 msgr2=0x7f47f4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 -- 172.21.15.5:0/3033681462 shutdown_connections 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 -- 172.21.15.5:0/3033681462 wait complete. 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 Processor -- start 2024-04-22T13:36:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.924+0000 7f47fa2fd640 1 -- start start 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47fa2fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47f4100dd0 0x7f47f41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47fa2fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 0x7f47f41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47fa2fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f410e8f0 0x7f47f419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47fa2fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41144b0 con 0x7f47f4101780 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47fa2fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4114330 con 0x7f47f4100dd0 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47fa2fd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4114630 con 0x7f47f410e8f0 2024-04-22T13:36:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 0x7f47f41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 0x7f47f41a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37292/0 (socket says 172.21.15.5:37292) 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 -- 172.21.15.5:0/424191624 learned_addr learned my addr 172.21.15.5:0/424191624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f3fff640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47f4100dd0 0x7f47f41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f889a640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f410e8f0 0x7f47f419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 -- 172.21.15.5:0/424191624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f410e8f0 msgr2=0x7f47f419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f410e8f0 0x7f47f419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 -- 172.21.15.5:0/424191624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47f4100dd0 msgr2=0x7f47f41a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47f4100dd0 0x7f47f41a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 -- 172.21.15.5:0/424191624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f419b020 con 0x7f47f4101780 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f37fe640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 0x7f47f41a0c20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00e9f0 tx=0x7f47dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.925+0000 7f47f17fa640 1 -- 172.21.15.5:0/424191624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc00cd80 con 0x7f47f4101780 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.926+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f419b310 con 0x7f47f4101780 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.926+0000 7f47f17fa640 1 -- 172.21.15.5:0/424191624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0108e0 con 0x7f47f4101780 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.926+0000 7f47f17fa640 1 -- 172.21.15.5:0/424191624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc018980 con 0x7f47f4101780 2024-04-22T13:36:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.926+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a7530 con 0x7f47f4101780 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 Processor -- start 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47f17fa640 1 -- 172.21.15.5:0/424191624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f47dc010430 con 0x7f47f4101780 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 -- start start 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e155f8e0 0x5569e155fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1568840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1568e10 0x5569e156b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569e14c3f00 con 0x5569e1560290 2024-04-22T13:36:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5569e1461590 con 0x5569e1568e10 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569e14bfec0 con 0x5569e155f8e0 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1568840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1568840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37300/0 (socket says 172.21.15.5:37300) 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.928+0000 7f47f3fff640 1 -- 172.21.15.5:0/2652226793 learned_addr learned my addr 172.21.15.5:0/2652226793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f889a640 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e155f8e0 0x5569e155fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f37fe640 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1568e10 0x5569e156b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f3fff640 1 -- 172.21.15.5:0/2652226793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e155f8e0 msgr2=0x5569e155fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f3fff640 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e155f8e0 0x5569e155fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f3fff640 1 -- 172.21.15.5:0/2652226793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1568e10 msgr2=0x5569e156b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f3fff640 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1568e10 0x5569e156b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f3fff640 1 -- 172.21.15.5:0/2652226793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e129e7e0 con 0x5569e1560290 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f889a640 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e155f8e0 0x5569e155fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f17fa640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47e00a3d20 0x7f47e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f3fff640 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1568840 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009d20 tx=0x7f47e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d81aa4e81c32ce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:00.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f889a640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47e00a3d20 0x7f47e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47f17fa640 1 -- 172.21.15.5:0/424191624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f47dc0d0b20 con 0x7f47f4101780 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47ea7fc640 1 -- 172.21.15.5:0/2652226793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e401e070 con 0x5569e1560290 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47ea7fc640 1 -- 172.21.15.5:0/2652226793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e4019ae0 con 0x5569e1560290 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.929+0000 7f47ea7fc640 1 -- 172.21.15.5:0/2652226793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e400ea80 con 0x5569e1560290 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2652226793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 msgr2=0x5569e1568840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47f889a640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47e00a3d20 0x7f47e00a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47ec02c700 tx=0x7f47ec02e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1568840 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009d20 tx=0x7f47e4011810 comp rx=0 tx=0).stop 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2652226793 shutdown_connections 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1568840 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1568e10 0x5569e156b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2652226793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e155f8e0 0x5569e155fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2652226793 >> 172.21.15.5:0/2652226793 conn(0x5569e1559fb0 msgr2=0x5569e155a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2652226793 shutdown_connections 2024-04-22T13:36:00.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2652226793 wait complete. 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.930+0000 7f47fcfc8740 1 -- start start 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 Processor -- start 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 0x5569e1565de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1560290 0x5569e1566320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e1566860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569e156bd80 con 0x5569e155f8e0 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5569e147d580 con 0x5569e1560290 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569e156bf00 con 0x5569e1568e10 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47fcfc8740 10 client.0 _ll_register_callbacks cb 0x5569e1483650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 0x5569e1565de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 0x5569e1565de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37308/0 (socket says 172.21.15.5:37308) 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 -- 172.21.15.5:0/4134835384 learned_addr learned my addr 172.21.15.5:0/4134835384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f37fe640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1560290 0x5569e1566320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f889a640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e1566860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 -- 172.21.15.5:0/4134835384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 msgr2=0x5569e1566860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e1566860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 -- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1560290 msgr2=0x5569e1566320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1560290 0x5569e1566320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f3fff640 1 -- 172.21.15.5:0/4134835384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e12fef70 con 0x5569e155f8e0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.931+0000 7f47f37fe640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1560290 0x5569e1566320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 0x5569e1565de0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f47e4005e00 tx=0x7f47e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21573697c425f3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47d37fe640 1 -- 172.21.15.5:0/4134835384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e401e050 con 0x5569e155f8e0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47d37fe640 1 -- 172.21.15.5:0/4134835384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f47e4019630 con 0x5569e155f8e0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e4019ae0 con 0x5569e155f8e0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47d37fe640 1 -- 172.21.15.5:0/4134835384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47e4033050 con 0x5569e155f8e0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 10 client.18750 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569e1196170 con 0x5569e155f8e0 2024-04-22T13:36:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47d37fe640 1 -- 172.21.15.5:0/4134835384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e4019ae0 con 0x5569e155f8e0 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47d37fe640 1 -- 172.21.15.5:0/4134835384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47e40036d0 con 0x5569e155f8e0 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 10 client.18750 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 20 client.18750 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e1579280 0x5569e157b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 4 client.18750 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.932+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569e14d1920 con 0x5569e1579280 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.933+0000 7f47f37fe640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e1579280 0x5569e157b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.933+0000 7f47f37fe640 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e1579280 0x5569e157b670 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312a69766a3ad57c server_cookie=20af5263a9d76016 in_seq=0 out_seq=0 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.933+0000 7f47d37fe640 10 client.18750 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47d37fe640 1 -- 172.21.15.5:0/4134835384 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569e14d1920 con 0x5569e1579280 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47d37fe640 10 client.18750 handle_command_reply: tid=0 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 client.18750 shutdown 2024-04-22T13:36:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 msgr2=0x5569e1565de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 0x5569e1565de0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f47e4005e00 tx=0x7f47e4004270 comp rx=0 tx=0).stop 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 shutdown_connections 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e1579280 0x5569e157b670 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e1566860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e1560290 0x5569e1566320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47d37fe640 0 client.18750 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4134835384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e155f8e0 0x5569e1565de0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 >> 172.21.15.5:0/4134835384 conn(0x5569e1559fb0 msgr2=0x5569e155bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 shutdown_connections 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4134835384 wait complete. 2024-04-22T13:36:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.934+0000 7f47fcfc8740 20 client.18750 trim_cache size 0 max 16384 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 Processor -- start 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 -- start start 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e15684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1574c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e155bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41afe00 con 0x5569e1560290 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b1940 con 0x5569e155f8e0 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b04d0 con 0x5569e1568e10 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1574c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1574c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37320/0 (socket says 172.21.15.5:37320) 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 -- 172.21.15.5:0/2660915633 learned_addr learned my addr 172.21.15.5:0/2660915633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f889a640 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e155bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f3fff640 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e15684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 -- 172.21.15.5:0/2660915633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 msgr2=0x5569e155bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e155bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 -- 172.21.15.5:0/2660915633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 msgr2=0x5569e15684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e15684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f37fe640 1 -- 172.21.15.5:0/2660915633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e1560030 con 0x5569e1560290 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.952+0000 7f47f3fff640 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e15684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47f37fe640 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1574c40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ce870 tx=0x7f47dc0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8311714fd184cd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47e9ffb640 1 -- 172.21.15.5:0/2660915633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc09cac0 con 0x5569e1560290 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47e9ffb640 1 -- 172.21.15.5:0/2660915633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0d5040 con 0x5569e1560290 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47e9ffb640 1 -- 172.21.15.5:0/2660915633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc099130 con 0x5569e1560290 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2660915633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 msgr2=0x5569e1574c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1574c40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ce870 tx=0x7f47dc0bd040 comp rx=0 tx=0).stop 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2660915633 shutdown_connections 2024-04-22T13:36:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1560290 0x5569e1574c40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1568e10 0x5569e155bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/2660915633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e15684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2660915633 >> 172.21.15.5:0/2660915633 conn(0x5569e1559fb0 msgr2=0x5569e155b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2660915633 shutdown_connections 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.953+0000 7f47fcfc8740 1 -- 172.21.15.5:0/2660915633 wait complete. 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 -- start start 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 Processor -- start 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e156ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1560290 0x5569e156d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 0x5569e156d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b1270 con 0x5569e1568e10 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41b0ba0 con 0x5569e155f8e0 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41ade00 con 0x5569e1560290 2024-04-22T13:36:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47fcfc8740 10 client.0 _ll_register_callbacks cb 0x7f47e401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f889a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 0x5569e156d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f889a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 0x5569e156d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37322/0 (socket says 172.21.15.5:37322) 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f889a640 1 -- 172.21.15.5:0/4021562471 learned_addr learned my addr 172.21.15.5:0/4021562471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f37fe640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1560290 0x5569e156d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e156ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f889a640 1 -- 172.21.15.5:0/4021562471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1560290 msgr2=0x5569e156d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.954+0000 7f47f889a640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1560290 0x5569e156d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47f889a640 1 -- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 msgr2=0x5569e156ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47f889a640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e156ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47f889a640 1 -- 172.21.15.5:0/4021562471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e15de970 con 0x5569e1568e10 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e156ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47f889a640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 0x5569e156d8d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009ca0 tx=0x7f47ec02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61394af66bed3914 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47d37fe640 1 -- 172.21.15.5:0/4021562471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec051030 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47dc0d5040 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47d37fe640 1 -- 172.21.15.5:0/4021562471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f47ec003970 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47d37fe640 1 -- 172.21.15.5:0/4021562471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47ec039040 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47fcfc8740 10 client.18756 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569e15dfc20 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47d37fe640 1 -- 172.21.15.5:0/4021562471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0d5040 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.955+0000 7f47d37fe640 1 -- 172.21.15.5:0/4021562471 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47ec05f480 con 0x5569e1568e10 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47fcfc8740 10 client.18756 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47fcfc8740 10 client.18756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47fcfc8740 20 client.18756 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e15e0340 0x5569e15e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47fcfc8740 4 client.18756 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5569e14cb790 con 0x5569e15e0340 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e15e0340 0x5569e15e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47f3fff640 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e15e0340 0x5569e15e2760 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd07de95ab498d5 server_cookie=ee8241bf994fc6a4 in_seq=0 out_seq=0 2024-04-22T13:36:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.956+0000 7f47d37fe640 10 client.18756 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47d37fe640 1 -- 172.21.15.5:0/4021562471 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5569e14cb790 con 0x5569e15e0340 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47d37fe640 10 client.18756 handle_command_reply: tid=0 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 client.18756 shutdown 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 msgr2=0x5569e156d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 0x5569e156d8d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009ca0 tx=0x7f47ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 shutdown_connections 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5569e15e0340 0x5569e15e2760 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569e1568e10 0x5569e156d8d0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569e1560290 0x5569e156d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 --2- 172.21.15.5:0/4021562471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5569e155f8e0 0x5569e156ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 >> 172.21.15.5:0/4021562471 conn(0x5569e1559fb0 msgr2=0x5569e155bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47d37fe640 0 client.18756 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 shutdown_connections 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 1 -- 172.21.15.5:0/4021562471 wait complete. 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.958+0000 7f47fcfc8740 20 client.18756 trim_cache size 0 max 16384 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e3d4570c-e889-4c68-bd63-84d8bb7964a2", 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:36:00.959 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:36:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47e00a3d20 msgr2=0x7f47e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47e00a3d20 0x7f47e00a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47ec02c700 tx=0x7f47ec02e040 comp rx=0 tx=0).stop 2024-04-22T13:36:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 msgr2=0x7f47f41a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 0x7f47f41a0c20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00e9f0 tx=0x7f47dc00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 shutdown_connections 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f4101780 0x7f47f41a0c20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f47e00a3d20 0x7f47e00a61e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f410e8f0 0x7f47f419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 --2- 172.21.15.5:0/424191624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47f4100dd0 0x7f47f41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 >> 172.21.15.5:0/424191624 conn(0x7f47f4077f70 msgr2=0x7f47f410fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 shutdown_connections 2024-04-22T13:36:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:00.959+0000 7f47fa2fd640 1 -- 172.21.15.5:0/424191624 wait complete. 2024-04-22T13:36:00.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:36:00.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:36:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:00 smithi107 ceph-mon[31859]: pgmap v273: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:00 smithi005 ceph-mon[28006]: pgmap v273: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.074+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1227960677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069f20 msgr2=0x7f4f44106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.074+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1227960677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069f20 0x7f4f44106b50 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009900 tx=0x7f4f3802bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1227960677 shutdown_connections 2024-04-22T13:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1227960677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069f20 0x7f4f44106b50 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1227960677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f44107230 0x7f4f4410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1227960677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f44069570 0x7f4f44069950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1227960677 >> 172.21.15.5:0/1227960677 conn(0x7f4f44077fa0 msgr2=0x7f4f440783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1227960677 shutdown_connections 2024-04-22T13:36:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1227960677 wait complete. 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 Processor -- start 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.075+0000 7f4f48ef1640 1 -- start start 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f48ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 0x7f4f44198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f48ef1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f44107230 0x7f4f44198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f48ef1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4419cff0 0x7f4f4419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f48ef1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44112170 con 0x7f4f44069570 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f48ef1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44111ff0 con 0x7f4f4419cff0 2024-04-22T13:36:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f48ef1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441122f0 con 0x7f4f44107230 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 0x7f4f44198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 0x7f4f44198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37340/0 (socket says 172.21.15.5:37340) 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 -- 172.21.15.5:0/1732367296 learned_addr learned my addr 172.21.15.5:0/1732367296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f44107230 0x7f4f44198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4419cff0 0x7f4f4419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 -- 172.21.15.5:0/1732367296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f44107230 msgr2=0x7f4f44198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f44107230 0x7f4f44198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 -- 172.21.15.5:0/1732367296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4419cff0 msgr2=0x7f4f4419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4419cff0 0x7f4f4419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 -- 172.21.15.5:0/1732367296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4419dae0 con 0x7f4f44069570 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4419cff0 0x7f4f4419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f4259c640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 0x7f4f44198730 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00c890 tx=0x7f4f3c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.076+0000 7f4f337fe640 1 -- 172.21.15.5:0/1732367296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c004220 con 0x7f4f44069570 2024-04-22T13:36:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.077+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4419ddd0 con 0x7f4f44069570 2024-04-22T13:36:01.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.077+0000 7f4f337fe640 1 -- 172.21.15.5:0/1732367296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0043c0 con 0x7f4f44069570 2024-04-22T13:36:01.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.077+0000 7f4f337fe640 1 -- 172.21.15.5:0/1732367296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c004a00 con 0x7f4f44069570 2024-04-22T13:36:01.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.077+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a55e0 con 0x7f4f44069570 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.079+0000 7f4f337fe640 1 -- 172.21.15.5:0/1732367296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f4f3c0079b0 con 0x7f4f44069570 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.079+0000 7f4f4bbb6740 1 Processor -- start 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.079+0000 7f4f4bbb6740 1 -- start start 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd4883540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd4885e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bd47df1e0 con 0x558bd487a580 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558bd477c390 con 0x558bd487aea0 2024-04-22T13:36:01.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558bd47db1a0 con 0x558bd4883a80 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37350/0 (socket says 172.21.15.5:37350) 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 -- 172.21.15.5:0/1081758065 learned_addr learned my addr 172.21.15.5:0/1081758065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f337fe640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4f2c0a3d20 0x7f4f2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f4259c640 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd4883540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd4885e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4f2c0a3d20 0x7f4f2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 -- 172.21.15.5:0/1081758065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 msgr2=0x558bd4885e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd4885e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 -- 172.21.15.5:0/1081758065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 msgr2=0x558bd4883540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd4883540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f337fe640 1 -- 172.21.15.5:0/1732367296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f4f3c0d0660 con 0x7f4f44069570 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 -- 172.21.15.5:0/1081758065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd45bb7d0 con 0x558bd487a580 2024-04-22T13:36:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd4885e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487a960 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4f34013910 tx=0x7f4f34013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68340953bfd21fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.080+0000 7f4f30ff9640 1 -- 172.21.15.5:0/1081758065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34004b20 con 0x558bd487a580 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f30ff9640 1 -- 172.21.15.5:0/1081758065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34004cc0 con 0x558bd487a580 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f30ff9640 1 -- 172.21.15.5:0/1081758065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34005750 con 0x558bd487a580 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4f2c0a3d20 0x7f4f2c0a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f34020000 tx=0x7f4f3401f000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/1081758065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 msgr2=0x558bd487a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487a960 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4f34013910 tx=0x7f4f34013dd0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/1081758065 shutdown_connections 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd4885e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd4883540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/1081758065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487a960 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/1081758065 >> 172.21.15.5:0/1081758065 conn(0x558bd4874cb0 msgr2=0x558bd4875090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/1081758065 shutdown_connections 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.081+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/1081758065 wait complete. 2024-04-22T13:36:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 -- start start 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 Processor -- start 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487a580 0x558bd4880ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd4881020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 0x558bd4881560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bd4886870 con 0x558bd4883a80 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558bd47982f0 con 0x558bd487a580 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558bd48869f0 con 0x558bd487aea0 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4bbb6740 10 client.0 _ll_register_callbacks cb 0x558bd47a5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 0x558bd4881560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 0x558bd4881560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37360/0 (socket says 172.21.15.5:37360) 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 -- 172.21.15.5:0/3667405498 learned_addr learned my addr 172.21.15.5:0/3667405498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487a580 0x558bd4880ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd4881020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 -- 172.21.15.5:0/3667405498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 msgr2=0x558bd4881020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd4881020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 -- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487a580 msgr2=0x558bd4880ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487a580 0x558bd4880ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f42d9d640 1 -- 172.21.15.5:0/3667405498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd44f13f0 con 0x558bd4883a80 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd4881020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.082+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487a580 0x558bd4880ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 0x558bd4881560 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4f340092f0 tx=0x7f4f34009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca0b4d3c2fd82c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f29ffb640 1 -- 172.21.15.5:0/3667405498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34004ac0 con 0x558bd4883a80 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f29ffb640 1 -- 172.21.15.5:0/3667405498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f4f34022070 con 0x558bd4883a80 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f34004cc0 con 0x558bd4883a80 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f29ffb640 1 -- 172.21.15.5:0/3667405498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f340056e0 con 0x558bd4883a80 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 10 client.18768 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558bd44b10f0 con 0x558bd4883a80 2024-04-22T13:36:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f29ffb640 1 -- 172.21.15.5:0/3667405498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34004cc0 con 0x558bd4883a80 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f29ffb640 1 -- 172.21.15.5:0/3667405498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f34026780 con 0x558bd4883a80 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 10 client.18768 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 10 client.18768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 20 client.18768 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48939f0 0x558bd4895de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 4 client.18768 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.083+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bd47ecc00 con 0x558bd48939f0 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.084+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48939f0 0x558bd4895de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.084+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48939f0 0x558bd4895de0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6a5815ec0735 server_cookie=24f00fc805789100 in_seq=0 out_seq=0 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.084+0000 7f4f29ffb640 10 client.18768 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f29ffb640 1 -- 172.21.15.5:0/3667405498 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bd47ecc00 con 0x558bd48939f0 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f29ffb640 10 client.18768 handle_command_reply: tid=0 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 client.18768 shutdown 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 msgr2=0x558bd4881560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 0x558bd4881560 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4f340092f0 tx=0x7f4f34009850 comp rx=0 tx=0).stop 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 shutdown_connections 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48939f0 0x558bd4895de0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd4883a80 0x558bd4881560 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd4881020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f29ffb640 0 client.18768 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3667405498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487a580 0x558bd4880ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 >> 172.21.15.5:0/3667405498 conn(0x558bd4874cb0 msgr2=0x558bd4876a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 shutdown_connections 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3667405498 wait complete. 2024-04-22T13:36:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.085+0000 7f4f4bbb6740 20 client.18768 trim_cache size 0 max 16384 2024-04-22T13:36:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 Processor -- start 2024-04-22T13:36:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 -- start start 2024-04-22T13:36:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd4883210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd488f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd48811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441adc90 con 0x558bd487a580 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441af7d0 con 0x558bd487aea0 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441ae360 con 0x558bd4883a80 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd4883210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd4883210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37368/0 (socket says 172.21.15.5:37368) 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f4259c640 1 -- 172.21.15.5:0/3011833691 learned_addr learned my addr 172.21.15.5:0/3011833691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.103+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd48811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4259c640 1 -- 172.21.15.5:0/3011833691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 msgr2=0x558bd48811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd48811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4259c640 1 -- 172.21.15.5:0/3011833691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 msgr2=0x558bd488f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd488f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4259c640 1 -- 172.21.15.5:0/3011833691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd487acd0 con 0x558bd487a580 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f42d9d640 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd48811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:01.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4259c640 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd4883210 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005fd0 tx=0x7f4f3c06dfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422b1862adf0be71 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f2bfff640 1 -- 172.21.15.5:0/3011833691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c01d040 con 0x558bd487a580 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f2bfff640 1 -- 172.21.15.5:0/3011833691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0a0950 con 0x558bd487a580 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f2bfff640 1 -- 172.21.15.5:0/3011833691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0cdfd0 con 0x558bd487a580 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3011833691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 msgr2=0x558bd4883210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd4883210 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005fd0 tx=0x7f4f3c06dfe0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3011833691 shutdown_connections 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd4883a80 0x558bd48811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd487aea0 0x558bd488f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.104+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/3011833691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd4883210 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3011833691 >> 172.21.15.5:0/3011833691 conn(0x558bd4874cb0 msgr2=0x558bd4880aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3011833691 shutdown_connections 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/3011833691 wait complete. 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- start start 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 Processor -- start 2024-04-22T13:36:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd487e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd4883a80 0x558bd48f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441af100 con 0x558bd487a580 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441aea30 con 0x558bd4883a80 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441abc90 con 0x558bd487aea0 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4bbb6740 10 client.0 _ll_register_callbacks cb 0x7f4f34013710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37380/0 (socket says 172.21.15.5:37380) 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 -- 172.21.15.5:0/995336905 learned_addr learned my addr 172.21.15.5:0/995336905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 -- 172.21.15.5:0/995336905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 msgr2=0x558bd487e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd487e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 -- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd4883a80 msgr2=0x558bd48f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd4883a80 0x558bd48f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.105+0000 7f4f4259c640 1 -- 172.21.15.5:0/995336905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd48f91f0 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4259c640 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487dbe0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c09c150 tx=0x7f4f3c098840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3489bf19acd011c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f29ffb640 1 -- 172.21.15.5:0/995336905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c01d070 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f29ffb640 1 -- 172.21.15.5:0/995336905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f4f3c098c10 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c0a0950 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f29ffb640 1 -- 172.21.15.5:0/995336905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f3c085040 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 10 client.18774 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558bd48fa330 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f29ffb640 1 -- 172.21.15.5:0/995336905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0a0950 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f29ffb640 1 -- 172.21.15.5:0/995336905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f3c085ce0 con 0x558bd487a580 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 10 client.18774 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 10 client.18774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 20 client.18774 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48faa80 0x558bd48fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 4 client.18774 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:36:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.106+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bd47e6a70 con 0x558bd48faa80 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48faa80 0x558bd48fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f29ffb640 10 client.18774 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f41d9b640 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48faa80 0x558bd48fcec0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1126bb73c71edec7 server_cookie=deb07a48ad9a2904 in_seq=0 out_seq=0 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f29ffb640 1 -- 172.21.15.5:0/995336905 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558bd47e6a70 con 0x558bd48faa80 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f29ffb640 10 client.18774 handle_command_reply: tid=0 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 client.18774 shutdown 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 msgr2=0x558bd487dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487dbe0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c09c150 tx=0x7f4f3c098840 comp rx=0 tx=0).stop 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 shutdown_connections 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x558bd48faa80 0x558bd48fcec0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558bd4883a80 0x558bd48f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f29ffb640 0 client.18774 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bd487aea0 0x558bd487e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 --2- 172.21.15.5:0/995336905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bd487a580 0x558bd487dbe0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.107+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 >> 172.21.15.5:0/995336905 conn(0x558bd4874cb0 msgr2=0x558bd4876880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 shutdown_connections 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f4bbb6740 1 -- 172.21.15.5:0/995336905 wait complete. 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f4bbb6740 20 client.18774 trim_cache size 0 max 16384 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: "e3d4570c-e889-4c68-bd63-84d8bb7964a2": { 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e3d4570c-e889-4c68-bd63-84d8bb7964a2", 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:36:01.109 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4f2c0a3d20 msgr2=0x7f4f2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4f2c0a3d20 0x7f4f2c0a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f34020000 tx=0x7f4f3401f000 comp rx=0 tx=0).stop 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 msgr2=0x7f4f44198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 0x7f4f44198730 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00c890 tx=0x7f4f3c00cd50 comp rx=0 tx=0).stop 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 shutdown_connections 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f4f2c0a3d20 0x7f4f2c0a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4419cff0 0x7f4f4419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f44107230 0x7f4f44198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 --2- 172.21.15.5:0/1732367296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f44069570 0x7f4f44198730 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 >> 172.21.15.5:0/1732367296 conn(0x7f4f44077fa0 msgr2=0x7f4f440ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 shutdown_connections 2024-04-22T13:36:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:01.108+0000 7f4f48ef1640 1 -- 172.21.15.5:0/1732367296 wait complete. 2024-04-22T13:36:01.114 INFO:tasks.cephfs.filesystem:scrub status for tag:e3d4570c-e889-4c68-bd63-84d8bb7964a2 - {'path': '/', 'tag': 'e3d4570c-e889-4c68-bd63-84d8bb7964a2', 'options': 'recursive,force'} 2024-04-22T13:36:01.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:00 smithi049 ceph-mon[31379]: pgmap v273: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:02.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:01 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:36:02.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:01 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:02.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:01 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:36:02.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:01 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:02.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:01 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:36:02.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:01 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:02.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:01 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:36:02.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:01 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:02.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:01 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:36:02.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:01 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:02.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:01 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:36:02.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:01 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:03.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:02 smithi005 ceph-mon[28006]: pgmap v274: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:36:03.185 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:02 smithi107 ceph-mon[31859]: pgmap v274: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:36:03.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:02 smithi049 ceph-mon[31379]: pgmap v274: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:36:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:04 smithi005 ceph-mon[28006]: pgmap v275: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:36:05.072 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:04 smithi049 ceph-mon[31379]: pgmap v275: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:36:05.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:04 smithi107 ceph-mon[31859]: pgmap v275: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:36:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:05 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:36:06.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:05 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:36:06.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:05 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:36:07.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:06 smithi005 ceph-mon[28006]: pgmap v276: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:36:07.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:06 smithi005 ceph-mon[28006]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T13:36:07.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:06 smithi049 ceph-mon[31379]: pgmap v276: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:36:07.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:06 smithi049 ceph-mon[31379]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T13:36:07.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:06 smithi107 ceph-mon[31859]: pgmap v276: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:36:07.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:06 smithi107 ceph-mon[31859]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T13:36:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:08 smithi005 ceph-mon[28006]: pgmap v278: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:36:09.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:08 smithi049 ceph-mon[31379]: pgmap v278: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:36:09.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:08 smithi107 ceph-mon[31859]: pgmap v278: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:36:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:10 smithi005 ceph-mon[28006]: pgmap v279: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:36:11.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:10 smithi049 ceph-mon[31379]: pgmap v279: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:36:11.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:10 smithi107 ceph-mon[31859]: pgmap v279: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-22T13:36:13.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:12 smithi005 ceph-mon[28006]: pgmap v280: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:36:13.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:12 smithi005 ceph-mon[28006]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T13:36:13.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:12 smithi049 ceph-mon[31379]: pgmap v280: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:36:13.271 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:12 smithi049 ceph-mon[31379]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T13:36:13.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:12 smithi107 ceph-mon[31859]: pgmap v280: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:36:13.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:12 smithi107 ceph-mon[31859]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T13:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:14 smithi005 ceph-mon[28006]: pgmap v282: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:36:15.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:14 smithi049 ceph-mon[31379]: pgmap v282: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:36:15.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:14 smithi107 ceph-mon[31859]: pgmap v282: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:36:17.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:16 smithi049 ceph-mon[31379]: pgmap v283: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-04-22T13:36:17.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:16 smithi107 ceph-mon[31859]: pgmap v283: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-04-22T13:36:17.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:16 smithi005 ceph-mon[28006]: pgmap v283: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-04-22T13:36:19.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:18 smithi049 ceph-mon[31379]: pgmap v284: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:36:19.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:18 smithi107 ceph-mon[31859]: pgmap v284: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:36:19.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:18 smithi005 ceph-mon[28006]: pgmap v284: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:36:21.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:20 smithi049 ceph-mon[31379]: pgmap v285: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:36:21.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:20 smithi107 ceph-mon[31859]: pgmap v285: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:36:21.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:20 smithi005 ceph-mon[28006]: pgmap v285: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:36:23.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:22 smithi049 ceph-mon[31379]: pgmap v286: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:36:23.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:22 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:36:23.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:22 smithi107 ceph-mon[31859]: pgmap v286: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:36:23.283 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:22 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:36:23.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:22 smithi005 ceph-mon[28006]: pgmap v286: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:36:23.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:22 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:36:24.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:23 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:36:24.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:23 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:36:24.270 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:23 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:36:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:23 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:36:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:23 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:36:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:23 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:36:24.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:23 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:36:24.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:23 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:36:24.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:23 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:36:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:25 smithi049 ceph-mon[31379]: pgmap v287: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-22T13:36:25.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:25 smithi107 ceph-mon[31859]: pgmap v287: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-22T13:36:25.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:25 smithi005 ceph-mon[28006]: pgmap v287: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-22T13:36:27.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:27 smithi049 ceph-mon[31379]: pgmap v288: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:36:27.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:27 smithi107 ceph-mon[31859]: pgmap v288: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:36:27.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:27 smithi005 ceph-mon[28006]: pgmap v288: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:36:29.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:29 smithi049 ceph-mon[31379]: pgmap v289: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:29.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:29 smithi107 ceph-mon[31859]: pgmap v289: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:29 smithi005 ceph-mon[28006]: pgmap v289: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:31.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:36:31.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 -- 172.21.15.5:0/964086752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8069560 msgr2=0x7f98c8069940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/964086752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8069560 0x7f98c8069940 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009900 tx=0x7f98b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 -- 172.21.15.5:0/964086752 shutdown_connections 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/964086752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c81072a0 0x7f98c810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/964086752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c8069f10 0x7f98c8106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/964086752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8069560 0x7f98c8069940 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 -- 172.21.15.5:0/964086752 >> 172.21.15.5:0/964086752 conn(0x7f98c8077fa0 msgr2=0x7f98c80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 -- 172.21.15.5:0/964086752 shutdown_connections 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 -- 172.21.15.5:0/964086752 wait complete. 2024-04-22T13:36:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.238+0000 7f98cd1e6640 1 Processor -- start 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 -- start start 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c8069560 0x7f98c81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 0x7f98c8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c819cfa0 0x7f98c819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81121a0 con 0x7f98c81072a0 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8112020 con 0x7f98c819cfa0 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98cd1e6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8112320 con 0x7f98c8069560 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 0x7f98c8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 0x7f98c8198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50924/0 (socket says 172.21.15.5:50924) 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 -- 172.21.15.5:0/2915665288 learned_addr learned my addr 172.21.15.5:0/2915665288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c8069560 0x7f98c81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c759e640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c819cfa0 0x7f98c819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 -- 172.21.15.5:0/2915665288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c8069560 msgr2=0x7f98c81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c8069560 0x7f98c81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 -- 172.21.15.5:0/2915665288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c819cfa0 msgr2=0x7f98c819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c819cfa0 0x7f98c819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.239+0000 7f98c659c640 1 -- 172.21.15.5:0/2915665288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c819da90 con 0x7f98c81072a0 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.240+0000 7f98c659c640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 0x7f98c8198c20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00d750 tx=0x7f98bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.240+0000 7f98affff640 1 -- 172.21.15.5:0/2915665288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc017070 con 0x7f98c81072a0 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.240+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c819dd80 con 0x7f98c81072a0 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.240+0000 7f98affff640 1 -- 172.21.15.5:0/2915665288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc0054d0 con 0x7f98c81072a0 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.240+0000 7f98affff640 1 -- 172.21.15.5:0/2915665288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc004c30 con 0x7f98c81072a0 2024-04-22T13:36:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.240+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81a5570 con 0x7f98c81072a0 2024-04-22T13:36:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98affff640 1 -- 172.21.15.5:0/2915665288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f98bc004dd0 con 0x7f98c81072a0 2024-04-22T13:36:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 Processor -- start 2024-04-22T13:36:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 -- start start 2024-04-22T13:36:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bab580 0x556609bab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 0x556609bb6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556609b101e0 con 0x556609bb4a80 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556609aad390 con 0x556609bab580 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556609b0c1a0 con 0x556609babea0 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 0x556609bb6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 0x556609bb6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50930/0 (socket says 172.21.15.5:50930) 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98c659c640 1 -- 172.21.15.5:0/4003316113 learned_addr learned my addr 172.21.15.5:0/4003316113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.243+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c759e640 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bab580 0x556609bab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98affff640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98b00a3d20 0x7f98b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c659c640 1 -- 172.21.15.5:0/4003316113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 msgr2=0x556609bb4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c659c640 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c659c640 1 -- 172.21.15.5:0/4003316113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bab580 msgr2=0x556609bab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c659c640 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bab580 0x556609bab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c659c640 1 -- 172.21.15.5:0/4003316113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566098ec7d0 con 0x556609bb4a80 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c759e640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98b00a3d20 0x7f98b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98affff640 1 -- 172.21.15.5:0/2915665288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f98bc0d01c0 con 0x7f98c81072a0 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98c659c640 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 0x556609bb6e70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0cd7f0 tx=0x7f98bc0c7210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be70ad6ab55335d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98ad7fa640 1 -- 172.21.15.5:0/4003316113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc020070 con 0x556609bb4a80 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98ad7fa640 1 -- 172.21.15.5:0/4003316113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc0cab30 con 0x556609bb4a80 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.244+0000 7f98ad7fa640 1 -- 172.21.15.5:0/4003316113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0cad50 con 0x556609bb4a80 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 -- 172.21.15.5:0/4003316113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 msgr2=0x556609bb6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 0x556609bb6e70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0cd7f0 tx=0x7f98bc0c7210 comp rx=0 tx=0).stop 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98c759e640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98b00a3d20 0x7f98b00a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009370 tx=0x7f98c0004a40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 -- 172.21.15.5:0/4003316113 shutdown_connections 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bb4a80 0x556609bb6e70 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/4003316113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bab580 0x556609bab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 -- 172.21.15.5:0/4003316113 >> 172.21.15.5:0/4003316113 conn(0x556609ba5cb0 msgr2=0x556609ba6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 -- 172.21.15.5:0/4003316113 shutdown_connections 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 -- 172.21.15.5:0/4003316113 wait complete. 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 -- start start 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.245+0000 7f98cfdd2740 1 Processor -- start 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bb1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609bb2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556609bb77b0 con 0x556609bab580 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556609ac92f0 con 0x556609bb4a80 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556609bb7930 con 0x556609babea0 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98cfdd2740 10 client.0 _ll_register_callbacks cb 0x556609ad6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bb1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bb1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50936/0 (socket says 172.21.15.5:50936) 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 -- 172.21.15.5:0/263487216 learned_addr learned my addr 172.21.15.5:0/263487216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c659c640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c759e640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609bb2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 -- 172.21.15.5:0/263487216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 msgr2=0x556609bb1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 -- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 msgr2=0x556609bb2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609bb2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 -- 172.21.15.5:0/263487216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566098223f0 con 0x556609bab580 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bb1a80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f98b8035000 tx=0x7f98b802c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a870cb47e8a354c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.246+0000 7f98aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98aaff5640 1 -- 172.21.15.5:0/263487216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8005030 con 0x556609bab580 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98aaff5640 1 -- 172.21.15.5:0/263487216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f98b80051d0 con 0x556609bab580 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556609b1dc00 con 0x556609bab580 2024-04-22T13:36:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98aaff5640 1 -- 172.21.15.5:0/263487216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98b80053d0 con 0x556609bab580 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 10 client.18786 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566097e20f0 con 0x556609bab580 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98aaff5640 1 -- 172.21.15.5:0/263487216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556609b1dc00 con 0x556609bab580 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98aaff5640 1 -- 172.21.15.5:0/263487216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f98b8003780 con 0x556609bab580 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 10 client.18786 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 10 client.18786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 20 client.18786 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609bc48d0 0x556609bc6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 4 client.18786 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.247+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556609b17a70 con 0x556609bc48d0 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.248+0000 7f98c659c640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609bc48d0 0x556609bc6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.248+0000 7f98c659c640 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609bc48d0 0x556609bc6d70 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d360bb7d19ea9271 server_cookie=abc9fcda04c9626b in_seq=0 out_seq=0 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.248+0000 7f98aaff5640 10 client.18786 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98aaff5640 1 -- 172.21.15.5:0/263487216 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556609b17a70 con 0x556609bc48d0 2024-04-22T13:36:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98aaff5640 10 client.18786 handle_command_reply: tid=0 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 client.18786 shutdown 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 msgr2=0x556609bb1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bb1a80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f98b8035000 tx=0x7f98b802c9f0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 shutdown_connections 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609bc48d0 0x556609bc6d70 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609bb2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609babea0 0x556609bb1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/263487216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bb1a80 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98aaff5640 0 client.18786 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 >> 172.21.15.5:0/263487216 conn(0x556609ba5cb0 msgr2=0x556609ba7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 shutdown_connections 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 1 -- 172.21.15.5:0/263487216 wait complete. 2024-04-22T13:36:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.249+0000 7f98cfdd2740 20 client.18786 trim_cache size 0 max 16384 2024-04-22T13:36:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.266+0000 7f98cfdd2740 1 Processor -- start 2024-04-22T13:36:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.266+0000 7f98cfdd2740 1 -- start start 2024-04-22T13:36:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bc0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609babea0 0x556609bb0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bb4a80 0x556609ba4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81adb80 con 0x556609bab580 2024-04-22T13:36:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81af6c0 con 0x556609babea0 2024-04-22T13:36:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ae250 con 0x556609bb4a80 2024-04-22T13:36:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bc0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bc0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50944/0 (socket says 172.21.15.5:50944) 2024-04-22T13:36:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 -- 172.21.15.5:0/394920299 learned_addr learned my addr 172.21.15.5:0/394920299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c659c640 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609babea0 0x556609bb0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 -- 172.21.15.5:0/394920299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bb4a80 msgr2=0x556609ba4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bb4a80 0x556609ba4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 -- 172.21.15.5:0/394920299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609babea0 msgr2=0x556609bb0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609babea0 0x556609bb0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 -- 172.21.15.5:0/394920299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556609bb0a50 con 0x556609bab580 2024-04-22T13:36:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bc0670 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f98b802c170 tx=0x7f98b802c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de73d04f9139ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98acff9640 1 -- 172.21.15.5:0/394920299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8005570 con 0x556609bab580 2024-04-22T13:36:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98acff9640 1 -- 172.21.15.5:0/394920299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b8005710 con 0x556609bab580 2024-04-22T13:36:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.267+0000 7f98acff9640 1 -- 172.21.15.5:0/394920299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8053b80 con 0x556609bab580 2024-04-22T13:36:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 -- 172.21.15.5:0/394920299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 msgr2=0x556609bc0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bc0670 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f98b802c170 tx=0x7f98b802c6d0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 -- 172.21.15.5:0/394920299 shutdown_connections 2024-04-22T13:36:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bb4a80 0x556609ba4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609babea0 0x556609bb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/394920299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609bab580 0x556609bc0670 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 -- 172.21.15.5:0/394920299 >> 172.21.15.5:0/394920299 conn(0x556609ba5cb0 msgr2=0x556609ba44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 -- 172.21.15.5:0/394920299 shutdown_connections 2024-04-22T13:36:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 -- 172.21.15.5:0/394920299 wait complete. 2024-04-22T13:36:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 -- start start 2024-04-22T13:36:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.268+0000 7f98cfdd2740 1 Processor -- start 2024-04-22T13:36:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bab580 0x556609bae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 0x556609baeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609baf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81aeff0 con 0x556609babea0 2024-04-22T13:36:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ae920 con 0x556609bb4a80 2024-04-22T13:36:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81abb80 con 0x556609bab580 2024-04-22T13:36:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 10 client.0 _ll_register_callbacks cb 0x7f98b8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 0x556609baeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 0x556609baeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50948/0 (socket says 172.21.15.5:50948) 2024-04-22T13:36:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 -- 172.21.15.5:0/501699269 learned_addr learned my addr 172.21.15.5:0/501699269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c759e640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609baf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 -- 172.21.15.5:0/501699269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bab580 msgr2=0x556609bae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bab580 0x556609bae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 -- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 msgr2=0x556609baf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609baf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 -- 172.21.15.5:0/501699269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556609babcd0 con 0x556609babea0 2024-04-22T13:36:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c759e640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609baf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98c659c640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 0x556609baeb90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f98bc09beb0 tx=0x7f98bc0d3c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e93ac63c7e95eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98aaff5640 1 -- 172.21.15.5:0/501699269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc020040 con 0x556609babea0 2024-04-22T13:36:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98aaff5640 1 -- 172.21.15.5:0/501699269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f98bc005680 con 0x556609babea0 2024-04-22T13:36:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b8005710 con 0x556609babea0 2024-04-22T13:36:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98aaff5640 1 -- 172.21.15.5:0/501699269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98bc09bb10 con 0x556609babea0 2024-04-22T13:36:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 10 client.18792 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.269+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556609c2aea0 con 0x556609babea0 2024-04-22T13:36:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98aaff5640 1 -- 172.21.15.5:0/501699269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b8005710 con 0x556609babea0 2024-04-22T13:36:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98aaff5640 1 -- 172.21.15.5:0/501699269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f98bc0cd0f0 con 0x556609babea0 2024-04-22T13:36:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98cfdd2740 10 client.18792 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98cfdd2740 10 client.18792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98cfdd2740 20 client.18792 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609c2b2a0 0x556609c2d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98cfdd2740 4 client.18792 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:36:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556609a93590 con 0x556609c2b2a0 2024-04-22T13:36:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609c2b2a0 0x556609c2d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98c6d9d640 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609c2b2a0 0x556609c2d690 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe67f5c7bcd63ae8 server_cookie=589c8094b30acb90 in_seq=0 out_seq=0 2024-04-22T13:36:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.270+0000 7f98aaff5640 10 client.18792 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98aaff5640 1 -- 172.21.15.5:0/501699269 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556609a93590 con 0x556609c2b2a0 2024-04-22T13:36:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98aaff5640 10 client.18792 handle_command_reply: tid=0 2024-04-22T13:36:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 client.18792 shutdown 2024-04-22T13:36:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 msgr2=0x556609baeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 0x556609baeb90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f98bc09beb0 tx=0x7f98bc0d3c70 comp rx=0 tx=0).stop 2024-04-22T13:36:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 shutdown_connections 2024-04-22T13:36:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556609c2b2a0 0x556609c2d690 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556609babea0 0x556609baeb90 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556609bb4a80 0x556609baf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98aaff5640 0 client.18792 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 --2- 172.21.15.5:0/501699269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556609bab580 0x556609bae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 >> 172.21.15.5:0/501699269 conn(0x556609ba5cb0 msgr2=0x556609ba4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 shutdown_connections 2024-04-22T13:36:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 1 -- 172.21.15.5:0/501699269 wait complete. 2024-04-22T13:36:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cfdd2740 20 client.18792 trim_cache size 0 max 16384 2024-04-22T13:36:31.287 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:36:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:36:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:36:31.287 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:36:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98b00a3d20 msgr2=0x7f98b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98b00a3d20 0x7f98b00a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009370 tx=0x7f98c0004a40 comp rx=0 tx=0).stop 2024-04-22T13:36:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 msgr2=0x7f98c8198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.271+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 0x7f98c8198c20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00d750 tx=0x7f98bc00dc10 comp rx=0 tx=0).stop 2024-04-22T13:36:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 shutdown_connections 2024-04-22T13:36:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81072a0 0x7f98c8198c20 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f98b00a3d20 0x7f98b00a61e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c819cfa0 0x7f98c819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 --2- 172.21.15.5:0/2915665288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c8069560 0x7f98c81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 >> 172.21.15.5:0/2915665288 conn(0x7f98c8077fa0 msgr2=0x7f98c80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 shutdown_connections 2024-04-22T13:36:31.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.272+0000 7f98cd1e6640 1 -- 172.21.15.5:0/2915665288 wait complete. 2024-04-22T13:36:31.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:36:31.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 -- 172.21.15.5:0/3698479364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 msgr2=0x7f0b8c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 --2- 172.21.15.5:0/3698479364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c10ee40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ca0 tx=0x7f0b7c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 -- 172.21.15.5:0/3698479364 shutdown_connections 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 --2- 172.21.15.5:0/3698479364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c10ee40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 --2- 172.21.15.5:0/3698479364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c102b50 0x7f0b8c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 --2- 172.21.15.5:0/3698479364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8c1021a0 0x7f0b8c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 -- 172.21.15.5:0/3698479364 >> 172.21.15.5:0/3698479364 conn(0x7f0b8c0793e0 msgr2=0x7f0b8c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 -- 172.21.15.5:0/3698479364 shutdown_connections 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.400+0000 7f0b94523640 1 -- 172.21.15.5:0/3698479364 wait complete. 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 Processor -- start 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 -- start start 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8c1021a0 0x7f0b8c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c19cf60 0x7f0b8c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1121e0 con 0x7f0b8c107270 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112060 con 0x7f0b8c19cf60 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b94523640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112360 con 0x7f0b8c1021a0 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50960/0 (socket says 172.21.15.5:50960) 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 -- 172.21.15.5:0/2480865530 learned_addr learned my addr 172.21.15.5:0/2480865530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b92ac0640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c19cf60 0x7f0b8c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b922bf640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8c1021a0 0x7f0b8c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 -- 172.21.15.5:0/2480865530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8c1021a0 msgr2=0x7f0b8c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8c1021a0 0x7f0b8c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 -- 172.21.15.5:0/2480865530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c19cf60 msgr2=0x7f0b8c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c19cf60 0x7f0b8c19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b91abe640 1 -- 172.21.15.5:0/2480865530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c19da50 con 0x7f0b8c107270 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.401+0000 7f0b92ac0640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c19cf60 0x7f0b8c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.402+0000 7f0b91abe640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c198be0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000e9c0 tx=0x7f0b8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.402+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000cd80 con 0x7f0b8c107270 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.402+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b800054d0 con 0x7f0b8c107270 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.402+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c19dd40 con 0x7f0b8c107270 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.402+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80010430 con 0x7f0b8c107270 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.402+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a5530 con 0x7f0b8c107270 2024-04-22T13:36:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.403+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c069250 con 0x7f0b8c107270 2024-04-22T13:36:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.404+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0b80004fd0 con 0x7f0b8c107270 2024-04-22T13:36:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.404+0000 7f0b8b7fe640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0b740a3d20 0x7f0b740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.405+0000 7f0b922bf640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0b740a3d20 0x7f0b740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.405+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f0b800d0d10 con 0x7f0b8c107270 2024-04-22T13:36:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.405+0000 7f0b922bf640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0b740a3d20 0x7f0b740a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005fd0 tx=0x7f0b84005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.409+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b800d6020 con 0x7f0b8c107270 2024-04-22T13:36:31.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:31 smithi049 ceph-mon[31379]: pgmap v290: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:31.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:31 smithi107 ceph-mon[31859]: pgmap v290: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:31.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:31 smithi005 ceph-mon[28006]: pgmap v290: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:31.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.639+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8c102b50 con 0x7f0b8c107270 2024-04-22T13:36:31.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.639+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/2480865530 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f0b8009d350 con 0x7f0b8c107270 2024-04-22T13:36:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:36:31.635 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:36:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:36:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0b740a3d20 msgr2=0x7f0b740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0b740a3d20 0x7f0b740a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005fd0 tx=0x7f0b84005850 comp rx=0 tx=0).stop 2024-04-22T13:36:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 msgr2=0x7f0b8c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c198be0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000e9c0 tx=0x7f0b8000ee80 comp rx=0 tx=0).stop 2024-04-22T13:36:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 shutdown_connections 2024-04-22T13:36:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c107270 0x7f0b8c198be0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0b740a3d20 0x7f0b740a61e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b8c19cf60 0x7f0b8c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 --2- 172.21.15.5:0/2480865530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8c1021a0 0x7f0b8c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 >> 172.21.15.5:0/2480865530 conn(0x7f0b8c0793e0 msgr2=0x7f0b8c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.641+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 shutdown_connections 2024-04-22T13:36:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.642+0000 7f0b94523640 1 -- 172.21.15.5:0/2480865530 wait complete. 2024-04-22T13:36:31.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 -- 172.21.15.5:0/2797067643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5050101a40 msgr2=0x7f5050105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 --2- 172.21.15.5:0/2797067643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5050101a40 0x7f5050105a80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5040009900 tx=0x7f504002bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 -- 172.21.15.5:0/2797067643 shutdown_connections 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 --2- 172.21.15.5:0/2797067643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5050101a40 0x7f5050105a80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 --2- 172.21.15.5:0/2797067643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050106160 0x7f505010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 --2- 172.21.15.5:0/2797067643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050101090 0x7f5050101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 -- 172.21.15.5:0/2797067643 >> 172.21.15.5:0/2797067643 conn(0x7f5050078000 msgr2=0x7f5050078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 -- 172.21.15.5:0/2797067643 shutdown_connections 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 -- 172.21.15.5:0/2797067643 wait complete. 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.757+0000 7f505799d640 1 Processor -- start 2024-04-22T13:36:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 -- start start 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050101090 0x7f50501995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050106160 0x7f5050199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 0x7f50501a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50501110a0 con 0x7f505019a090 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5050110f20 con 0x7f5050106160 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f505799d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5050111220 con 0x7f5050101090 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 0x7f50501a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 0x7f50501a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50978/0 (socket says 172.21.15.5:50978) 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 -- 172.21.15.5:0/1196870500 learned_addr learned my addr 172.21.15.5:0/1196870500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5054f38640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050106160 0x7f5050199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055739640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050101090 0x7f50501995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 -- 172.21.15.5:0/1196870500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050101090 msgr2=0x7f50501995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050101090 0x7f50501995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 -- 172.21.15.5:0/1196870500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050106160 msgr2=0x7f5050199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050106160 0x7f5050199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 -- 172.21.15.5:0/1196870500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50501a4e10 con 0x7f505019a090 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5054f38640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050106160 0x7f5050199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055739640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050101090 0x7f50501995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.758+0000 7f5055f3a640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 0x7f50501a4640 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f504800ea60 tx=0x7f504800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.759+0000 7f50467fc640 1 -- 172.21.15.5:0/1196870500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504800ce40 con 0x7f505019a090 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.759+0000 7f50467fc640 1 -- 172.21.15.5:0/1196870500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5048005020 con 0x7f505019a090 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.759+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50501a5100 con 0x7f505019a090 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.759+0000 7f50467fc640 1 -- 172.21.15.5:0/1196870500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048010670 con 0x7f505019a090 2024-04-22T13:36:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.759+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50501a5620 con 0x7f505019a090 2024-04-22T13:36:31.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f50467fc640 1 -- 172.21.15.5:0/1196870500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f50480054b0 con 0x7f505019a090 2024-04-22T13:36:31.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 Processor -- start 2024-04-22T13:36:31.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 -- start start 2024-04-22T13:36:31.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea46790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 0x55873ea4f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea4fe60 0x55873ea502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55873e9ab1f0 con 0x55873ea46d60 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55873e9484b0 con 0x55873ea463b0 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f505a58f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55873e9a71b0 con 0x55873ea4fe60 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5055739640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 0x55873ea4f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5055739640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 0x55873ea4f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50992/0 (socket says 172.21.15.5:50992) 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5055739640 1 -- 172.21.15.5:0/1338780132 learned_addr learned my addr 172.21.15.5:0/1338780132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f50467fc640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f503c0a3d20 0x7f503c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5054f38640 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea4fe60 0x55873ea502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5055739640 1 -- 172.21.15.5:0/1338780132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea4fe60 msgr2=0x55873ea502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5054f38640 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea46790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5055739640 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea4fe60 0x55873ea502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.762+0000 7f5055739640 1 -- 172.21.15.5:0/1338780132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 msgr2=0x55873ea46790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f5055f3a640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f503c0a3d20 0x7f503c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f5055739640 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea46790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f5055739640 1 -- 172.21.15.5:0/1338780132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873e93a1b0 con 0x55873ea46d60 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f5054f38640 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea46790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f50467fc640 1 -- 172.21.15.5:0/1196870500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f50480d0d60 con 0x7f505019a090 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f5055739640 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 0x55873ea4f890 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f504c009d60 tx=0x7f504c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac380d9c775fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f503b7fe640 1 -- 172.21.15.5:0/1338780132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c012650 con 0x55873ea46d60 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f5055f3a640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f503c0a3d20 0x7f503c0a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50480ce560 tx=0x7f504800ef90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f503b7fe640 1 -- 172.21.15.5:0/1338780132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f504c009e50 con 0x55873ea46d60 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.763+0000 7f503b7fe640 1 -- 172.21.15.5:0/1338780132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c00fc00 con 0x55873ea46d60 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 -- 172.21.15.5:0/1338780132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 msgr2=0x55873ea4f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 0x55873ea4f890 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f504c009d60 tx=0x7f504c0117b0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 -- 172.21.15.5:0/1338780132 shutdown_connections 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea46d60 0x55873ea4f890 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea4fe60 0x55873ea502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 --2- 172.21.15.5:0/1338780132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea46790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 -- 172.21.15.5:0/1338780132 >> 172.21.15.5:0/1338780132 conn(0x55873ea40ba0 msgr2=0x55873ea40f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 -- 172.21.15.5:0/1338780132 shutdown_connections 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 -- 172.21.15.5:0/1338780132 wait complete. 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 -- start start 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 Processor -- start 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.764+0000 7f505a58f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea56280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea53610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 0x55873ea53b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55873ea56da0 con 0x55873ea463b0 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55873e964500 con 0x55873ea4fe60 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55873ea56f20 con 0x55873ea46d60 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 10 client.0 _ll_register_callbacks cb 0x55873e932b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea56280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea56280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50996/0 (socket says 172.21.15.5:50996) 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 -- 172.21.15.5:0/3865148099 learned_addr learned my addr 172.21.15.5:0/3865148099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 -- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 msgr2=0x55873ea53610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea53610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 -- 172.21.15.5:0/3865148099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 msgr2=0x55873ea53b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 0x55873ea53b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 -- 172.21.15.5:0/3865148099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873e6bd3f0 con 0x55873ea463b0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5054f38640 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea56280 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f50400096c0 tx=0x7f5040005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e464900dfacc8a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5038ff9640 1 -- 172.21.15.5:0/3865148099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040046020 con 0x55873ea463b0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5038ff9640 1 -- 172.21.15.5:0/3865148099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f504002cbf0 con 0x55873ea463b0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f504c009e50 con 0x55873ea463b0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f5038ff9640 1 -- 172.21.15.5:0/3865148099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5040005030 con 0x55873ea463b0 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 10 client.18810 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:31.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.765+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55873e7877d0 con 0x55873ea463b0 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f5038ff9640 1 -- 172.21.15.5:0/3865148099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f504c009e50 con 0x55873ea463b0 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f5038ff9640 1 -- 172.21.15.5:0/3865148099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5040040040 con 0x55873ea463b0 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f505a58f740 10 client.18810 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f505a58f740 10 client.18810 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f505a58f740 20 client.18810 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f505a58f740 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873ea63f90 0x55873ea66380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f505a58f740 4 client.18810 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873e94a930 con 0x55873ea63f90 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.766+0000 7f5055739640 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873ea63f90 0x55873ea66380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.767+0000 7f5055739640 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873ea63f90 0x55873ea66380 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73ea7c5f7f6e9140 server_cookie=7cbe72ae83be3a24 in_seq=0 out_seq=0 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.767+0000 7f5038ff9640 10 client.18810 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.767+0000 7f5038ff9640 1 -- 172.21.15.5:0/3865148099 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55873e94a930 con 0x55873ea63f90 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.767+0000 7f5038ff9640 10 client.18810 handle_command_reply: tid=0 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.767+0000 7f505a58f740 1 client.18810 shutdown 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 msgr2=0x55873ea56280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea56280 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f50400096c0 tx=0x7f5040005e10 comp rx=0 tx=0).stop 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 shutdown_connections 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 0x55873ea53b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea53610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873ea63f90 0x55873ea66380 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 --2- 172.21.15.5:0/3865148099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea56280 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f5038ff9640 0 client.18810 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 >> 172.21.15.5:0/3865148099 conn(0x55873ea40ba0 msgr2=0x55873ea428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 shutdown_connections 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 1 -- 172.21.15.5:0/3865148099 wait complete. 2024-04-22T13:36:31.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.768+0000 7f505a58f740 20 client.18810 trim_cache size 0 max 16384 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 Processor -- start 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 -- start start 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea5fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea4e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 0x55873ea426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ab9e0 con 0x55873ea463b0 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ad520 con 0x55873ea4fe60 2024-04-22T13:36:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f505a58f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ac0b0 con 0x55873ea46d60 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f5054f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea5fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f5054f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea5fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51000/0 (socket says 172.21.15.5:51000) 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.781+0000 7f5054f38640 1 -- 172.21.15.5:0/1259993150 learned_addr learned my addr 172.21.15.5:0/1259993150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5055739640 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea4e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5054f38640 1 -- 172.21.15.5:0/1259993150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 msgr2=0x55873ea4e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5054f38640 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea4e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5054f38640 1 -- 172.21.15.5:0/1259993150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 msgr2=0x55873ea426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5054f38640 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 0x55873ea426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5054f38640 1 -- 172.21.15.5:0/1259993150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873ea45560 con 0x55873ea463b0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f5054f38640 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea5fca0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5040004dc0 tx=0x7f50400045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b864d502e772af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f503affd640 1 -- 172.21.15.5:0/1259993150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504005a070 con 0x55873ea463b0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f503affd640 1 -- 172.21.15.5:0/1259993150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50400046c0 con 0x55873ea463b0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.782+0000 7f503affd640 1 -- 172.21.15.5:0/1259993150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040046040 con 0x55873ea463b0 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 -- 172.21.15.5:0/1259993150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 msgr2=0x55873ea5fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea5fca0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5040004dc0 tx=0x7f50400045f0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 -- 172.21.15.5:0/1259993150 shutdown_connections 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea4fe60 0x55873ea426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea4e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 --2- 172.21.15.5:0/1259993150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea463b0 0x55873ea5fca0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 -- 172.21.15.5:0/1259993150 >> 172.21.15.5:0/1259993150 conn(0x55873ea40ba0 msgr2=0x55873ea41fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 -- 172.21.15.5:0/1259993150 shutdown_connections 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 -- 172.21.15.5:0/1259993150 wait complete. 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 -- start start 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 Processor -- start 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea5c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.783+0000 7f505a58f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 0x55873ea55610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ace50 con 0x55873ea4fe60 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ac780 con 0x55873ea463b0 2024-04-22T13:36:31.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50501a9a00 con 0x55873ea46d60 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 10 client.0 _ll_register_callbacks cb 0x7f5040009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 0x55873ea55610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea5c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 0x55873ea55610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51014/0 (socket says 172.21.15.5:51014) 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 -- 172.21.15.5:0/2772955925 learned_addr learned my addr 172.21.15.5:0/2772955925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5055739640 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 -- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 msgr2=0x55873ea550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 -- 172.21.15.5:0/2772955925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 msgr2=0x55873ea5c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea5c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 -- 172.21.15.5:0/2772955925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873ea55e40 con 0x55873ea4fe60 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5054f38640 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 0x55873ea55610 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x55873ea56790 tx=0x7f50400095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3736d1df040ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5038ff9640 1 -- 172.21.15.5:0/2772955925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504005a030 con 0x55873ea4fe60 2024-04-22T13:36:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5038ff9640 1 -- 172.21.15.5:0/2772955925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f5040003800 con 0x55873ea4fe60 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50400046c0 con 0x55873ea4fe60 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f5038ff9640 1 -- 172.21.15.5:0/2772955925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5040003a00 con 0x55873ea4fe60 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 10 client.18816 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.784+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55873eaca460 con 0x55873ea4fe60 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f5038ff9640 1 -- 172.21.15.5:0/2772955925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50400046c0 con 0x55873ea4fe60 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f5038ff9640 1 -- 172.21.15.5:0/2772955925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50400559f0 con 0x55873ea4fe60 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f505a58f740 10 client.18816 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f505a58f740 10 client.18816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f505a58f740 20 client.18816 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f505a58f740 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873eaca730 0x55873eaccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f505a58f740 4 client.18816 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55873ea53ee0 con 0x55873eaca730 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f5055739640 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873eaca730 0x55873eaccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f5055739640 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873eaca730 0x55873eaccb50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78ca20e05942268a server_cookie=6d46fac668d9c937 in_seq=0 out_seq=0 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.785+0000 7f5038ff9640 10 client.18816 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f5038ff9640 1 -- 172.21.15.5:0/2772955925 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55873ea53ee0 con 0x55873eaca730 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f5038ff9640 10 client.18816 handle_command_reply: tid=0 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 client.18816 shutdown 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 msgr2=0x55873ea55610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 0x55873ea55610 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x55873ea56790 tx=0x7f50400095a0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 shutdown_connections 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55873eaca730 0x55873eaccb50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55873ea4fe60 0x55873ea55610 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f5038ff9640 0 client.18816 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873ea46d60 0x55873ea550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 --2- 172.21.15.5:0/2772955925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55873ea463b0 0x55873ea5c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 >> 172.21.15.5:0/2772955925 conn(0x55873ea40ba0 msgr2=0x55873ea42830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 shutdown_connections 2024-04-22T13:36:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 1 -- 172.21.15.5:0/2772955925 wait complete. 2024-04-22T13:36:31.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505a58f740 20 client.18816 trim_cache size 0 max 16384 2024-04-22T13:36:31.788 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:36:31.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f503c0a3d20 msgr2=0x7f503c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505799d640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f503c0a3d20 0x7f503c0a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50480ce560 tx=0x7f504800ef90 comp rx=0 tx=0).stop 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 msgr2=0x7f50501a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.786+0000 7f505799d640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 0x7f50501a4640 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f504800ea60 tx=0x7f504800ef20 comp rx=0 tx=0).stop 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 shutdown_connections 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f505019a090 0x7f50501a4640 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f503c0a3d20 0x7f503c0a61e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5050106160 0x7f5050199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 --2- 172.21.15.5:0/1196870500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5050101090 0x7f50501995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 >> 172.21.15.5:0/1196870500 conn(0x7f5050078000 msgr2=0x7f50500ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 shutdown_connections 2024-04-22T13:36:31.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.787+0000 7f505799d640 1 -- 172.21.15.5:0/1196870500 wait complete. 2024-04-22T13:36:31.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:36:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 -- 172.21.15.5:0/3185088016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af81031c0 msgr2=0x7f9af81035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 --2- 172.21.15.5:0/3185088016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af81031c0 0x7f9af81035a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009900 tx=0x7f9af402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 -- 172.21.15.5:0/3185088016 shutdown_connections 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 --2- 172.21.15.5:0/3185088016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af810c500 0x7f9af810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 --2- 172.21.15.5:0/3185088016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af8103ae0 0x7f9af810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 --2- 172.21.15.5:0/3185088016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af81031c0 0x7f9af81035a0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 -- 172.21.15.5:0/3185088016 >> 172.21.15.5:0/3185088016 conn(0x7f9af8078000 msgr2=0x7f9af8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 -- 172.21.15.5:0/3185088016 shutdown_connections 2024-04-22T13:36:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 -- 172.21.15.5:0/3185088016 wait complete. 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 Processor -- start 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.901+0000 7f9b00752640 1 -- start start 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9b00752640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af81031c0 0x7f9af8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9b00752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af8103ae0 0x7f9af8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9b00752640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 0x7f9af8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9b00752640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8112190 con 0x7f9af810c500 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9b00752640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8112010 con 0x7f9af81031c0 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9b00752640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8112310 con 0x7f9af8103ae0 2024-04-22T13:36:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 0x7f9af8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 0x7f9af8101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51028/0 (socket says 172.21.15.5:51028) 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 -- 172.21.15.5:0/2609491808 learned_addr learned my addr 172.21.15.5:0/2609491808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af81031c0 0x7f9af8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afdced640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af8103ae0 0x7f9af8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 -- 172.21.15.5:0/2609491808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af8103ae0 msgr2=0x7f9af8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af8103ae0 0x7f9af8101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 -- 172.21.15.5:0/2609491808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af81031c0 msgr2=0x7f9af8100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af81031c0 0x7f9af8100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 -- 172.21.15.5:0/2609491808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af81a6c30 con 0x7f9af810c500 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9afecef640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 0x7f9af8101890 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9af000cab0 tx=0x7f9af000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.902+0000 7f9aef7fe640 1 -- 172.21.15.5:0/2609491808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af000adc0 con 0x7f9af810c500 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.903+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81a6f20 con 0x7f9af810c500 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.903+0000 7f9aef7fe640 1 -- 172.21.15.5:0/2609491808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af000f900 con 0x7f9af810c500 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.903+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81a7440 con 0x7f9af810c500 2024-04-22T13:36:31.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.903+0000 7f9aef7fe640 1 -- 172.21.15.5:0/2609491808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00189a0 con 0x7f9af810c500 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9aef7fe640 1 -- 172.21.15.5:0/2609491808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9af000f450 con 0x7f9af810c500 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 Processor -- start 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 -- start start 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9aef7fe640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ae40a3da0 0x7f9ae40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b0b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b16e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55add0a701f0 con 0x55add0b0b4d0 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55add0a0d4b0 con 0x55add0b0be80 2024-04-22T13:36:31.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.905+0000 7f9b03427740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55add0a6c1b0 con 0x55add0b14a00 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b0b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afdced640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ae40a3da0 0x7f9ae40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b0b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51040/0 (socket says 172.21.15.5:51040) 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 -- 172.21.15.5:0/682482632 learned_addr learned my addr 172.21.15.5:0/682482632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9aef7fe640 1 -- 172.21.15.5:0/2609491808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9af00d0df0 con 0x7f9af810c500 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afecef640 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b16e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afdced640 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 -- 172.21.15.5:0/682482632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 msgr2=0x55add0b16e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b16e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 -- 172.21.15.5:0/682482632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 msgr2=0x55add0b14430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b14430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 -- 172.21.15.5:0/682482632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add09ff1b0 con 0x55add0b0b4d0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afdced640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ae40a3da0 0x7f9ae40a6260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9af81028a0 tx=0x7f9ae80138e0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b0b8b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009900 tx=0x7f9af4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ad6602ab5589b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9aecff9640 1 -- 172.21.15.5:0/682482632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004230 con 0x55add0b0b4d0 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9aecff9640 1 -- 172.21.15.5:0/682482632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af40043d0 con 0x55add0b0b4d0 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.906+0000 7f9aecff9640 1 -- 172.21.15.5:0/682482632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af40046c0 con 0x55add0b0b4d0 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 -- 172.21.15.5:0/682482632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 msgr2=0x55add0b0b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b0b8b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009900 tx=0x7f9af4004040 comp rx=0 tx=0).stop 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 -- 172.21.15.5:0/682482632 shutdown_connections 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b16e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 --2- 172.21.15.5:0/682482632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b0b8b0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 -- 172.21.15.5:0/682482632 >> 172.21.15.5:0/682482632 conn(0x55add0b05ba0 msgr2=0x55add0b05f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 -- 172.21.15.5:0/682482632 shutdown_connections 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 -- 172.21.15.5:0/682482632 wait complete. 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.907+0000 7f9b03427740 1 -- start start 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 Processor -- start 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0b4d0 0x55add0b142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b11bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55add0b17910 con 0x55add0b14a00 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55add0a29500 con 0x55add0b0be80 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55add0b17a90 con 0x55add0b0b4d0 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9b03427740 10 client.0 _ll_register_callbacks cb 0x55add09f7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b11bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afe4ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0b4d0 0x55add0b142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b11bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51050/0 (socket says 172.21.15.5:51050) 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afe4ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0b4d0 0x55add0b142e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.5:50480/0 (socket says 172.21.15.5:50480) 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 -- 172.21.15.5:0/2624235988 learned_addr learned my addr 172.21.15.5:0/2624235988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afdced640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 -- 172.21.15.5:0/2624235988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0b4d0 msgr2=0x55add0b142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0b4d0 0x55add0b142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 -- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 msgr2=0x55add0b11670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b11670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afecef640 1 -- 172.21.15.5:0/2624235988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add07823f0 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.908+0000 7f9afdced640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9afecef640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b11bd0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9af000ca80 tx=0x7f9af009c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b746d3ae3cc68a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/2624235988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af009d490 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/2624235988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9af0099400 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af40043d0 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/2624235988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9af009cd90 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 10 client.18828 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55add084c7d0 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/2624235988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af40043d0 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/2624235988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9af00b68d0 con 0x55add0b14a00 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 10 client.18828 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 10 client.18828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:36:31.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 20 client.18828 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b24ba0 0x55add0b27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 4 client.18828 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.909+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55add0a0f930 con 0x55add0b24ba0 2024-04-22T13:36:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.910+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b24ba0 0x55add0b27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.910+0000 7f9ae1ffb640 10 client.18828 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:36:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.910+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b24ba0 0x55add0b27000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a5b3affad9225d1 server_cookie=55b47d73225f7ac0 in_seq=0 out_seq=0 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/2624235988 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55add0a0f930 con 0x55add0b24ba0 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9ae1ffb640 10 client.18828 handle_command_reply: tid=0 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 client.18828 shutdown 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 msgr2=0x55add0b11bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b11bd0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9af000ca80 tx=0x7f9af009c7b0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 shutdown_connections 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b24ba0 0x55add0b27000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b11bd0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9ae1ffb640 0 client.18828 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 --2- 172.21.15.5:0/2624235988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0b4d0 0x55add0b142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 >> 172.21.15.5:0/2624235988 conn(0x55add0b05ba0 msgr2=0x55add0b078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 shutdown_connections 2024-04-22T13:36:31.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 1 -- 172.21.15.5:0/2624235988 wait complete. 2024-04-22T13:36:31.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.911+0000 7f9b03427740 20 client.18828 trim_cache size 0 max 16384 2024-04-22T13:36:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.924+0000 7f9b03427740 1 Processor -- start 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.924+0000 7f9b03427740 1 -- start start 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.924+0000 7f9b03427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b20800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.924+0000 7f9b03427740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b0c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.924+0000 7f9b03427740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b13da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.924+0000 7f9b03427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81afc80 con 0x55add0b0b4d0 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9b03427740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b17c0 con 0x55add0b0be80 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9b03427740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b0350 con 0x55add0b14a00 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b20800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b20800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51052/0 (socket says 172.21.15.5:51052) 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 -- 172.21.15.5:0/4164623841 learned_addr learned my addr 172.21.15.5:0/4164623841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afecef640 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b13da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 -- 172.21.15.5:0/4164623841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 msgr2=0x55add0b13da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b13da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 -- 172.21.15.5:0/4164623841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 msgr2=0x55add0b0c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afdced640 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b0c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b0c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 -- 172.21.15.5:0/4164623841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add0b11f40 con 0x55add0b0b4d0 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b20800 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9af402cb50 tx=0x7f9af4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be405fe3efd6284 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9ae3fff640 1 -- 172.21.15.5:0/4164623841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004430 con 0x55add0b0b4d0 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9ae3fff640 1 -- 172.21.15.5:0/4164623841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af4003aa0 con 0x55add0b0b4d0 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.925+0000 7f9ae3fff640 1 -- 172.21.15.5:0/4164623841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4054400 con 0x55add0b0b4d0 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- 172.21.15.5:0/4164623841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 msgr2=0x55add0b20800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b20800 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9af402cb50 tx=0x7f9af4005fd0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- 172.21.15.5:0/4164623841 shutdown_connections 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b14a00 0x55add0b13da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0be80 0x55add0b0c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- 172.21.15.5:0/4164623841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b0b4d0 0x55add0b20800 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- 172.21.15.5:0/4164623841 >> 172.21.15.5:0/4164623841 conn(0x55add0b05ba0 msgr2=0x55add0b0c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- 172.21.15.5:0/4164623841 shutdown_connections 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- 172.21.15.5:0/4164623841 wait complete. 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- start start 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 Processor -- start 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0b4d0 0x55add0b1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0be80 0x55add0b185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b10f0 con 0x55add0b14a00 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81b0a20 con 0x55add0b0b4d0 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.926+0000 7f9b03427740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81adc80 con 0x55add0b0be80 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9b03427740 10 client.0 _ll_register_callbacks cb 0x7f9af0099090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b18cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51060/0 (socket says 172.21.15.5:51060) 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 -- 172.21.15.5:0/284880813 learned_addr learned my addr 172.21.15.5:0/284880813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0b4d0 0x55add0b1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afdced640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0be80 0x55add0b185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 -- 172.21.15.5:0/284880813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0be80 msgr2=0x55add0b185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0be80 0x55add0b185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 -- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0b4d0 msgr2=0x55add0b1d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0b4d0 0x55add0b1d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 -- 172.21.15.5:0/284880813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add0b8a140 con 0x55add0b14a00 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afdced640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0be80 0x55add0b185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9afecef640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b18cd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9af0099760 tx=0x7f9af00626e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb12fa19697ab8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/284880813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0080060 con 0x55add0b14a00 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/284880813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9af0099400 con 0x55add0b14a00 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af4003aa0 con 0x55add0b14a00 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/284880813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9af00bb070 con 0x55add0b14a00 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9b03427740 10 client.18834 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55add0b8b2e0 con 0x55add0b14a00 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.927+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/284880813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af4003aa0 con 0x55add0b14a00 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/284880813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9af007bbb0 con 0x55add0b14a00 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9b03427740 10 client.18834 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9b03427740 10 client.18834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9b03427740 20 client.18834 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9b03427740 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b8b9a0 0x55add0b8dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9b03427740 4 client.18834 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55add0a7dc10 con 0x55add0b8b9a0 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b8b9a0 0x55add0b8dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9ae1ffb640 10 client.18834 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.928+0000 7f9afe4ee640 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b8b9a0 0x55add0b8dd90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5709ffd9470c5ac server_cookie=79ae4e08de52eea9 in_seq=0 out_seq=0 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9ae1ffb640 1 -- 172.21.15.5:0/284880813 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55add0a7dc10 con 0x55add0b8b9a0 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9ae1ffb640 10 client.18834 handle_command_reply: tid=0 2024-04-22T13:36:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 client.18834 shutdown 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 msgr2=0x55add0b18cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b18cd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9af0099760 tx=0x7f9af00626e0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 shutdown_connections 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x55add0b8b9a0 0x55add0b8dd90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add0b14a00 0x55add0b18cd0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55add0b0be80 0x55add0b185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9ae1ffb640 0 client.18834 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 --2- 172.21.15.5:0/284880813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55add0b0b4d0 0x55add0b1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 >> 172.21.15.5:0/284880813 conn(0x55add0b05ba0 msgr2=0x55add0b0cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 shutdown_connections 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 1 -- 172.21.15.5:0/284880813 wait complete. 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b03427740 20 client.18834 trim_cache size 0 max 16384 2024-04-22T13:36:31.931 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:36:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ae40a3da0 msgr2=0x7f9ae40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ae40a3da0 0x7f9ae40a6260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9af81028a0 tx=0x7f9ae80138e0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 msgr2=0x7f9af8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 0x7f9af8101890 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9af000cab0 tx=0x7f9af000cf70 comp rx=0 tx=0).stop 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 shutdown_connections 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9af810c500 0x7f9af8101890 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9ae40a3da0 0x7f9ae40a6260 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af8103ae0 0x7f9af8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 --2- 172.21.15.5:0/2609491808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af81031c0 0x7f9af8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 >> 172.21.15.5:0/2609491808 conn(0x7f9af8078000 msgr2=0x7f9af80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 shutdown_connections 2024-04-22T13:36:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:31.929+0000 7f9b00752640 1 -- 172.21.15.5:0/2609491808 wait complete. 2024-04-22T13:36:31.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:36:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 -- 172.21.15.5:0/1228475014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0100a80 msgr2=0x7f9eb0100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 --2- 172.21.15.5:0/1228475014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0100a80 0x7f9eb0100e60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009900 tx=0x7f9ea402bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 -- 172.21.15.5:0/1228475014 shutdown_connections 2024-04-22T13:36:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 --2- 172.21.15.5:0/1228475014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb010a700 0x7f9eb01110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 --2- 172.21.15.5:0/1228475014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eb0101430 0x7f9eb0109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 --2- 172.21.15.5:0/1228475014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0100a80 0x7f9eb0100e60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 -- 172.21.15.5:0/1228475014 >> 172.21.15.5:0/1228475014 conn(0x7f9eb0076170 msgr2=0x7f9eb0076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 -- 172.21.15.5:0/1228475014 shutdown_connections 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.044+0000 7f9eb6270640 1 -- 172.21.15.5:0/1228475014 wait complete. 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 Processor -- start 2024-04-22T13:36:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 -- start start 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0100a80 0x7f9eb01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 0x7f9eb01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eb019a750 0x7f9eb019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0114470 con 0x7f9eb010a700 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01142f0 con 0x7f9eb019a750 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eb6270640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01145f0 con 0x7f9eb0100a80 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 0x7f9eb01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 0x7f9eb01a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51084/0 (socket says 172.21.15.5:51084) 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.045+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/2589789549 learned_addr learned my addr 172.21.15.5:0/2589789549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eb480d640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eb019a750 0x7f9eb019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaffff640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0100a80 0x7f9eb01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/2589789549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0100a80 msgr2=0x7f9eb01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0100a80 0x7f9eb01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/2589789549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eb019a750 msgr2=0x7f9eb019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eb019a750 0x7f9eb019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/2589789549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb019b350 con 0x7f9eb010a700 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaffff640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0100a80 0x7f9eb01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 0x7f9eb01a0bd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000d950 tx=0x7f9ea000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9ead7fa640 1 -- 172.21.15.5:0/2589789549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0017070 con 0x7f9eb010a700 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9ead7fa640 1 -- 172.21.15.5:0/2589789549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea0009c30 con 0x7f9eb010a700 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb01a7430 con 0x7f9eb010a700 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9ead7fa640 1 -- 172.21.15.5:0/2589789549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00047a0 con 0x7f9eb010a700 2024-04-22T13:36:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.046+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a7830 con 0x7f9eb010a700 2024-04-22T13:36:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.048+0000 7f9ead7fa640 1 -- 172.21.15.5:0/2589789549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9ea0004020 con 0x7f9eb010a700 2024-04-22T13:36:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.048+0000 7f9eb8e59740 1 Processor -- start 2024-04-22T13:36:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 -- start start 2024-04-22T13:36:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 0x55b8eefa08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa0e80 0x55b8eefa9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8eef051f0 con 0x55b8eefa04d0 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8eeea24b0 con 0x55b8eefa0e80 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b8eef011b0 con 0x55b8eefa9a00 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 0x55b8eefa08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 0x55b8eefa08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51096/0 (socket says 172.21.15.5:51096) 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 -- 172.21.15.5:0/795185526 learned_addr learned my addr 172.21.15.5:0/795185526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eaffff640 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa0e80 0x55b8eefa9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9ead7fa640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e980a3d20 0x7f9e980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 -- 172.21.15.5:0/795185526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 msgr2=0x55b8eefabe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefabe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 -- 172.21.15.5:0/795185526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa0e80 msgr2=0x55b8eefa9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa0e80 0x55b8eefa9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 -- 172.21.15.5:0/795185526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8eee941b0 con 0x55b8eefa04d0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9ead7fa640 1 -- 172.21.15.5:0/2589789549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9ea00d0510 con 0x7f9eb010a700 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e980a3d20 0x7f9e980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.049+0000 7f9eb480d640 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 0x55b8eefa08b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8011590 tx=0x7f9ea8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4cf3e77b0e540c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9e967fc640 1 -- 172.21.15.5:0/795185526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8017050 con 0x55b8eefa04d0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9e967fc640 1 -- 172.21.15.5:0/795185526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea801bb90 con 0x55b8eefa04d0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9e967fc640 1 -- 172.21.15.5:0/795185526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8025e10 con 0x55b8eefa04d0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb480d640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e980a3d20 0x7f9e980a61e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8022000 tx=0x7f9ea8023000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 -- 172.21.15.5:0/795185526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 msgr2=0x55b8eefa08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 0x55b8eefa08b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8011590 tx=0x7f9ea8011a50 comp rx=0 tx=0).stop 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 -- 172.21.15.5:0/795185526 shutdown_connections 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa0e80 0x55b8eefa9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/795185526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa04d0 0x55b8eefa08b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 -- 172.21.15.5:0/795185526 >> 172.21.15.5:0/795185526 conn(0x55b8eef9aba0 msgr2=0x55b8eef9af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.050+0000 7f9eb8e59740 1 -- 172.21.15.5:0/795185526 shutdown_connections 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 -- 172.21.15.5:0/795185526 wait complete. 2024-04-22T13:36:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 -- start start 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 Processor -- start 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefa92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefa6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefa6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8eefac970 con 0x55b8eefa0e80 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55b8eeebe500 con 0x55b8eefa9a00 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b8eefacaf0 con 0x55b8eefa04d0 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eb8e59740 10 client.0 _ll_register_callbacks cb 0x55b8eee8cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefa6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefa6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51104/0 (socket says 172.21.15.5:51104) 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.051+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4276044905 learned_addr learned my addr 172.21.15.5:0/4276044905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaffff640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefa92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eb480d640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefa6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 msgr2=0x55b8eefa92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefa92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4276044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 msgr2=0x55b8eefa6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefa6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4276044905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8eec173f0 con 0x55b8eefa0e80 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eb480d640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefa6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefa6660 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0098e80 tx=0x7f9ea009c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194a08f231fb0931 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9e8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/4276044905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0009de0 con 0x55b8eefa0e80 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/4276044905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9ea00cdee0 con 0x55b8eefa0e80 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea801bb90 con 0x55b8eefa0e80 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/4276044905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ea009c4b0 con 0x55b8eefa0e80 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eb8e59740 10 client.18846 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8eece17d0 con 0x55b8eefa0e80 2024-04-22T13:36:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.052+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/4276044905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea801bb90 con 0x55b8eefa0e80 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/4276044905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9ea00bb2b0 con 0x55b8eefa0e80 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eb8e59740 10 client.18846 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eb8e59740 10 client.18846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eb8e59740 20 client.18846 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8eefb9b20 0x55b8eefbbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eb8e59740 4 client.18846 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8eeea4930 con 0x55b8eefb9b20 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eaffff640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8eefb9b20 0x55b8eefbbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9e8f7fe640 10 client.18846 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.053+0000 7f9eaffff640 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8eefb9b20 0x55b8eefbbf10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=700dcc0773668bda server_cookie=214283fea6c6087b in_seq=0 out_seq=0 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/4276044905 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8eeea4930 con 0x55b8eefb9b20 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9e8f7fe640 10 client.18846 handle_command_reply: tid=0 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 client.18846 shutdown 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 msgr2=0x55b8eefa6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefa6660 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0098e80 tx=0x7f9ea009c0b0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 shutdown_connections 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8eefb9b20 0x55b8eefbbf10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefa6660 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefa6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9e8f7fe640 0 client.18846 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4276044905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefa92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.054+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 >> 172.21.15.5:0/4276044905 conn(0x55b8eef9aba0 msgr2=0x55b8eef9c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.055+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 shutdown_connections 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.055+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4276044905 wait complete. 2024-04-22T13:36:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.055+0000 7f9eb8e59740 20 client.18846 trim_cache size 0 max 16384 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 Processor -- start 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 -- start start 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa04d0 0x55b8eefb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eef99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefa8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01afe80 con 0x55b8eefa0e80 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b19c0 con 0x55b8eefa04d0 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b0550 con 0x55b8eefa9a00 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eef99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eef99a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51108/0 (socket says 172.21.15.5:51108) 2024-04-22T13:36:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4200827269 learned_addr learned my addr 172.21.15.5:0/4200827269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb480d640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefa8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaffff640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa04d0 0x55b8eefb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4200827269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 msgr2=0x55b8eefa8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefa8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4200827269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa04d0 msgr2=0x55b8eefb5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa04d0 0x55b8eefb5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/4200827269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8eefa6f30 con 0x55b8eefa0e80 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eb480d640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefa8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaffff640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa04d0 0x55b8eefb5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.068+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eef99a80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9ea009c440 tx=0x7f9ea0005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb1787c1bf5ef2b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9e95ffb640 1 -- 172.21.15.5:0/4200827269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00a0860 con 0x55b8eefa0e80 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9e95ffb640 1 -- 172.21.15.5:0/4200827269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea00a5120 con 0x55b8eefa0e80 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9e95ffb640 1 -- 172.21.15.5:0/4200827269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0098b20 con 0x55b8eefa0e80 2024-04-22T13:36:32.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4200827269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 msgr2=0x55b8eef99a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eef99a80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9ea009c440 tx=0x7f9ea0005190 comp rx=0 tx=0).stop 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4200827269 shutdown_connections 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eef99a80 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa9a00 0x55b8eefa8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/4200827269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa04d0 0x55b8eefb5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4200827269 >> 172.21.15.5:0/4200827269 conn(0x55b8eef9aba0 msgr2=0x55b8eef99370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4200827269 shutdown_connections 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.069+0000 7f9eb8e59740 1 -- 172.21.15.5:0/4200827269 wait complete. 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 -- start start 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 Processor -- start 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefadd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b12f0 con 0x55b8eefa0e80 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b0c20 con 0x55b8eefa9a00 2024-04-22T13:36:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01ade80 con 0x55b8eefa04d0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb8e59740 10 client.0 _ll_register_callbacks cb 0x7f9ea00056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefadd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefadd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51122/0 (socket says 172.21.15.5:51122) 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/72120975 learned_addr learned my addr 172.21.15.5:0/72120975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaffff640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb480d640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 msgr2=0x55b8eefad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/72120975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 msgr2=0x55b8eefae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eaf7fe640 1 -- 172.21.15.5:0/72120975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ef01f0d0 con 0x55b8eefa0e80 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.070+0000 7f9eb480d640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eaf7fe640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefadd80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009610 tx=0x7f9ea009bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884a4f056bec7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9e8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/72120975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00aa020 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/72120975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9ea00bb2f0 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea00a5120 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/72120975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0082050 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 10 client.18852 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8ef0203d0 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/72120975 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea00a5120 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/72120975 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9ea00a19f0 con 0x55b8eefa0e80 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 10 client.18852 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 10 client.18852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 20 client.18852 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8ef020890 0x55b8ef022c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 4 client.18852 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.071+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8eef12c10 con 0x55b8ef020890 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eaffff640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8ef020890 0x55b8ef022c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eaffff640 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8ef020890 0x55b8ef022c80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6e0926b07da80d4 server_cookie=63c26d81543e58dc in_seq=0 out_seq=0 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9e8f7fe640 10 client.18852 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9e8f7fe640 1 -- 172.21.15.5:0/72120975 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8eef12c10 con 0x55b8ef020890 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9e8f7fe640 10 client.18852 handle_command_reply: tid=0 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 client.18852 shutdown 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 msgr2=0x55b8eefadd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefadd80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009610 tx=0x7f9ea009bed0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 shutdown_connections 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55b8ef020890 0x55b8ef022c80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8eefa0e80 0x55b8eefadd80 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b8eefa9a00 0x55b8eefae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9e8f7fe640 0 client.18852 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 --2- 172.21.15.5:0/72120975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b8eefa04d0 0x55b8eefad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 >> 172.21.15.5:0/72120975 conn(0x55b8eef9aba0 msgr2=0x55b8eefa1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 shutdown_connections 2024-04-22T13:36:32.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 1 -- 172.21.15.5:0/72120975 wait complete. 2024-04-22T13:36:32.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.072+0000 7f9eb8e59740 20 client.18852 trim_cache size 0 max 16384 2024-04-22T13:36:32.075 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e980a3d20 msgr2=0x7f9e980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e980a3d20 0x7f9e980a61e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8022000 tx=0x7f9ea8023000 comp rx=0 tx=0).stop 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 msgr2=0x7f9eb01a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 0x7f9eb01a0bd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000d950 tx=0x7f9ea000de10 comp rx=0 tx=0).stop 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 shutdown_connections 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb010a700 0x7f9eb01a0bd0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f9e980a3d20 0x7f9e980a61e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9eb019a750 0x7f9eb019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 --2- 172.21.15.5:0/2589789549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb0100a80 0x7f9eb01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 >> 172.21.15.5:0/2589789549 conn(0x7f9eb0076170 msgr2=0x7f9eb01081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 shutdown_connections 2024-04-22T13:36:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.073+0000 7f9eb6270640 1 -- 172.21.15.5:0/2589789549 wait complete. 2024-04-22T13:36:32.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:36:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 -- 172.21.15.5:0/27643892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394100810 msgr2=0x7f3394100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 --2- 172.21.15.5:0/27643892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394100810 0x7f3394100c90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3384009930 tx=0x7f338402bcd0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 -- 172.21.15.5:0/27643892 shutdown_connections 2024-04-22T13:36:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 --2- 172.21.15.5:0/27643892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394100810 0x7f3394100c90 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 --2- 172.21.15.5:0/27643892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33941011d0 0x7f339410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 --2- 172.21.15.5:0/27643892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3394108150 0x7f3394108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 -- 172.21.15.5:0/27643892 >> 172.21.15.5:0/27643892 conn(0x7f3394076170 msgr2=0x7f3394076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 -- 172.21.15.5:0/27643892 shutdown_connections 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.188+0000 7f339bab2640 1 -- 172.21.15.5:0/27643892 wait complete. 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 Processor -- start 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 -- start start 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394100810 0x7f339419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33941011d0 0x7f339419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 0x7f3394198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3394112220 con 0x7f3394108150 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33941120a0 con 0x7f33941011d0 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339bab2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33941123a0 con 0x7f3394100810 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 0x7f3394198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 0x7f3394198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51144/0 (socket says 172.21.15.5:51144) 2024-04-22T13:36:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 -- 172.21.15.5:0/784543392 learned_addr learned my addr 172.21.15.5:0/784543392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339984e640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394100810 0x7f339419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339904d640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33941011d0 0x7f339419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 -- 172.21.15.5:0/784543392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394100810 msgr2=0x7f339419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394100810 0x7f339419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 -- 172.21.15.5:0/784543392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33941011d0 msgr2=0x7f339419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33941011d0 0x7f339419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.189+0000 7f339a04f640 1 -- 172.21.15.5:0/784543392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3394198d80 con 0x7f3394108150 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.190+0000 7f339a04f640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 0x7f3394198600 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f338c00eaa0 tx=0x7f338c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.190+0000 7f338affd640 1 -- 172.21.15.5:0/784543392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c00cdb0 con 0x7f3394108150 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.190+0000 7f338affd640 1 -- 172.21.15.5:0/784543392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f338c005020 con 0x7f3394108150 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.190+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3394199070 con 0x7f3394108150 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.190+0000 7f338affd640 1 -- 172.21.15.5:0/784543392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0116d0 con 0x7f3394108150 2024-04-22T13:36:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.190+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33941a5310 con 0x7f3394108150 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.192+0000 7f338affd640 1 -- 172.21.15.5:0/784543392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f338c0051d0 con 0x7f3394108150 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.192+0000 7f339e776740 1 Processor -- start 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 -- start start 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a74d0 0x55c19c8a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 0x55c19c8b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c19c80c1f0 con 0x55c19c8a7e80 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c19c7a94b0 con 0x55c19c8b0a00 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339e776740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c19c8081b0 con 0x55c19c8a74d0 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 0x55c19c8b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 0x55c19c8b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51150/0 (socket says 172.21.15.5:51150) 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 -- 172.21.15.5:0/1969748689 learned_addr learned my addr 172.21.15.5:0/1969748689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f338affd640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f33800a3d20 0x7f33800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339904d640 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a74d0 0x55c19c8a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339904d640 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339a04f640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f33800a3d20 0x7f33800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 -- 172.21.15.5:0/1969748689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a74d0 msgr2=0x55c19c8a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a74d0 0x55c19c8a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 -- 172.21.15.5:0/1969748689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 msgr2=0x55c19c8b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339984e640 1 -- 172.21.15.5:0/1969748689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19c79b1b0 con 0x55c19c8a7e80 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f339904d640 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.193+0000 7f338affd640 1 -- 172.21.15.5:0/784543392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f338c0d0ad0 con 0x7f3394108150 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f339984e640 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 0x55c19c8b0430 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3390009c80 tx=0x7f33900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83631fa7b4172a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f339a04f640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f33800a3d20 0x7f33800a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f338c00efd0 tx=0x7f338c07d040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f337ffff640 1 -- 172.21.15.5:0/1969748689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390012450 con 0x55c19c8a7e80 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f337ffff640 1 -- 172.21.15.5:0/1969748689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3390012aa0 con 0x55c19c8a7e80 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f337ffff640 1 -- 172.21.15.5:0/1969748689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339000fae0 con 0x55c19c8a7e80 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f339e776740 1 -- 172.21.15.5:0/1969748689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 msgr2=0x55c19c8b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.194+0000 7f339e776740 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 0x55c19c8b0430 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3390009c80 tx=0x7f33900117b0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- 172.21.15.5:0/1969748689 shutdown_connections 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a7e80 0x55c19c8b0430 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 --2- 172.21.15.5:0/1969748689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a74d0 0x55c19c8a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- 172.21.15.5:0/1969748689 >> 172.21.15.5:0/1969748689 conn(0x55c19c8a1ba0 msgr2=0x55c19c8a1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- 172.21.15.5:0/1969748689 shutdown_connections 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- 172.21.15.5:0/1969748689 wait complete. 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- start start 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 Processor -- start 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8b0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c8ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c19c8b3970 con 0x55c19c8a74d0 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c19c7c5500 con 0x55c19c8b0a00 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.195+0000 7f339e776740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c19c8b3af0 con 0x55c19c8a7e80 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339e776740 10 client.0 _ll_register_callbacks cb 0x55c19c793b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51162/0 (socket says 172.21.15.5:51162) 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 -- 172.21.15.5:0/3281941998 learned_addr learned my addr 172.21.15.5:0/3281941998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339984e640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c8ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 -- 172.21.15.5:0/3281941998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 msgr2=0x55c19c8ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c8ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 -- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 msgr2=0x55c19c8adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 -- 172.21.15.5:0/3281941998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19c51e3f0 con 0x55c19c8a74d0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339984e640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c8ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339904d640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8b0290 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3384036040 tx=0x7f338402c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e258d6bee57036 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f337d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f337d7fa640 1 -- 172.21.15.5:0/3281941998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384047020 con 0x55c19c8a74d0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f337d7fa640 1 -- 172.21.15.5:0/3281941998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3384005030 con 0x55c19c8a74d0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3390012aa0 con 0x55c19c8a74d0 2024-04-22T13:36:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f337d7fa640 1 -- 172.21.15.5:0/3281941998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3384052050 con 0x55c19c8a74d0 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339e776740 10 client.18864 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.196+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c19c5e87d0 con 0x55c19c8a74d0 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f337d7fa640 1 -- 172.21.15.5:0/3281941998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3390012aa0 con 0x55c19c8a74d0 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f337d7fa640 1 -- 172.21.15.5:0/3281941998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3384040040 con 0x55c19c8a74d0 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339e776740 10 client.18864 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339e776740 10 client.18864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339e776740 20 client.18864 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339e776740 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c8c0be0 0x55c19c8c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339e776740 4 client.18864 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c19c7ab930 con 0x55c19c8c0be0 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339984e640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c8c0be0 0x55c19c8c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f337d7fa640 10 client.18864 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.197+0000 7f339984e640 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c8c0be0 0x55c19c8c3050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89153425f9d2dd24 server_cookie=28ee3235388c4f9a in_seq=0 out_seq=0 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f337d7fa640 1 -- 172.21.15.5:0/3281941998 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c19c7ab930 con 0x55c19c8c0be0 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f337d7fa640 10 client.18864 handle_command_reply: tid=0 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 client.18864 shutdown 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 msgr2=0x55c19c8b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8b0290 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3384036040 tx=0x7f338402c350 comp rx=0 tx=0).stop 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 shutdown_connections 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c8c0be0 0x55c19c8c3050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8b0a00 0x55c19c8adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c8ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 --2- 172.21.15.5:0/3281941998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8b0290 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f337d7fa640 0 client.18864 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 >> 172.21.15.5:0/3281941998 conn(0x55c19c8a1ba0 msgr2=0x55c19c8a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 shutdown_connections 2024-04-22T13:36:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.198+0000 7f339e776740 1 -- 172.21.15.5:0/3281941998 wait complete. 2024-04-22T13:36:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.199+0000 7f339e776740 20 client.18864 trim_cache size 0 max 16384 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 Processor -- start 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 -- start start 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a7e80 0x55c19c8a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8b0a00 0x55c19c8afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33941adcd0 con 0x55c19c8a74d0 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33941af810 con 0x55c19c8a7e80 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339e776740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33941ae3a0 con 0x55c19c8b0a00 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8bc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51170/0 (socket says 172.21.15.5:51170) 2024-04-22T13:36:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 -- 172.21.15.5:0/1196975455 learned_addr learned my addr 172.21.15.5:0/1196975455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339984e640 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a7e80 0x55c19c8a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 -- 172.21.15.5:0/1196975455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8b0a00 msgr2=0x55c19c8afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8b0a00 0x55c19c8afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 -- 172.21.15.5:0/1196975455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a7e80 msgr2=0x55c19c8a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a7e80 0x55c19c8a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.212+0000 7f339904d640 1 -- 172.21.15.5:0/1196975455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19c8a9250 con 0x55c19c8a74d0 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339904d640 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8bc890 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f338402c880 tx=0x7f33840098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0873d983ba05ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f337f7fe640 1 -- 172.21.15.5:0/1196975455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384047040 con 0x55c19c8a74d0 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f337f7fe640 1 -- 172.21.15.5:0/1196975455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3384004310 con 0x55c19c8a74d0 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f337f7fe640 1 -- 172.21.15.5:0/1196975455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384005200 con 0x55c19c8a74d0 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 -- 172.21.15.5:0/1196975455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 msgr2=0x55c19c8bc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8bc890 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f338402c880 tx=0x7f33840098e0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 -- 172.21.15.5:0/1196975455 shutdown_connections 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8b0a00 0x55c19c8afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a7e80 0x55c19c8a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 --2- 172.21.15.5:0/1196975455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8a74d0 0x55c19c8bc890 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 -- 172.21.15.5:0/1196975455 >> 172.21.15.5:0/1196975455 conn(0x55c19c8a1ba0 msgr2=0x55c19c8a8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 -- 172.21.15.5:0/1196975455 shutdown_connections 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.213+0000 7f339e776740 1 -- 172.21.15.5:0/1196975455 wait complete. 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 -- start start 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 Processor -- start 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a74d0 0x55c19c923df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c924330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 0x55c19c91e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33941af140 con 0x55c19c8b0a00 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33941aea70 con 0x55c19c8a74d0 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33941abcd0 con 0x55c19c8a7e80 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339e776740 10 client.0 _ll_register_callbacks cb 0x7f338402cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339904d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 0x55c19c91e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339904d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 0x55c19c91e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51172/0 (socket says 172.21.15.5:51172) 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339904d640 1 -- 172.21.15.5:0/3868025143 learned_addr learned my addr 172.21.15.5:0/3868025143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339904d640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a74d0 0x55c19c923df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.214+0000 7f339984e640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c924330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339904d640 1 -- 172.21.15.5:0/3868025143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 msgr2=0x55c19c924330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339904d640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c924330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339904d640 1 -- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a74d0 msgr2=0x55c19c923df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339904d640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a74d0 0x55c19c923df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339904d640 1 -- 172.21.15.5:0/3868025143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19c91e8f0 con 0x55c19c8b0a00 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339904d640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 0x55c19c91e200 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3384006010 tx=0x7f338404e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509961ab49525759 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f337d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f337d7fa640 1 -- 172.21.15.5:0/3868025143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338405b040 con 0x55c19c8b0a00 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f337d7fa640 1 -- 172.21.15.5:0/3868025143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f33840038c0 con 0x55c19c8b0a00 2024-04-22T13:36:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3384004310 con 0x55c19c8b0a00 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f337d7fa640 1 -- 172.21.15.5:0/3868025143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3384003ac0 con 0x55c19c8b0a00 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339e776740 10 client.18870 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c19c92b5f0 con 0x55c19c8b0a00 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f337d7fa640 1 -- 172.21.15.5:0/3868025143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3384004310 con 0x55c19c8b0a00 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f337d7fa640 1 -- 172.21.15.5:0/3868025143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3384061070 con 0x55c19c8b0a00 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339e776740 10 client.18870 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.215+0000 7f339e776740 10 client.18870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 20 client.18870 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c92bc50 0x55c19c92e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 4 client.18870 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c19c819c10 con 0x55c19c92bc50 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339984e640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c92bc50 0x55c19c92e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339984e640 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c92bc50 0x55c19c92e070 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47f9121fc246ac07 server_cookie=bf2c393a64144bb2 in_seq=0 out_seq=0 2024-04-22T13:36:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f337d7fa640 10 client.18870 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f337d7fa640 1 -- 172.21.15.5:0/3868025143 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c19c819c10 con 0x55c19c92bc50 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f337d7fa640 10 client.18870 handle_command_reply: tid=0 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 1 client.18870 shutdown 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 msgr2=0x55c19c91e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.216+0000 7f339e776740 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 0x55c19c91e200 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3384006010 tx=0x7f338404e040 comp rx=0 tx=0).stop 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 shutdown_connections 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55c19c92bc50 0x55c19c92e070 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c8b0a00 0x55c19c91e200 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c19c8a7e80 0x55c19c924330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f337d7fa640 0 client.18870 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 --2- 172.21.15.5:0/3868025143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c19c8a74d0 0x55c19c923df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 >> 172.21.15.5:0/3868025143 conn(0x55c19c8a1ba0 msgr2=0x55c19c8bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 shutdown_connections 2024-04-22T13:36:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 1 -- 172.21.15.5:0/3868025143 wait complete. 2024-04-22T13:36:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339e776740 20 client.18870 trim_cache size 0 max 16384 2024-04-22T13:36:32.219 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f33800a3d20 msgr2=0x7f33800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f33800a3d20 0x7f33800a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f338c00efd0 tx=0x7f338c07d040 comp rx=0 tx=0).stop 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 msgr2=0x7f3394198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 0x7f3394198600 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f338c00eaa0 tx=0x7f338c00ef60 comp rx=0 tx=0).stop 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 shutdown_connections 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3394108150 0x7f3394198600 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f33800a3d20 0x7f33800a61e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33941011d0 0x7f339419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 --2- 172.21.15.5:0/784543392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394100810 0x7f339419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 >> 172.21.15.5:0/784543392 conn(0x7f3394076170 msgr2=0x7f33940ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 shutdown_connections 2024-04-22T13:36:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.217+0000 7f339bab2640 1 -- 172.21.15.5:0/784543392 wait complete. 2024-04-22T13:36:32.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:32 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2480865530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:36:32.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:36:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 -- 172.21.15.5:0/2371129607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102b40 msgr2=0x7f359c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 --2- 172.21.15.5:0/2371129607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102b40 0x7f359c106b80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 -- 172.21.15.5:0/2371129607 shutdown_connections 2024-04-22T13:36:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 --2- 172.21.15.5:0/2371129607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102b40 0x7f359c106b80 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 --2- 172.21.15.5:0/2371129607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f359c107260 0x7f359c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 --2- 172.21.15.5:0/2371129607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c102190 0x7f359c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 -- 172.21.15.5:0/2371129607 >> 172.21.15.5:0/2371129607 conn(0x7f359c0fda10 msgr2=0x7f359c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 -- 172.21.15.5:0/2371129607 shutdown_connections 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.332+0000 7f35a3359640 1 -- 172.21.15.5:0/2371129607 wait complete. 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 Processor -- start 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 -- start start 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 0x7f359c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f359c107260 0x7f359c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c19d000 0x7f359c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112180 con 0x7f359c102190 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112000 con 0x7f359c19d000 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a3359640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112300 con 0x7f359c107260 2024-04-22T13:36:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a10f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 0x7f359c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a10f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 0x7f359c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51194/0 (socket says 172.21.15.5:51194) 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a10f5640 1 -- 172.21.15.5:0/2329897363 learned_addr learned my addr 172.21.15.5:0/2329897363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.333+0000 7f35a08f4640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f359c107260 0x7f359c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a18f6640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c19d000 0x7f359c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a10f5640 1 -- 172.21.15.5:0/2329897363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f359c107260 msgr2=0x7f359c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a10f5640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f359c107260 0x7f359c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a10f5640 1 -- 172.21.15.5:0/2329897363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c19d000 msgr2=0x7f359c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a10f5640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c19d000 0x7f359c19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a10f5640 1 -- 172.21.15.5:0/2329897363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c19daf0 con 0x7f359c102190 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a10f5640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 0x7f359c198740 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f359000c890 tx=0x7f359000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f358e7fc640 1 -- 172.21.15.5:0/2329897363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590004220 con 0x7f359c102190 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f358e7fc640 1 -- 172.21.15.5:0/2329897363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35900043c0 con 0x7f359c102190 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c19dde0 con 0x7f359c102190 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f358e7fc640 1 -- 172.21.15.5:0/2329897363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590004a00 con 0x7f359c102190 2024-04-22T13:36:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.334+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a55a0 con 0x7f359c102190 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 Processor -- start 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f358e7fc640 1 -- 172.21.15.5:0/2329897363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f35900079b0 con 0x7f359c102190 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 -- start start 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 0x55818eca1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818eca1a00 0x55818eca3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55818ebfd1f0 con 0x55818ec98e80 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55818eb9a4b0 con 0x55818eca1a00 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a602d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55818ebf91b0 con 0x55818ec984d0 2024-04-22T13:36:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a10f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 0x55818eca1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a10f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 0x55818eca1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51204/0 (socket says 172.21.15.5:51204) 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a10f5640 1 -- 172.21.15.5:0/453029209 learned_addr learned my addr 172.21.15.5:0/453029209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a18f6640 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.337+0000 7f35a08f4640 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818eca1a00 0x55818eca3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a18f6640 1 -- 172.21.15.5:0/453029209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818eca1a00 msgr2=0x55818eca3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a18f6640 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818eca1a00 0x55818eca3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a18f6640 1 -- 172.21.15.5:0/453029209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 msgr2=0x55818eca1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a18f6640 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 0x55818eca1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a18f6640 1 -- 172.21.15.5:0/453029209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818eb8c1b0 con 0x55818ec984d0 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a10f5640 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 0x55818eca1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f358e7fc640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35880a3d20 0x7f35880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a10f5640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35880a3d20 0x7f35880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35a18f6640 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec988b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3594009e50 tx=0x7f35940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70f09bc9c570ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f358e7fc640 1 -- 172.21.15.5:0/2329897363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f35900d0550 con 0x7f359c102190 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35877fe640 1 -- 172.21.15.5:0/453029209 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594003b70 con 0x55818ec984d0 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35877fe640 1 -- 172.21.15.5:0/453029209 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594015070 con 0x55818ec984d0 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.338+0000 7f35877fe640 1 -- 172.21.15.5:0/453029209 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35940116e0 con 0x55818ec984d0 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a10f5640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35880a3d20 0x7f35880a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55818ec93c10 tx=0x7f35900d6110 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 -- 172.21.15.5:0/453029209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 msgr2=0x55818ec988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec988b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3594009e50 tx=0x7f35940107a0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 -- 172.21.15.5:0/453029209 shutdown_connections 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec98e80 0x55818eca1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818eca1a00 0x55818eca3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 --2- 172.21.15.5:0/453029209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec988b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 -- 172.21.15.5:0/453029209 >> 172.21.15.5:0/453029209 conn(0x55818ec92ba0 msgr2=0x55818ec92f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 -- 172.21.15.5:0/453029209 shutdown_connections 2024-04-22T13:36:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 -- 172.21.15.5:0/453029209 wait complete. 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 -- start start 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.339+0000 7f35a602d740 1 Processor -- start 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec984d0 0x55818eca12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec98e80 0x55818ec9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55818eca4970 con 0x55818eca1a00 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55818ebb6500 con 0x55818ec984d0 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55818eca4af0 con 0x55818ec98e80 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a602d740 10 client.0 _ll_register_callbacks cb 0x55818eb84b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51216/0 (socket says 172.21.15.5:51216) 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 -- 172.21.15.5:0/3451729629 learned_addr learned my addr 172.21.15.5:0/3451729629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a08f4640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec98e80 0x55818ec9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec984d0 0x55818eca12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 -- 172.21.15.5:0/3451729629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec98e80 msgr2=0x55818ec9e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec98e80 0x55818ec9e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 -- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec984d0 msgr2=0x55818eca12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec984d0 0x55818eca12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 -- 172.21.15.5:0/3451729629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818e90f3f0 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f35a18f6640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9ec00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3594004710 tx=0x7f35940048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cd49930ffdd47b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f3584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.340+0000 7f3584ff9640 1 -- 172.21.15.5:0/3451729629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594025040 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3594015070 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f3584ff9640 1 -- 172.21.15.5:0/3451729629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3594004a40 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f3584ff9640 1 -- 172.21.15.5:0/3451729629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3594033050 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 10 client.18882 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55818e9d97d0 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f3584ff9640 1 -- 172.21.15.5:0/3451729629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594015070 con 0x55818eca1a00 2024-04-22T13:36:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f3584ff9640 1 -- 172.21.15.5:0/3451729629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3594011940 con 0x55818eca1a00 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 10 client.18882 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 20 client.18882 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ecb1b40 0x55818ecb3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 4 client.18882 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55818eb9c930 con 0x55818ecb1b40 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.341+0000 7f35a08f4640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ecb1b40 0x55818ecb3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.342+0000 7f35a08f4640 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ecb1b40 0x55818ecb3f30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84afa8a5be15c3fc server_cookie=91f3fbd48a66ea2e in_seq=0 out_seq=0 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.342+0000 7f3584ff9640 10 client.18882 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f3584ff9640 1 -- 172.21.15.5:0/3451729629 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55818eb9c930 con 0x55818ecb1b40 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f3584ff9640 10 client.18882 handle_command_reply: tid=0 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 client.18882 shutdown 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 msgr2=0x55818ec9ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9ec00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3594004710 tx=0x7f35940048b0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 shutdown_connections 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ecb1b40 0x55818ecb3f30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9ec00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec98e80 0x55818ec9e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 --2- 172.21.15.5:0/3451729629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec984d0 0x55818eca12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f3584ff9640 0 client.18882 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 >> 172.21.15.5:0/3451729629 conn(0x55818ec92ba0 msgr2=0x55818ec948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 shutdown_connections 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 1 -- 172.21.15.5:0/3451729629 wait complete. 2024-04-22T13:36:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.343+0000 7f35a602d740 20 client.18882 trim_cache size 0 max 16384 2024-04-22T13:36:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.356+0000 7f35a602d740 1 Processor -- start 2024-04-22T13:36:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.356+0000 7f35a602d740 1 -- start start 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.356+0000 7f35a602d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 0x55818ec91e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.356+0000 7f35a602d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ecad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.356+0000 7f35a602d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818eca1a00 0x55818eca0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a602d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1adc80 con 0x55818ec984d0 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a602d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1af7c0 con 0x55818ec98e80 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a602d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1ae350 con 0x55818eca1a00 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 0x55818ec91e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 0x55818ec91e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51226/0 (socket says 172.21.15.5:51226) 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 -- 172.21.15.5:0/298562425 learned_addr learned my addr 172.21.15.5:0/298562425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a08f4640 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ecad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 -- 172.21.15.5:0/298562425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818eca1a00 msgr2=0x55818eca0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818eca1a00 0x55818eca0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 -- 172.21.15.5:0/298562425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 msgr2=0x55818ecad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ecad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 -- 172.21.15.5:0/298562425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818ec923f0 con 0x55818ec984d0 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a08f4640 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ecad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f35a18f6640 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 0x55818ec91e40 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3594000f80 tx=0x7f35940092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10c1b7c5f936a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f3586ffd640 1 -- 172.21.15.5:0/298562425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359403b070 con 0x55818ec984d0 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f3586ffd640 1 -- 172.21.15.5:0/298562425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594015040 con 0x55818ec984d0 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.357+0000 7f3586ffd640 1 -- 172.21.15.5:0/298562425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594003bd0 con 0x55818ec984d0 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 -- 172.21.15.5:0/298562425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 msgr2=0x55818ec91e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 0x55818ec91e40 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3594000f80 tx=0x7f35940092a0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 -- 172.21.15.5:0/298562425 shutdown_connections 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818eca1a00 0x55818eca0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ecad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 --2- 172.21.15.5:0/298562425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818ec984d0 0x55818ec91e40 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 -- 172.21.15.5:0/298562425 >> 172.21.15.5:0/298562425 conn(0x55818ec92ba0 msgr2=0x55818ec99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 -- 172.21.15.5:0/298562425 shutdown_connections 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 -- 172.21.15.5:0/298562425 wait complete. 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 -- start start 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.358+0000 7f35a602d740 1 Processor -- start 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec9b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ec9b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1af0f0 con 0x55818eca1a00 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1aea20 con 0x55818ec98e80 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1abc80 con 0x55818ec984d0 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 10 client.0 _ll_register_callbacks cb 0x7f3594011750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51242/0 (socket says 172.21.15.5:51242) 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 -- 172.21.15.5:0/1781468108 learned_addr learned my addr 172.21.15.5:0/1781468108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a08f4640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ec9b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec9b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 -- 172.21.15.5:0/1781468108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 msgr2=0x55818ec9b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec9b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 -- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 msgr2=0x55818ec9b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ec9b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 -- 172.21.15.5:0/1781468108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818ed16f10 con 0x55818eca1a00 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a18f6640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9beb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3594011b70 tx=0x7f3594005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa80a9371fe4f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f3584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f3584ff9640 1 -- 172.21.15.5:0/1781468108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359403b040 con 0x55818eca1a00 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f3584ff9640 1 -- 172.21.15.5:0/1781468108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3594004410 con 0x55818eca1a00 2024-04-22T13:36:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3594015040 con 0x55818eca1a00 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f3584ff9640 1 -- 172.21.15.5:0/1781468108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3594003970 con 0x55818eca1a00 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 10 client.18888 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.359+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55818ed18180 con 0x55818eca1a00 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f3584ff9640 1 -- 172.21.15.5:0/1781468108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594015040 con 0x55818eca1a00 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f3584ff9640 1 -- 172.21.15.5:0/1781468108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f35940052a0 con 0x55818eca1a00 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a602d740 10 client.18888 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a602d740 10 client.18888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a602d740 20 client.18888 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a602d740 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ed186e0 0x55818ed1aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a602d740 4 client.18888 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55818ec0ac10 con 0x55818ed186e0 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.360+0000 7f35a08f4640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ed186e0 0x55818ed1aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f3584ff9640 10 client.18888 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a08f4640 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ed186e0 0x55818ed1aad0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdf35ef27213f1a server_cookie=1ab606e35e24c550 in_seq=0 out_seq=0 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f3584ff9640 1 -- 172.21.15.5:0/1781468108 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55818ec0ac10 con 0x55818ed186e0 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f3584ff9640 10 client.18888 handle_command_reply: tid=0 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 client.18888 shutdown 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 msgr2=0x55818ec9beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9beb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3594011b70 tx=0x7f3594005cf0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 shutdown_connections 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55818ed186e0 0x55818ed1aad0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818eca1a00 0x55818ec9beb0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55818ec98e80 0x55818ec9b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f3584ff9640 0 client.18888 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 --2- 172.21.15.5:0/1781468108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55818ec984d0 0x55818ec9b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 >> 172.21.15.5:0/1781468108 conn(0x55818ec92ba0 msgr2=0x55818ec93e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 shutdown_connections 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 1 -- 172.21.15.5:0/1781468108 wait complete. 2024-04-22T13:36:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.361+0000 7f35a602d740 20 client.18888 trim_cache size 0 max 16384 2024-04-22T13:36:32.364 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:36:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35880a3d20 msgr2=0x7f35880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35880a3d20 0x7f35880a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55818ec93c10 tx=0x7f35900d6110 comp rx=0 tx=0).stop 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 msgr2=0x7f359c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 0x7f359c198740 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f359000c890 tx=0x7f359000cd50 comp rx=0 tx=0).stop 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 shutdown_connections 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f35880a3d20 0x7f35880a61e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f359c19d000 0x7f359c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f359c107260 0x7f359c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 --2- 172.21.15.5:0/2329897363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359c102190 0x7f359c198740 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 >> 172.21.15.5:0/2329897363 conn(0x7f359c0fda10 msgr2=0x7f359c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 shutdown_connections 2024-04-22T13:36:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:32.362+0000 7f35a3359640 1 -- 172.21.15.5:0/2329897363 wait complete. 2024-04-22T13:36:32.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:32 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2480865530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:36:32.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:32 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2480865530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:36:33.369 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:36:33.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:36:33.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:36:33.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:33 smithi005 ceph-mon[28006]: pgmap v291: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:33.434 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:33 smithi107 ceph-mon[31859]: pgmap v291: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 -- 172.21.15.5:0/2895940012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8103b50 msgr2=0x7fb8b8103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/2895940012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8103b50 0x7fb8b8103fb0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009930 tx=0x7fb8a802bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 -- 172.21.15.5:0/2895940012 shutdown_connections 2024-04-22T13:36:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/2895940012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8103b50 0x7fb8b8103fb0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/2895940012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8b81044f0 0x7fb8b810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/2895940012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b8102270 0x7fb8b8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 -- 172.21.15.5:0/2895940012 >> 172.21.15.5:0/2895940012 conn(0x7fb8b80fda10 msgr2=0x7fb8b80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 -- 172.21.15.5:0/2895940012 shutdown_connections 2024-04-22T13:36:33.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.492+0000 7fb8bfb96640 1 -- 172.21.15.5:0/2895940012 wait complete. 2024-04-22T13:36:33.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 Processor -- start 2024-04-22T13:36:33.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 -- start start 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 0x7fb8b819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8b8103b50 0x7fb8b819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b81044f0 0x7fb8b8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112290 con 0x7fb8b8102270 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112110 con 0x7fb8b8103b50 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bfb96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112410 con 0x7fb8b81044f0 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bd932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 0x7fb8b819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bd932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 0x7fb8b819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51256/0 (socket says 172.21.15.5:51256) 2024-04-22T13:36:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bd932640 1 -- 172.21.15.5:0/625167845 learned_addr learned my addr 172.21.15.5:0/625167845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.493+0000 7fb8bd131640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8b8103b50 0x7fb8b819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8be133640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b81044f0 0x7fb8b8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bd932640 1 -- 172.21.15.5:0/625167845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b81044f0 msgr2=0x7fb8b8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bd932640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b81044f0 0x7fb8b8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bd932640 1 -- 172.21.15.5:0/625167845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8b8103b50 msgr2=0x7fb8b819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bd932640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8b8103b50 0x7fb8b819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bd932640 1 -- 172.21.15.5:0/625167845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b8198d90 con 0x7fb8b8102270 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8be133640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b81044f0 0x7fb8b8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bd932640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 0x7fb8b819e460 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e9f0 tx=0x7fb8b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8aeffd640 1 -- 172.21.15.5:0/625167845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400cd80 con 0x7fb8b8102270 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b8199080 con 0x7fb8b8102270 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8aeffd640 1 -- 172.21.15.5:0/625167845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b40054d0 con 0x7fb8b8102270 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8aeffd640 1 -- 172.21.15.5:0/625167845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4010690 con 0x7fb8b8102270 2024-04-22T13:36:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.494+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b81a5320 con 0x7fb8b8102270 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.496+0000 7fb8aeffd640 1 -- 172.21.15.5:0/625167845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb8b4004e90 con 0x7fb8b8102270 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 Processor -- start 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 -- start start 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a1782cf00 con 0x559a178c8740 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a177ca590 con 0x559a178c90f0 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8c2785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a17828ec0 con 0x559a178d2250 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8aeffd640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb8a40a3d20 0x7fb8a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.497+0000 7fb8be133640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8be133640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51270/0 (socket says 172.21.15.5:51270) 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8be133640 1 -- 172.21.15.5:0/1269514281 learned_addr learned my addr 172.21.15.5:0/1269514281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd932640 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8be133640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb8a40a3d20 0x7fb8a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8aeffd640 1 -- 172.21.15.5:0/625167845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb8b40d0d10 con 0x7fb8b8102270 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 -- 172.21.15.5:0/1269514281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 msgr2=0x559a178d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 -- 172.21.15.5:0/1269514281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 msgr2=0x559a178c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 -- 172.21.15.5:0/1269514281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a176077e0 con 0x559a178d2250 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8be133640 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8bd131640 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d26d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009230 tx=0x7fb8a8009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe22c303bfdb41f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8be133640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb8a40a3d20 0x7fb8a40a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009fd0 tx=0x7fb8b0004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8a3fff640 1 -- 172.21.15.5:0/1269514281 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8045020 con 0x559a178d2250 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8a3fff640 1 -- 172.21.15.5:0/1269514281 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a802c7e0 con 0x559a178d2250 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.498+0000 7fb8a3fff640 1 -- 172.21.15.5:0/1269514281 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80055b0 con 0x559a178d2250 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 -- 172.21.15.5:0/1269514281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 msgr2=0x559a178d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d26d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009230 tx=0x7fb8a8009610 comp rx=0 tx=0).stop 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 -- 172.21.15.5:0/1269514281 shutdown_connections 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d26d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 --2- 172.21.15.5:0/1269514281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 -- 172.21.15.5:0/1269514281 >> 172.21.15.5:0/1269514281 conn(0x559a178c2fb0 msgr2=0x559a178c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 -- 172.21.15.5:0/1269514281 shutdown_connections 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.499+0000 7fb8c2785740 1 -- 172.21.15.5:0/1269514281 wait complete. 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 -- start start 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 Processor -- start 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a178d9170 con 0x559a178c8740 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559a177e6580 con 0x559a178c90f0 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a178d92f0 con 0x559a178d2250 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8c2785740 10 client.0 _ll_register_callbacks cb 0x559a177ec650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8bd932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8bd932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178d5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51272/0 (socket says 172.21.15.5:51272) 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8bd932640 1 -- 172.21.15.5:0/127528292 learned_addr learned my addr 172.21.15.5:0/127528292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8be133640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.500+0000 7fb8bd131640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd932640 1 -- 172.21.15.5:0/127528292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 msgr2=0x559a178d6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd932640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd932640 1 -- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 msgr2=0x559a178d6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd932640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd932640 1 -- 172.21.15.5:0/127528292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a17667f70 con 0x559a178c8740 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd131640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8bd932640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178d5cf0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4099620 tx=0x7fb8b406c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc44131be17cd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8a17fa640 1 -- 172.21.15.5:0/127528292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40ce880 con 0x559a178c8740 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8a17fa640 1 -- 172.21.15.5:0/127528292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb8b40bd0b0 con 0x559a178c8740 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a802c7e0 con 0x559a178c8740 2024-04-22T13:36:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8a17fa640 1 -- 172.21.15.5:0/127528292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8b40bd2b0 con 0x559a178c8740 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8c2785740 10 client.18900 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a174ff170 con 0x559a178c8740 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.501+0000 7fb8a17fa640 1 -- 172.21.15.5:0/127528292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a802c7e0 con 0x559a178c8740 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8a17fa640 1 -- 172.21.15.5:0/127528292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb8b40a5430 con 0x559a178c8740 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8c2785740 10 client.18900 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8c2785740 10 client.18900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8c2785740 20 client.18900 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8c2785740 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a178e6780 0x559a178e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8c2785740 4 client.18900 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a1783a920 con 0x559a178e6780 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8bd131640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a178e6780 0x559a178e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8bd131640 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a178e6780 0x559a178e8c10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8252f4321ef27dad server_cookie=f217e0b7b2e8a601 in_seq=0 out_seq=0 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.502+0000 7fb8a17fa640 10 client.18900 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8a17fa640 1 -- 172.21.15.5:0/127528292 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a1783a920 con 0x559a178e6780 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8a17fa640 10 client.18900 handle_command_reply: tid=0 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 client.18900 shutdown 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 msgr2=0x559a178d5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178d5cf0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4099620 tx=0x7fb8b406c5f0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 shutdown_connections 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a178e6780 0x559a178e8c10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178d2250 0x559a178d6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178c90f0 0x559a178d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8a17fa640 0 client.18900 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 --2- 172.21.15.5:0/127528292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178d5cf0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 >> 172.21.15.5:0/127528292 conn(0x559a178c2fb0 msgr2=0x559a178c4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 shutdown_connections 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 1 -- 172.21.15.5:0/127528292 wait complete. 2024-04-22T13:36:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.503+0000 7fb8c2785740 20 client.18900 trim_cache size 0 max 16384 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.521+0000 7fb8c2785740 1 Processor -- start 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.521+0000 7fb8c2785740 1 -- start start 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.521+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c8740 0x559a178d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 0x559a178e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a178d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8c2785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ab8b0 con 0x559a178c90f0 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8c2785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ad3f0 con 0x559a178d2250 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8c2785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81abf80 con 0x559a178c8740 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 0x559a178e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 0x559a178e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51286/0 (socket says 172.21.15.5:51286) 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 -- 172.21.15.5:0/736099616 learned_addr learned my addr 172.21.15.5:0/736099616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8be133640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a178d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd932640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c8740 0x559a178d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 -- 172.21.15.5:0/736099616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c8740 msgr2=0x559a178d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c8740 0x559a178d8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 -- 172.21.15.5:0/736099616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 msgr2=0x559a178d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a178d3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 -- 172.21.15.5:0/736099616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a178c8e90 con 0x559a178c90f0 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd932640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c8740 0x559a178d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8be133640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a178d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8bd131640 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 0x559a178e2070 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005ab0 tx=0x7fb8a802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e389d3d74d4af66 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8a37fe640 1 -- 172.21.15.5:0/736099616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8045040 con 0x559a178c90f0 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8a37fe640 1 -- 172.21.15.5:0/736099616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a8004040 con 0x559a178c90f0 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.522+0000 7fb8a37fe640 1 -- 172.21.15.5:0/736099616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8003950 con 0x559a178c90f0 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 -- 172.21.15.5:0/736099616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 msgr2=0x559a178e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 0x559a178e2070 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005ab0 tx=0x7fb8a802bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 -- 172.21.15.5:0/736099616 shutdown_connections 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c90f0 0x559a178e2070 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a178d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 --2- 172.21.15.5:0/736099616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c8740 0x559a178d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 -- 172.21.15.5:0/736099616 >> 172.21.15.5:0/736099616 conn(0x559a178c2fb0 msgr2=0x559a178c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 -- 172.21.15.5:0/736099616 shutdown_connections 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 -- 172.21.15.5:0/736099616 wait complete. 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 -- start start 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.523+0000 7fb8c2785740 1 Processor -- start 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c90f0 0x559a178ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a1794bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81acd20 con 0x559a178c8740 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ac650 con 0x559a178d2250 2024-04-22T13:36:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81a98b0 con 0x559a178c90f0 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8c2785740 10 client.0 _ll_register_callbacks cb 0x7fb8b400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51298/0 (socket says 172.21.15.5:51298) 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 -- 172.21.15.5:0/2584843810 learned_addr learned my addr 172.21.15.5:0/2584843810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd131640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c90f0 0x559a178ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 -- 172.21.15.5:0/2584843810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c90f0 msgr2=0x559a178ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c90f0 0x559a178ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 -- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 msgr2=0x559a1794bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8be133640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a1794bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a1794bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 -- 172.21.15.5:0/2584843810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1794c290 con 0x559a178c8740 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd131640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c90f0 0x559a178ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8bd932640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178ee0e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40a5390 tx=0x7fb8b40ce8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db74bc242c18264 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.524+0000 7fb8a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8a17fa640 1 -- 172.21.15.5:0/2584843810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b406c040 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8a17fa640 1 -- 172.21.15.5:0/2584843810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb8b406c3f0 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a8004040 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8a17fa640 1 -- 172.21.15.5:0/2584843810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8b409c5b0 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 10 client.18906 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a1794d4f0 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8a17fa640 1 -- 172.21.15.5:0/2584843810 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a8004040 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8a17fa640 1 -- 172.21.15.5:0/2584843810 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb8b409c230 con 0x559a178c8740 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 10 client.18906 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 10 client.18906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 20 client.18906 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a1794dae0 0x559a1794ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 4 client.18906 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:36:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a17834790 con 0x559a1794dae0 2024-04-22T13:36:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.525+0000 7fb8bd131640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a1794dae0 0x559a1794ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.526+0000 7fb8bd131640 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a1794dae0 0x559a1794ff00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5486df6573ab75 server_cookie=bf44c9cc7d28cbff in_seq=0 out_seq=0 2024-04-22T13:36:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.526+0000 7fb8a17fa640 10 client.18906 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.527 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:33 smithi049 ceph-mon[31379]: pgmap v291: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8a17fa640 1 -- 172.21.15.5:0/2584843810 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a17834790 con 0x559a1794dae0 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8a17fa640 10 client.18906 handle_command_reply: tid=0 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 client.18906 shutdown 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 msgr2=0x559a178ee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178ee0e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40a5390 tx=0x7fb8b40ce8b0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 shutdown_connections 2024-04-22T13:36:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x559a1794dae0 0x559a1794ff00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a178d2250 0x559a1794bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a178c90f0 0x559a178ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8a17fa640 0 client.18906 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 --2- 172.21.15.5:0/2584843810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a178c8740 0x559a178ee0e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 >> 172.21.15.5:0/2584843810 conn(0x559a178c2fb0 msgr2=0x559a178d6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 shutdown_connections 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 1 -- 172.21.15.5:0/2584843810 wait complete. 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.527+0000 7fb8c2785740 20 client.18906 trim_cache size 0 max 16384 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "43ee8c9f-fb56-42ac-a670-455ba16f1dd9", 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:36:33.529 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:36:33.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb8a40a3d20 msgr2=0x7fb8a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb8a40a3d20 0x7fb8a40a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009fd0 tx=0x7fb8b0004ab0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 msgr2=0x7fb8b819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 0x7fb8b819e460 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e9f0 tx=0x7fb8b400edf0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 shutdown_connections 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb8a40a3d20 0x7fb8a40a61e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b81044f0 0x7fb8b8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8b8103b50 0x7fb8b819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 --2- 172.21.15.5:0/625167845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b8102270 0x7fb8b819e460 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 >> 172.21.15.5:0/625167845 conn(0x7fb8b80fda10 msgr2=0x7fb8b80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 shutdown_connections 2024-04-22T13:36:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.528+0000 7fb8bfb96640 1 -- 172.21.15.5:0/625167845 wait complete. 2024-04-22T13:36:33.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:36:33.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 -- 172.21.15.5:0/2068297161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c101a50 msgr2=0x7f168c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 --2- 172.21.15.5:0/2068297161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c101a50 0x7f168c105a90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1680009900 tx=0x7f168002bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 -- 172.21.15.5:0/2068297161 shutdown_connections 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 --2- 172.21.15.5:0/2068297161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c101a50 0x7f168c105a90 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 --2- 172.21.15.5:0/2068297161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 0x7f168c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 --2- 172.21.15.5:0/2068297161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f168c1010a0 0x7f168c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 -- 172.21.15.5:0/2068297161 >> 172.21.15.5:0/2068297161 conn(0x7f168c077f40 msgr2=0x7f168c078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 -- 172.21.15.5:0/2068297161 shutdown_connections 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 -- 172.21.15.5:0/2068297161 wait complete. 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.685+0000 7f1691462640 1 Processor -- start 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 -- start start 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f168c1010a0 0x7f168c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 0x7f168c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 0x7f168c1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f168c111100 con 0x7f168c1035b0 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f168c110f80 con 0x7f168c106170 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f1691462640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f168c111280 con 0x7f168c1010a0 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 0x7f168c1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 0x7f168c1a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51322/0 (socket says 172.21.15.5:51322) 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 -- 172.21.15.5:0/3003036150 learned_addr learned my addr 172.21.15.5:0/3003036150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168affd640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f168c1010a0 0x7f168c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 0x7f168c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 -- 172.21.15.5:0/3003036150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f168c1010a0 msgr2=0x7f168c102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f168c1010a0 0x7f168c102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 -- 172.21.15.5:0/3003036150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 msgr2=0x7f168c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 0x7f168c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168b7fe640 1 -- 172.21.15.5:0/3003036150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c1a7020 con 0x7f168c1035b0 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.686+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 0x7f168c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.687+0000 7f168b7fe640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 0x7f168c1a6920 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f168400cad0 tx=0x7f168400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.687+0000 7f1673fff640 1 -- 172.21.15.5:0/3003036150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400ae20 con 0x7f168c1035b0 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.687+0000 7f1673fff640 1 -- 172.21.15.5:0/3003036150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1684005020 con 0x7f168c1035b0 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.687+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c1a7310 con 0x7f168c1035b0 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.687+0000 7f1673fff640 1 -- 172.21.15.5:0/3003036150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400f6b0 con 0x7f168c1035b0 2024-04-22T13:36:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.687+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c1a7810 con 0x7f168c1035b0 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.689+0000 7f1673fff640 1 -- 172.21.15.5:0/3003036150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f16840051d0 con 0x7f168c1035b0 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.689+0000 7f169404c740 1 Processor -- start 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.689+0000 7f169404c740 1 -- start start 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.689+0000 7f169404c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba10f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f169404c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 0x55a7ba118540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f169404c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f169404c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7ba0741e0 con 0x55a7ba10fea0 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f169404c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a7ba011390 con 0x55a7ba118a80 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f169404c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a7ba0701a0 con 0x55a7ba10f580 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 0x55a7ba118540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 0x55a7ba118540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51336/0 (socket says 172.21.15.5:51336) 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 -- 172.21.15.5:0/3468799045 learned_addr learned my addr 172.21.15.5:0/3468799045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f1673fff640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16740a3d50 0x7f16740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba10f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168b7fe640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16740a3d50 0x7f16740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f1673fff640 1 -- 172.21.15.5:0/3003036150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f16840d0880 con 0x7f168c1035b0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 -- 172.21.15.5:0/3468799045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 msgr2=0x55a7ba10f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba10f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 -- 172.21.15.5:0/3468799045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 msgr2=0x55a7ba11ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 -- 172.21.15.5:0/3468799045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b9e507d0 con 0x55a7ba10fea0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168b7fe640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16740a3d50 0x7f16740a6210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f168409c4c0 tx=0x7f1684087040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f168affd640 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 0x55a7ba118540 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f167c00b760 tx=0x7f167c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bb3c62c409e9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.690+0000 7f16717fa640 1 -- 172.21.15.5:0/3468799045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c003d50 con 0x55a7ba10fea0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f16717fa640 1 -- 172.21.15.5:0/3468799045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c015070 con 0x55a7ba10fea0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f16717fa640 1 -- 172.21.15.5:0/3468799045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c004cf0 con 0x55a7ba10fea0 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 -- 172.21.15.5:0/3468799045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 msgr2=0x55a7ba118540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 0x55a7ba118540 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f167c00b760 tx=0x7f167c00bc20 comp rx=0 tx=0).stop 2024-04-22T13:36:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 -- 172.21.15.5:0/3468799045 shutdown_connections 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10fea0 0x55a7ba118540 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 --2- 172.21.15.5:0/3468799045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 -- 172.21.15.5:0/3468799045 >> 172.21.15.5:0/3468799045 conn(0x55a7ba109cb0 msgr2=0x55a7ba10a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 -- 172.21.15.5:0/3468799045 shutdown_connections 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 -- 172.21.15.5:0/3468799045 wait complete. 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.691+0000 7f169404c740 1 -- start start 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 Processor -- start 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba115af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba10fea0 0x55a7ba116030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 0x55a7ba116570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7ba11b870 con 0x55a7ba118a80 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55a7ba02d2f0 con 0x55a7ba10fea0 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a7ba11b9f0 con 0x55a7ba10f580 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f169404c740 10 client.0 _ll_register_callbacks cb 0x55a7ba03a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 0x55a7ba116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 0x55a7ba116570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51342/0 (socket says 172.21.15.5:51342) 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 -- 172.21.15.5:0/4256754952 learned_addr learned my addr 172.21.15.5:0/4256754952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168affd640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba10fea0 0x55a7ba116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 -- 172.21.15.5:0/4256754952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 msgr2=0x55a7ba115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 -- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba10fea0 msgr2=0x55a7ba116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba10fea0 0x55a7ba116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 -- 172.21.15.5:0/4256754952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b9d863f0 con 0x55a7ba118a80 2024-04-22T13:36:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f168a7fc640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 0x55a7ba116570 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f168002c780 tx=0x7f168002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8e61bf3b301189 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.692+0000 7f166eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f166eff5640 1 -- 172.21.15.5:0/4256754952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680005020 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f166eff5640 1 -- 172.21.15.5:0/4256754952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f16800051c0 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f167c015070 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f166eff5640 1 -- 172.21.15.5:0/4256754952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1680055050 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 10 client.18918 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7b9d460f0 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f166eff5640 1 -- 172.21.15.5:0/4256754952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c015070 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f166eff5640 1 -- 172.21.15.5:0/4256754952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16800046c0 con 0x55a7ba118a80 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 10 client.18918 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 10 client.18918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 20 client.18918 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba128ac0 0x55a7ba12aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 4 client.18918 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.693+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7ba081c00 con 0x55a7ba128ac0 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.694+0000 7f168affd640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba128ac0 0x55a7ba12aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.694+0000 7f168affd640 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba128ac0 0x55a7ba12aef0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9893132b5d8d1c8 server_cookie=b601dd1f10b38fd8 in_seq=0 out_seq=0 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.694+0000 7f166eff5640 10 client.18918 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f166eff5640 1 -- 172.21.15.5:0/4256754952 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7ba081c00 con 0x55a7ba128ac0 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f166eff5640 10 client.18918 handle_command_reply: tid=0 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 client.18918 shutdown 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 msgr2=0x55a7ba116570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 0x55a7ba116570 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f168002c780 tx=0x7f168002bc30 comp rx=0 tx=0).stop 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 shutdown_connections 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba128ac0 0x55a7ba12aef0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba118a80 0x55a7ba116570 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f166eff5640 0 client.18918 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba10fea0 0x55a7ba116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 --2- 172.21.15.5:0/4256754952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10f580 0x55a7ba115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 >> 172.21.15.5:0/4256754952 conn(0x55a7ba109cb0 msgr2=0x55a7ba10ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 shutdown_connections 2024-04-22T13:36:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 1 -- 172.21.15.5:0/4256754952 wait complete. 2024-04-22T13:36:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.695+0000 7f169404c740 20 client.18918 trim_cache size 0 max 16384 2024-04-22T13:36:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.712+0000 7f169404c740 1 Processor -- start 2024-04-22T13:36:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.712+0000 7f169404c740 1 -- start start 2024-04-22T13:36:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f169404c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba118220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f169404c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba124730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f169404c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba114c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f169404c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1afe20 con 0x55a7ba10f580 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f169404c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1b1960 con 0x55a7ba118a80 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f169404c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1b04f0 con 0x55a7ba10fea0 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba118220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba118220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51344/0 (socket says 172.21.15.5:51344) 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 -- 172.21.15.5:0/3601723864 learned_addr learned my addr 172.21.15.5:0/3601723864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168affd640 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba124730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 -- 172.21.15.5:0/3601723864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 msgr2=0x55a7ba124730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba124730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 -- 172.21.15.5:0/3601723864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 msgr2=0x55a7ba114c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba114c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 -- 172.21.15.5:0/3601723864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ba1097e0 con 0x55a7ba10f580 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168affd640 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba124730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f168a7fc640 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba118220 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f16800096c0 tx=0x7f1680004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d034331ce55caee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f1670ff9640 1 -- 172.21.15.5:0/3601723864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168002c2b0 con 0x55a7ba10f580 2024-04-22T13:36:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f1670ff9640 1 -- 172.21.15.5:0/3601723864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1680003cb0 con 0x55a7ba10f580 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.713+0000 7f1670ff9640 1 -- 172.21.15.5:0/3601723864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680005080 con 0x55a7ba10f580 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 -- 172.21.15.5:0/3601723864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 msgr2=0x55a7ba118220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba118220 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f16800096c0 tx=0x7f1680004390 comp rx=0 tx=0).stop 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 -- 172.21.15.5:0/3601723864 shutdown_connections 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba124730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 --2- 172.21.15.5:0/3601723864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba118220 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 -- 172.21.15.5:0/3601723864 >> 172.21.15.5:0/3601723864 conn(0x55a7ba109cb0 msgr2=0x55a7ba1084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 -- 172.21.15.5:0/3601723864 shutdown_connections 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 -- 172.21.15.5:0/3601723864 wait complete. 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 -- start start 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 Processor -- start 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.714+0000 7f169404c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba11c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba11cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1b1290 con 0x55a7ba10f580 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1b0bc0 con 0x55a7ba118a80 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ade20 con 0x55a7ba10fea0 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 10 client.0 _ll_register_callbacks cb 0x7f1680005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba11c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba11c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51346/0 (socket says 172.21.15.5:51346) 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 -- 172.21.15.5:0/866727614 learned_addr learned my addr 172.21.15.5:0/866727614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168affd640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba11cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 -- 172.21.15.5:0/866727614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 msgr2=0x55a7ba11cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba11cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 -- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 msgr2=0x55a7ba11d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 -- 172.21.15.5:0/866727614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ba18e030 con 0x55a7ba10f580 2024-04-22T13:36:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168affd640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba11cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f168a7fc640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba11c690 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1680009850 tx=0x7f16800041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6de951b62d362d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f166eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f166eff5640 1 -- 172.21.15.5:0/866727614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680060070 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f166eff5640 1 -- 172.21.15.5:0/866727614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f1680003800 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1680003cb0 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f166eff5640 1 -- 172.21.15.5:0/866727614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f168002c2b0 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 10 client.18924 fetch_fsmap learned FSMap version 28 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.715+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7ba18f280 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f166eff5640 1 -- 172.21.15.5:0/866727614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1680003cb0 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f166eff5640 1 -- 172.21.15.5:0/866727614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1680043450 con 0x55a7ba10f580 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f169404c740 10 client.18924 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f169404c740 10 client.18924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f169404c740 20 client.18924 populate_metadata read hostname 'smithi005' 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f169404c740 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba18f7e0 0x55a7ba191bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f169404c740 4 client.18924 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:36:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7ba07ba70 con 0x55a7ba18f7e0 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f168affd640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba18f7e0 0x55a7ba191bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f168affd640 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba18f7e0 0x55a7ba191bd0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1da5addc5a0f3dc server_cookie=c5b388f104fc43f2 in_seq=0 out_seq=0 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.716+0000 7f166eff5640 10 client.18924 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f166eff5640 1 -- 172.21.15.5:0/866727614 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a7ba07ba70 con 0x55a7ba18f7e0 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f166eff5640 10 client.18924 handle_command_reply: tid=0 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 client.18924 shutdown 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 msgr2=0x55a7ba11c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba11c690 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1680009850 tx=0x7f16800041b0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 shutdown_connections 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55a7ba18f7e0 0x55a7ba191bd0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7ba118a80 0x55a7ba11d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a7ba10fea0 0x55a7ba11cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 --2- 172.21.15.5:0/866727614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7ba10f580 0x55a7ba11c690 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f166eff5640 0 client.18924 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:36:33.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 >> 172.21.15.5:0/866727614 conn(0x55a7ba109cb0 msgr2=0x55a7ba116340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 shutdown_connections 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 1 -- 172.21.15.5:0/866727614 wait complete. 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f169404c740 20 client.18924 trim_cache size 0 max 16384 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "43ee8c9f-fb56-42ac-a670-455ba16f1dd9": { 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "43ee8c9f-fb56-42ac-a670-455ba16f1dd9", 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:36:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:36:33.720 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:36:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16740a3d50 msgr2=0x7f16740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16740a3d50 0x7f16740a6210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f168409c4c0 tx=0x7f1684087040 comp rx=0 tx=0).stop 2024-04-22T13:36:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 msgr2=0x7f168c1a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:36:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 0x7f168c1a6920 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f168400cad0 tx=0x7f168400cf90 comp rx=0 tx=0).stop 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 shutdown_connections 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f168c1035b0 0x7f168c1a6920 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f16740a3d50 0x7f16740a6210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f168c106170 0x7f168c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 --2- 172.21.15.5:0/3003036150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f168c1010a0 0x7f168c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.717+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 >> 172.21.15.5:0/3003036150 conn(0x7f168c077f40 msgr2=0x7f168c06a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.718+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 shutdown_connections 2024-04-22T13:36:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:36:33.718+0000 7f1691462640 1 -- 172.21.15.5:0/3003036150 wait complete. 2024-04-22T13:36:33.724 INFO:tasks.cephfs.filesystem:scrub status for tag:43ee8c9f-fb56-42ac-a670-455ba16f1dd9 - {'path': '/', 'tag': '43ee8c9f-fb56-42ac-a670-455ba16f1dd9', 'options': 'recursive,force'} 2024-04-22T13:36:34.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:34 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:36:34.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:34 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:34.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:34 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:36:34.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:34 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:34.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:34 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:36:34.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:34 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:34.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:34 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:36:34.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:34 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:34.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:34 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:36:34.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:34 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:34.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:34 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:36:34.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:34 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:36:35.422 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:35 smithi049 ceph-mon[31379]: pgmap v292: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:35.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:35 smithi107 ceph-mon[31859]: pgmap v292: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:35.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:35 smithi005 ceph-mon[28006]: pgmap v292: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:36.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:36 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:36:36.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:36 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:36:36.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:36 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:36:37.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:37 smithi049 ceph-mon[31379]: pgmap v293: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:37.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:37 smithi107 ceph-mon[31859]: pgmap v293: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:37 smithi005 ceph-mon[28006]: pgmap v293: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:39.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:39 smithi049 ceph-mon[31379]: pgmap v294: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:39.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:39 smithi107 ceph-mon[31859]: pgmap v294: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:39 smithi005 ceph-mon[28006]: pgmap v294: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:41.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:41 smithi049 ceph-mon[31379]: pgmap v295: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:41.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:41 smithi107 ceph-mon[31859]: pgmap v295: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:41.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:41 smithi005 ceph-mon[28006]: pgmap v295: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:43.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:43 smithi049 ceph-mon[31379]: pgmap v296: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:43.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:43 smithi107 ceph-mon[31859]: pgmap v296: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:43 smithi005 ceph-mon[28006]: pgmap v296: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:45.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:45 smithi049 ceph-mon[31379]: pgmap v297: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:45.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:45 smithi107 ceph-mon[31859]: pgmap v297: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:45.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:45 smithi005 ceph-mon[28006]: pgmap v297: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:36:47.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:47 smithi049 ceph-mon[31379]: pgmap v298: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:47.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:47 smithi107 ceph-mon[31859]: pgmap v298: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:47.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:47 smithi005 ceph-mon[28006]: pgmap v298: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:36:49.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:49 smithi049 ceph-mon[31379]: pgmap v299: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:49.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:49 smithi107 ceph-mon[31859]: pgmap v299: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:49.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:49 smithi005 ceph-mon[28006]: pgmap v299: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:51.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:51 smithi049 ceph-mon[31379]: pgmap v300: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:51.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:51 smithi107 ceph-mon[31859]: pgmap v300: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:51.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:51 smithi005 ceph-mon[28006]: pgmap v300: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:53.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:53 smithi049 ceph-mon[31379]: pgmap v301: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:53.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:53 smithi107 ceph-mon[31859]: pgmap v301: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:53.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:53 smithi005 ceph-mon[28006]: pgmap v301: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:55.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:55 smithi049 ceph-mon[31379]: pgmap v302: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:55.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:55 smithi107 ceph-mon[31859]: pgmap v302: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:55.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:55 smithi005 ceph-mon[28006]: pgmap v302: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:57.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:57 smithi049 ceph-mon[31379]: pgmap v303: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:57.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:57 smithi107 ceph-mon[31859]: pgmap v303: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:57 smithi005 ceph-mon[28006]: pgmap v303: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:36:59.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:36:59 smithi049 ceph-mon[31379]: pgmap v304: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:59.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:36:59 smithi107 ceph-mon[31859]: pgmap v304: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:36:59.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:36:59 smithi005 ceph-mon[28006]: pgmap v304: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:01.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:01 smithi049 ceph-mon[31379]: pgmap v305: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:01.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:01 smithi107 ceph-mon[31859]: pgmap v305: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:01 smithi005 ceph-mon[28006]: pgmap v305: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:03.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:03 smithi049 ceph-mon[31379]: pgmap v306: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:37:03.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:03 smithi107 ceph-mon[31859]: pgmap v306: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:37:03.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:03 smithi005 ceph-mon[28006]: pgmap v306: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:37:03.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:37:03.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:37:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 -- 172.21.15.5:0/2199034448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e81010c0 msgr2=0x7f37e81014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 --2- 172.21.15.5:0/2199034448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e81010c0 0x7f37e81014a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009900 tx=0x7f37d802bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 -- 172.21.15.5:0/2199034448 shutdown_connections 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 --2- 172.21.15.5:0/2199034448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37e8106190 0x7f37e810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 --2- 172.21.15.5:0/2199034448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37e8101a70 0x7f37e8105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 --2- 172.21.15.5:0/2199034448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e81010c0 0x7f37e81014a0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 -- 172.21.15.5:0/2199034448 >> 172.21.15.5:0/2199034448 conn(0x7f37e8077fa0 msgr2=0x7f37e80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 -- 172.21.15.5:0/2199034448 shutdown_connections 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 -- 172.21.15.5:0/2199034448 wait complete. 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 Processor -- start 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 -- start start 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37e81010c0 0x7f37e81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.848+0000 7f37eeac1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37e8106190 0x7f37e8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37eeac1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 0x7f37e81a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37eeac1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81110f0 con 0x7f37e8199fa0 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37eeac1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8110f70 con 0x7f37e81010c0 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37eeac1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8111270 con 0x7f37e8106190 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 0x7f37e81a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 0x7f37e81a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41868/0 (socket says 172.21.15.5:41868) 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 -- 172.21.15.5:0/3782835401 learned_addr learned my addr 172.21.15.5:0/3782835401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ec85d640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37e81010c0 0x7f37e81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37e7fff640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37e8106190 0x7f37e8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 -- 172.21.15.5:0/3782835401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37e8106190 msgr2=0x7f37e8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37e8106190 0x7f37e8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 -- 172.21.15.5:0/3782835401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37e81010c0 msgr2=0x7f37e81994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37e81010c0 0x7f37e81994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 -- 172.21.15.5:0/3782835401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81a4d50 con 0x7f37e8199fa0 2024-04-22T13:37:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37ed05e640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 0x7f37e81a4550 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f37e000cce0 tx=0x7f37e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37e5ffb640 1 -- 172.21.15.5:0/3782835401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e000f450 con 0x7f37e8199fa0 2024-04-22T13:37:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81a5040 con 0x7f37e8199fa0 2024-04-22T13:37:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37e5ffb640 1 -- 172.21.15.5:0/3782835401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e000faa0 con 0x7f37e8199fa0 2024-04-22T13:37:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.849+0000 7f37e5ffb640 1 -- 172.21.15.5:0/3782835401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0018b70 con 0x7f37e8199fa0 2024-04-22T13:37:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.850+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a5560 con 0x7f37e8199fa0 2024-04-22T13:37:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.852+0000 7f37e5ffb640 1 -- 172.21.15.5:0/3782835401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f37e0018d10 con 0x7f37e8199fa0 2024-04-22T13:37:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 Processor -- start 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 -- start start 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14157960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad14160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37e5ffb640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37d00a3d20 0x7f37d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad14162e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad140bc1e0 con 0x55ad14157ea0 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad14059390 con 0x55ad14160a80 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37f178e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ad140b81a0 con 0x55ad14157580 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad14160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37ec85d640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37d00a3d20 0x7f37d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad14160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41884/0 (socket says 172.21.15.5:41884) 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37e7fff640 1 -- 172.21.15.5:0/1143938034 learned_addr learned my addr 172.21.15.5:0/1143938034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37e5ffb640 1 -- 172.21.15.5:0/3782835401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f37e00d0bc0 con 0x7f37e8199fa0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37ed05e640 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad14162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.853+0000 7f37e7fff640 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37e7fff640 1 -- 172.21.15.5:0/1143938034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 msgr2=0x55ad14157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37e7fff640 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37e7fff640 1 -- 172.21.15.5:0/1143938034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 msgr2=0x55ad14162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37e7fff640 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad14162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37e7fff640 1 -- 172.21.15.5:0/1143938034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad13e987d0 con 0x55ad14157ea0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37ec85d640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37d00a3d20 0x7f37d00a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009230 tx=0x7f37d8005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37e7fff640 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad14160540 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005c30 tx=0x7f37dc00a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76df0da480b3e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37ceffd640 1 -- 172.21.15.5:0/1143938034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc004220 con 0x55ad14157ea0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37ceffd640 1 -- 172.21.15.5:0/1143938034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37dc0043c0 con 0x55ad14157ea0 2024-04-22T13:37:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.854+0000 7f37ceffd640 1 -- 172.21.15.5:0/1143938034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00f690 con 0x55ad14157ea0 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 -- 172.21.15.5:0/1143938034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 msgr2=0x55ad14160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad14160540 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005c30 tx=0x7f37dc00a750 comp rx=0 tx=0).stop 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 -- 172.21.15.5:0/1143938034 shutdown_connections 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad14160540 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad14162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 --2- 172.21.15.5:0/1143938034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 -- 172.21.15.5:0/1143938034 >> 172.21.15.5:0/1143938034 conn(0x55ad14151cb0 msgr2=0x55ad14152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 -- 172.21.15.5:0/1143938034 shutdown_connections 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 -- 172.21.15.5:0/1143938034 wait complete. 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 -- start start 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.855+0000 7f37f178e740 1 Processor -- start 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad1415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad1415e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad1415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ad14163870 con 0x55ad14157ea0 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55ad140752f0 con 0x55ad14160a80 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ad141639f0 con 0x55ad14157580 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37f178e740 10 client.0 _ll_register_callbacks cb 0x55ad14082580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad1415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad1415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41894/0 (socket says 172.21.15.5:41894) 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 -- 172.21.15.5:0/2090996813 learned_addr learned my addr 172.21.15.5:0/2090996813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37ed05e640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad1415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37ec85d640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad1415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 -- 172.21.15.5:0/2090996813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 msgr2=0x55ad1415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad1415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 -- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 msgr2=0x55ad1415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad1415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 -- 172.21.15.5:0/2090996813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad13dce3f0 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37ed05e640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad1415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37e7fff640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad1415e050 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005c00 tx=0x7f37dc004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88703bbddf05d66b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.856+0000 7f37c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37c7fff640 1 -- 172.21.15.5:0/2090996813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc02a060 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37c7fff640 1 -- 172.21.15.5:0/2090996813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f37dc0052b0 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37dc0043c0 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37c7fff640 1 -- 172.21.15.5:0/2090996813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f37dc0054b0 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 10 client.18936 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad13d8e0f0 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37c7fff640 1 -- 172.21.15.5:0/2090996813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37dc0043c0 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37c7fff640 1 -- 172.21.15.5:0/2090996813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f37dc01b030 con 0x55ad14157ea0 2024-04-22T13:37:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 10 client.18936 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 10 client.18936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 20 client.18936 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad14170a70 0x55ad14172e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 4 client.18936 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.857+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad140c9c00 con 0x55ad14170a70 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.858+0000 7f37ec85d640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad14170a70 0x55ad14172e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.858+0000 7f37c7fff640 10 client.18936 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.858+0000 7f37ec85d640 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad14170a70 0x55ad14172e60 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f584e6d9fa9016fe server_cookie=3270b8e443331697 in_seq=0 out_seq=0 2024-04-22T13:37:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37c7fff640 1 -- 172.21.15.5:0/2090996813 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad140c9c00 con 0x55ad14170a70 2024-04-22T13:37:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37c7fff640 10 client.18936 handle_command_reply: tid=0 2024-04-22T13:37:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 client.18936 shutdown 2024-04-22T13:37:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 msgr2=0x55ad1415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad1415e050 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005c00 tx=0x7f37dc004f10 comp rx=0 tx=0).stop 2024-04-22T13:37:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 shutdown_connections 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad14170a70 0x55ad14172e60 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14157ea0 0x55ad1415e050 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14160a80 0x55ad1415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37c7fff640 0 client.18936 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 --2- 172.21.15.5:0/2090996813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad1415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 >> 172.21.15.5:0/2090996813 conn(0x55ad14151cb0 msgr2=0x55ad14153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 shutdown_connections 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 1 -- 172.21.15.5:0/2090996813 wait complete. 2024-04-22T13:37:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.859+0000 7f37f178e740 20 client.18936 trim_cache size 0 max 16384 2024-04-22T13:37:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 Processor -- start 2024-04-22T13:37:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 -- start start 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14152090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1416c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81adab0 con 0x55ad14160a80 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81af5f0 con 0x55ad14157ea0 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37f178e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81ae180 con 0x55ad14157580 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41900/0 (socket says 172.21.15.5:41900) 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 -- 172.21.15.5:0/2232522761 learned_addr learned my addr 172.21.15.5:0/2232522761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 -- 172.21.15.5:0/2232522761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 msgr2=0x55ad14152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 -- 172.21.15.5:0/2232522761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 msgr2=0x55ad1416c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1416c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 -- 172.21.15.5:0/2232522761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad14157cd0 con 0x55ad14160a80 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ed05e640 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415e3b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f37e009ca90 tx=0x7f37e000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5897a47234fd8212 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.877+0000 7f37ce7fc640 1 -- 172.21.15.5:0/2232522761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e009d490 con 0x55ad14160a80 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37ce7fc640 1 -- 172.21.15.5:0/2232522761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e009de30 con 0x55ad14160a80 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37ce7fc640 1 -- 172.21.15.5:0/2232522761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00ce530 con 0x55ad14160a80 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 -- 172.21.15.5:0/2232522761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 msgr2=0x55ad1415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415e3b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f37e009ca90 tx=0x7f37e000ff50 comp rx=0 tx=0).stop 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 -- 172.21.15.5:0/2232522761 shutdown_connections 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415e3b0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1416c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 --2- 172.21.15.5:0/2232522761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 -- 172.21.15.5:0/2232522761 >> 172.21.15.5:0/2232522761 conn(0x55ad14151cb0 msgr2=0x55ad141504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 -- 172.21.15.5:0/2232522761 shutdown_connections 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 -- 172.21.15.5:0/2232522761 wait complete. 2024-04-22T13:37:03.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.878+0000 7f37f178e740 1 -- start start 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 Processor -- start 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1415a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aef20 con 0x55ad14160a80 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81ae850 con 0x55ad14157ea0 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81abab0 con 0x55ad14157580 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37f178e740 10 client.0 _ll_register_callbacks cb 0x7f37dc00af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41912/0 (socket says 172.21.15.5:41912) 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 -- 172.21.15.5:0/3524353752 learned_addr learned my addr 172.21.15.5:0/3524353752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ec85d640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37e7fff640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 -- 172.21.15.5:0/3524353752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 msgr2=0x55ad14168ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14168ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 -- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 msgr2=0x55ad1415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1415a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37ed05e640 1 -- 172.21.15.5:0/3524353752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1415af30 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.879+0000 7f37e7fff640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37ed05e640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415a8c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f37e00994d0 tx=0x7f37e00ce760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76982cc8a6e98eff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37c7fff640 1 -- 172.21.15.5:0/3524353752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00a5770 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37c7fff640 1 -- 172.21.15.5:0/3524353752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f37e00a52e0 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e009de30 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37c7fff640 1 -- 172.21.15.5:0/3524353752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f37e00a0eb0 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 10 client.18942 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad141d6ce0 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37c7fff640 1 -- 172.21.15.5:0/3524353752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e009de30 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37c7fff640 1 -- 172.21.15.5:0/3524353752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f37e0098850 con 0x55ad14160a80 2024-04-22T13:37:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 10 client.18942 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 20 client.18942 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad141d7450 0x55ad141d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 4 client.18942 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.880+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad140c3a70 con 0x55ad141d7450 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37ec85d640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad141d7450 0x55ad141d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37c7fff640 10 client.18942 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37ec85d640 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad141d7450 0x55ad141d9840 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6de9fda082ab34 server_cookie=a636ecb72704c995 in_seq=0 out_seq=0 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37c7fff640 1 -- 172.21.15.5:0/3524353752 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ad140c3a70 con 0x55ad141d7450 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37c7fff640 10 client.18942 handle_command_reply: tid=0 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 client.18942 shutdown 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 msgr2=0x55ad1415a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415a8c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f37e00994d0 tx=0x7f37e00ce760 comp rx=0 tx=0).stop 2024-04-22T13:37:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 shutdown_connections 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55ad141d7450 0x55ad141d9840 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad14160a80 0x55ad1415a8c0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad14157ea0 0x55ad1415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37c7fff640 0 client.18942 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 --2- 172.21.15.5:0/3524353752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad14157580 0x55ad14168ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 >> 172.21.15.5:0/3524353752 conn(0x55ad14151cb0 msgr2=0x55ad14150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 shutdown_connections 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.881+0000 7f37f178e740 1 -- 172.21.15.5:0/3524353752 wait complete. 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37f178e740 20 client.18942 trim_cache size 0 max 16384 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:37:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:37:03.884 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37d00a3d20 msgr2=0x7f37d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37d00a3d20 0x7f37d00a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009230 tx=0x7f37d8005fd0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 msgr2=0x7f37e81a4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 0x7f37e81a4550 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f37e000cce0 tx=0x7f37e000a620 comp rx=0 tx=0).stop 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 shutdown_connections 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8199fa0 0x7f37e81a4550 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f37d00a3d20 0x7f37d00a61e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37e8106190 0x7f37e8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 --2- 172.21.15.5:0/3782835401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37e81010c0 0x7f37e81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 >> 172.21.15.5:0/3782835401 conn(0x7f37e8077fa0 msgr2=0x7f37e80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 shutdown_connections 2024-04-22T13:37:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.882+0000 7f37eeac1640 1 -- 172.21.15.5:0/3782835401 wait complete. 2024-04-22T13:37:03.889 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:37:03.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:37:03.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 -- 172.21.15.5:0/751970641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c106140 msgr2=0x7f890c10dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 --2- 172.21.15.5:0/751970641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c106140 0x7f890c10dd10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8904009ca0 tx=0x7f890402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 -- 172.21.15.5:0/751970641 shutdown_connections 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 --2- 172.21.15.5:0/751970641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c106140 0x7f890c10dd10 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 --2- 172.21.15.5:0/751970641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c101a20 0x7f890c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 --2- 172.21.15.5:0/751970641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f890c101070 0x7f890c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 -- 172.21.15.5:0/751970641 >> 172.21.15.5:0/751970641 conn(0x7f890c0771d0 msgr2=0x7f890c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 -- 172.21.15.5:0/751970641 shutdown_connections 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 -- 172.21.15.5:0/751970641 wait complete. 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 Processor -- start 2024-04-22T13:37:03.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.996+0000 7f8911ec7640 1 -- start start 2024-04-22T13:37:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8911ec7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f890c101070 0x7f890c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8911ec7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c106140 0x7f890c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8911ec7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 0x7f890c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8911ec7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1110b0 con 0x7f890c19cf50 2024-04-22T13:37:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8911ec7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f890c110f30 con 0x7f890c106140 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8911ec7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f890c111230 con 0x7f890c101070 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 0x7f890c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 0x7f890c19d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41932/0 (socket says 172.21.15.5:41932) 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 -- 172.21.15.5:0/396797877 learned_addr learned my addr 172.21.15.5:0/396797877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890b7fe640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f890c101070 0x7f890c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8902ffd640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c106140 0x7f890c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 -- 172.21.15.5:0/396797877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f890c101070 msgr2=0x7f890c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f890c101070 0x7f890c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 -- 172.21.15.5:0/396797877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c106140 msgr2=0x7f890c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c106140 0x7f890c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 -- 172.21.15.5:0/396797877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c19da40 con 0x7f890c19cf50 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f8902ffd640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c106140 0x7f890c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f890bfff640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 0x7f890c19d400 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8904005e30 tx=0x7f89040346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.997+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904041020 con 0x7f890c19cf50 2024-04-22T13:37:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.998+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c19dcd0 con 0x7f890c19cf50 2024-04-22T13:37:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.998+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8904032970 con 0x7f890c19cf50 2024-04-22T13:37:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.998+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904039040 con 0x7f890c19cf50 2024-04-22T13:37:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.998+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a5490 con 0x7f890c19cf50 2024-04-22T13:37:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:03.999+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c068e00 con 0x7f890c19cf50 2024-04-22T13:37:03.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.000+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8904044020 con 0x7f890c19cf50 2024-04-22T13:37:03.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.000+0000 7f89097fa640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88f80a3d20 0x7f88f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:03.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.000+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f89040f2380 con 0x7f890c19cf50 2024-04-22T13:37:03.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.000+0000 7f890b7fe640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88f80a3d20 0x7f88f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:03.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.001+0000 7f890b7fe640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88f80a3d20 0x7f88f80a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009bd0 tx=0x7f88fc005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.005+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f890404b020 con 0x7f890c19cf50 2024-04-22T13:37:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.232+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890c101c00 con 0x7f890c19cf50 2024-04-22T13:37:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.232+0000 7f89097fa640 1 -- 172.21.15.5:0/396797877 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f89040beb30 con 0x7f890c19cf50 2024-04-22T13:37:04.228 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:37:04.228 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:37:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:37:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88f80a3d20 msgr2=0x7f88f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88f80a3d20 0x7f88f80a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009bd0 tx=0x7f88fc005f90 comp rx=0 tx=0).stop 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 msgr2=0x7f890c19d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 0x7f890c19d400 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8904005e30 tx=0x7f89040346d0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 shutdown_connections 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f890c19cf50 0x7f890c19d400 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f88f80a3d20 0x7f88f80a61e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c106140 0x7f890c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 --2- 172.21.15.5:0/396797877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f890c101070 0x7f890c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 >> 172.21.15.5:0/396797877 conn(0x7f890c0771d0 msgr2=0x7f890c107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 shutdown_connections 2024-04-22T13:37:04.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.235+0000 7f8911ec7640 1 -- 172.21.15.5:0/396797877 wait complete. 2024-04-22T13:37:04.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:37:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 -- 172.21.15.5:0/3937929116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c103400 msgr2=0x7fd26c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 --2- 172.21.15.5:0/3937929116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c103400 0x7fd26c10de00 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd264009cd0 tx=0x7fd26402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 -- 172.21.15.5:0/3937929116 shutdown_connections 2024-04-22T13:37:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 --2- 172.21.15.5:0/3937929116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c103400 0x7fd26c10de00 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 --2- 172.21.15.5:0/3937929116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd26c102a60 0x7fd26c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 --2- 172.21.15.5:0/3937929116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd26c101180 0x7fd26c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 -- 172.21.15.5:0/3937929116 >> 172.21.15.5:0/3937929116 conn(0x7fd26c077f70 msgr2=0x7fd26c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 -- 172.21.15.5:0/3937929116 shutdown_connections 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.350+0000 7fd271315640 1 -- 172.21.15.5:0/3937929116 wait complete. 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 Processor -- start 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 -- start start 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 0x7fd26c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd26c102a60 0x7fd26c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd26c103400 0x7fd26c1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1111a0 con 0x7fd26c101180 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c111020 con 0x7fd26c103400 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd271315640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c111320 con 0x7fd26c102a60 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd26affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 0x7fd26c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd26affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 0x7fd26c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41962/0 (socket says 172.21.15.5:41962) 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd26affd640 1 -- 172.21.15.5:0/2977909611 learned_addr learned my addr 172.21.15.5:0/2977909611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.351+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd26c103400 0x7fd26c1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd263fff640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd26c102a60 0x7fd26c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd26affd640 1 -- 172.21.15.5:0/2977909611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd26c102a60 msgr2=0x7fd26c199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd26affd640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd26c102a60 0x7fd26c199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd26affd640 1 -- 172.21.15.5:0/2977909611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd26c103400 msgr2=0x7fd26c1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd26affd640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd26c103400 0x7fd26c1a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd26affd640 1 -- 172.21.15.5:0/2977909611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c1a4c20 con 0x7fd26c101180 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd26affd640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 0x7fd26c199660 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d770 tx=0x7fd25c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd268ff9640 1 -- 172.21.15.5:0/2977909611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c017070 con 0x7fd26c101180 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c1a4df0 con 0x7fd26c101180 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd268ff9640 1 -- 172.21.15.5:0/2977909611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25c0054d0 con 0x7fd26c101180 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd268ff9640 1 -- 172.21.15.5:0/2977909611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c004c30 con 0x7fd26c101180 2024-04-22T13:37:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.352+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c1a52b0 con 0x7fd26c101180 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.354+0000 7fd268ff9640 1 -- 172.21.15.5:0/2977909611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd25c004e10 con 0x7fd26c101180 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 Processor -- start 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 -- start start 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f44d0 0x56077e1f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 0x56077e1fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1fda00 0x56077e1ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56077e1591f0 con 0x56077e1f4e80 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56077e0f64b0 con 0x56077e1fda00 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd273fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56077e1551b0 con 0x56077e1f44d0 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd268ff9640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2580a3d20 0x7fd2580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd26affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 0x56077e1fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd26affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 0x56077e1fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41972/0 (socket says 172.21.15.5:41972) 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd26affd640 1 -- 172.21.15.5:0/3904717119 learned_addr learned my addr 172.21.15.5:0/3904717119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f44d0 0x56077e1f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2580a3d20 0x7fd2580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd263fff640 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1fda00 0x56077e1ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.355+0000 7fd268ff9640 1 -- 172.21.15.5:0/2977909611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd25c0d02a0 con 0x7fd26c101180 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26affd640 1 -- 172.21.15.5:0/3904717119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f44d0 msgr2=0x56077e1f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26affd640 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f44d0 0x56077e1f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26affd640 1 -- 172.21.15.5:0/3904717119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1fda00 msgr2=0x56077e1ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26affd640 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1fda00 0x56077e1ffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26affd640 1 -- 172.21.15.5:0/3904717119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56077e0e81b0 con 0x56077e1f4e80 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2580a3d20 0x7fd2580a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd264005cc0 tx=0x7fd264005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd26affd640 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 0x56077e1fd430 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d740 tx=0x7fd25c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d0517b31ed2d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd2617fa640 1 -- 172.21.15.5:0/3904717119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c09c240 con 0x56077e1f4e80 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd2617fa640 1 -- 172.21.15.5:0/3904717119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25c0a05f0 con 0x56077e1f4e80 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.356+0000 7fd2617fa640 1 -- 172.21.15.5:0/3904717119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0988b0 con 0x56077e1f4e80 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- 172.21.15.5:0/3904717119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 msgr2=0x56077e1fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 0x56077e1fd430 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d740 tx=0x7fd25c009f90 comp rx=0 tx=0).stop 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- 172.21.15.5:0/3904717119 shutdown_connections 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f4e80 0x56077e1fd430 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1fda00 0x56077e1ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- 172.21.15.5:0/3904717119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f44d0 0x56077e1f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- 172.21.15.5:0/3904717119 >> 172.21.15.5:0/3904717119 conn(0x56077e1eeba0 msgr2=0x56077e1eef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- 172.21.15.5:0/3904717119 shutdown_connections 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- 172.21.15.5:0/3904717119 wait complete. 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- start start 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 Processor -- start 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f44d0 0x56077e1fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 0x56077e1fab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56077e200910 con 0x56077e1fda00 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56077e112500 con 0x56077e1f44d0 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.357+0000 7fd273fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56077e200a90 con 0x56077e1f4e80 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd273fe5740 10 client.0 _ll_register_callbacks cb 0x56077e0e0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 0x56077e1fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 0x56077e1fab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41978/0 (socket says 172.21.15.5:41978) 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 -- 172.21.15.5:0/1584014707 learned_addr learned my addr 172.21.15.5:0/1584014707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26affd640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f44d0 0x56077e1fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd263fff640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 -- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 msgr2=0x56077e1fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 -- 172.21.15.5:0/1584014707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f44d0 msgr2=0x56077e1fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f44d0 0x56077e1fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 -- 172.21.15.5:0/1584014707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077de6b3f0 con 0x56077e1fda00 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 0x56077e1fab70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd264009cd0 tx=0x7fd26402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f4e6903506ab server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd2527fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd2527fc640 1 -- 172.21.15.5:0/1584014707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264039040 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd2527fc640 1 -- 172.21.15.5:0/1584014707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd264004240 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd25c0a05f0 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd2527fc640 1 -- 172.21.15.5:0/1584014707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd26405c050 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd273fe5740 10 client.18960 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.358+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56077df357d0 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd2527fc640 1 -- 172.21.15.5:0/1584014707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25c0a05f0 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd2527fc640 1 -- 172.21.15.5:0/1584014707 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd26403f070 con 0x56077e1fda00 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd273fe5740 10 client.18960 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd273fe5740 10 client.18960 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd273fe5740 20 client.18960 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd273fe5740 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e20dc70 0x56077e2100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd273fe5740 4 client.18960 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56077e0f8930 con 0x56077e20dc70 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.359+0000 7fd26affd640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e20dc70 0x56077e2100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd2527fc640 10 client.18960 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd26affd640 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e20dc70 0x56077e2100d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de332c1fea6295a9 server_cookie=5a023020a3442f77 in_seq=0 out_seq=0 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd2527fc640 1 -- 172.21.15.5:0/1584014707 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56077e0f8930 con 0x56077e20dc70 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd2527fc640 10 client.18960 handle_command_reply: tid=0 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 client.18960 shutdown 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 msgr2=0x56077e1fab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 0x56077e1fab70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd264009cd0 tx=0x7fd26402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 shutdown_connections 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e20dc70 0x56077e2100d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1fda00 0x56077e1fab70 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd2527fc640 0 client.18960 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 --2- 172.21.15.5:0/1584014707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f44d0 0x56077e1fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.360+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 >> 172.21.15.5:0/1584014707 conn(0x56077e1eeba0 msgr2=0x56077e1f08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.361+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 shutdown_connections 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.361+0000 7fd273fe5740 1 -- 172.21.15.5:0/1584014707 wait complete. 2024-04-22T13:37:04.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.361+0000 7fd273fe5740 20 client.18960 trim_cache size 0 max 16384 2024-04-22T13:37:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 Processor -- start 2024-04-22T13:37:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 -- start start 2024-04-22T13:37:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1eeba0 0x56077e209800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e1eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1ada40 con 0x56077e1f44d0 2024-04-22T13:37:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1af580 con 0x56077e1eeba0 2024-04-22T13:37:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.374+0000 7fd273fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1ae110 con 0x56077e1f4e80 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e1eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e1eda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41992/0 (socket says 172.21.15.5:41992) 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 -- 172.21.15.5:0/21250099 learned_addr learned my addr 172.21.15.5:0/21250099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd26affd640 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1eeba0 0x56077e209800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 -- 172.21.15.5:0/21250099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 msgr2=0x56077e1fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 -- 172.21.15.5:0/21250099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1eeba0 msgr2=0x56077e209800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1eeba0 0x56077e209800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 -- 172.21.15.5:0/21250099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56077e1f3560 con 0x56077e1f44d0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd26affd640 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1eeba0 0x56077e209800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd263fff640 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e1eda80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c760 tx=0x7fd254004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdddcda1e25b2d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd260ff9640 1 -- 172.21.15.5:0/21250099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254003a70 con 0x56077e1f44d0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd260ff9640 1 -- 172.21.15.5:0/21250099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2540054d0 con 0x56077e1f44d0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.375+0000 7fd260ff9640 1 -- 172.21.15.5:0/21250099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254004d40 con 0x56077e1f44d0 2024-04-22T13:37:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 -- 172.21.15.5:0/21250099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 msgr2=0x56077e1eda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e1eda80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c760 tx=0x7fd254004610 comp rx=0 tx=0).stop 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 -- 172.21.15.5:0/21250099 shutdown_connections 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e1eda80 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1f4e80 0x56077e1fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 --2- 172.21.15.5:0/21250099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1eeba0 0x56077e209800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 -- 172.21.15.5:0/21250099 >> 172.21.15.5:0/21250099 conn(0x56077e1facd0 msgr2=0x56077e1ed370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 -- 172.21.15.5:0/21250099 shutdown_connections 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 -- 172.21.15.5:0/21250099 wait complete. 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 -- start start 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.376+0000 7fd273fe5740 1 Processor -- start 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1eeba0 0x56077e201770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e201cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f4e80 0x56077e2021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1aeeb0 con 0x56077e1f44d0 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1ae7e0 con 0x56077e1f4e80 2024-04-22T13:37:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1aba40 con 0x56077e1eeba0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 10 client.0 _ll_register_callbacks cb 0x7fd2640328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e201cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e201cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41998/0 (socket says 172.21.15.5:41998) 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 -- 172.21.15.5:0/2316495266 learned_addr learned my addr 172.21.15.5:0/2316495266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd26affd640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1eeba0 0x56077e201770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd26b7fe640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f4e80 0x56077e2021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 -- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1eeba0 msgr2=0x56077e201770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1eeba0 0x56077e201770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 -- 172.21.15.5:0/2316495266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f4e80 msgr2=0x56077e2021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f4e80 0x56077e2021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 -- 172.21.15.5:0/2316495266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077e273260 con 0x56077e1f44d0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd263fff640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e201cb0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd254004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ec6b24a92d0746 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd2527fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd2527fc640 1 -- 172.21.15.5:0/2316495266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25401f030 con 0x56077e1f44d0 2024-04-22T13:37:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2540054d0 con 0x56077e1f44d0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd2527fc640 1 -- 172.21.15.5:0/2316495266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd254005230 con 0x56077e1f44d0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.377+0000 7fd2527fc640 1 -- 172.21.15.5:0/2316495266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd254004a80 con 0x56077e1f44d0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 10 client.18966 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56077e274540 con 0x56077e1f44d0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd2527fc640 1 -- 172.21.15.5:0/2316495266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2540054d0 con 0x56077e1f44d0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd2527fc640 1 -- 172.21.15.5:0/2316495266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd25401bce0 con 0x56077e1f44d0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 10 client.18966 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 10 client.18966 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 20 client.18966 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e1fda00 0x56077e276c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 4 client.18966 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56077e166c10 con 0x56077e1fda00 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd26affd640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e1fda00 0x56077e276c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd26affd640 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e1fda00 0x56077e276c10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f11716b98151478e server_cookie=e5c67fdcfed2cab6 in_seq=0 out_seq=0 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.378+0000 7fd2527fc640 10 client.18966 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd2527fc640 1 -- 172.21.15.5:0/2316495266 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56077e166c10 con 0x56077e1fda00 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd2527fc640 10 client.18966 handle_command_reply: tid=0 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 client.18966 shutdown 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 msgr2=0x56077e201cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e201cb0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd254004290 comp rx=0 tx=0).stop 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 shutdown_connections 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x56077e1fda00 0x56077e276c10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56077e1f44d0 0x56077e201cb0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd2527fc640 0 client.18966 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56077e1f4e80 0x56077e2021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 --2- 172.21.15.5:0/2316495266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56077e1eeba0 0x56077e201770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 >> 172.21.15.5:0/2316495266 conn(0x56077e1facd0 msgr2=0x56077e1edc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 shutdown_connections 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 1 -- 172.21.15.5:0/2316495266 wait complete. 2024-04-22T13:37:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd273fe5740 20 client.18966 trim_cache size 0 max 16384 2024-04-22T13:37:04.381 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2580a3d20 msgr2=0x7fd2580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd271315640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2580a3d20 0x7fd2580a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd264005cc0 tx=0x7fd264005c50 comp rx=0 tx=0).stop 2024-04-22T13:37:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 msgr2=0x7fd26c199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.379+0000 7fd271315640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 0x7fd26c199660 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d770 tx=0x7fd25c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 shutdown_connections 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2580a3d20 0x7fd2580a61e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd26c103400 0x7fd26c1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd26c102a60 0x7fd26c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 --2- 172.21.15.5:0/2977909611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd26c101180 0x7fd26c199660 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 >> 172.21.15.5:0/2977909611 conn(0x7fd26c077f70 msgr2=0x7fd26c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 shutdown_connections 2024-04-22T13:37:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.380+0000 7fd271315640 1 -- 172.21.15.5:0/2977909611 wait complete. 2024-04-22T13:37:04.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 -- 172.21.15.5:0/384207123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc106160 msgr2=0x7f10cc10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 --2- 172.21.15.5:0/384207123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc106160 0x7f10cc10dd30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009ca0 tx=0x7f10c402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 -- 172.21.15.5:0/384207123 shutdown_connections 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 --2- 172.21.15.5:0/384207123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc106160 0x7f10cc10dd30 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 --2- 172.21.15.5:0/384207123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10cc101a40 0x7f10cc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 --2- 172.21.15.5:0/384207123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc101090 0x7f10cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 -- 172.21.15.5:0/384207123 >> 172.21.15.5:0/384207123 conn(0x7f10cc078000 msgr2=0x7f10cc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 -- 172.21.15.5:0/384207123 shutdown_connections 2024-04-22T13:37:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.494+0000 7f10d308e640 1 -- 172.21.15.5:0/384207123 wait complete. 2024-04-22T13:37:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 Processor -- start 2024-04-22T13:37:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 -- start start 2024-04-22T13:37:04.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10cc101090 0x7f10cc102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc106160 0x7f10cc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 0x7f10cc1a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1110b0 con 0x7f10cc1035a0 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc110f30 con 0x7f10cc101090 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d308e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc111230 con 0x7f10cc106160 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 0x7f10cc1a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 0x7f10cc1a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42010/0 (socket says 172.21.15.5:42010) 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 -- 172.21.15.5:0/876148721 learned_addr learned my addr 172.21.15.5:0/876148721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10cbfff640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc106160 0x7f10cc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 -- 172.21.15.5:0/876148721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc106160 msgr2=0x7f10cc103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10cc101090 0x7f10cc102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc106160 0x7f10cc103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 -- 172.21.15.5:0/876148721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10cc101090 msgr2=0x7f10cc102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10cc101090 0x7f10cc102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 -- 172.21.15.5:0/876148721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc1a6fc0 con 0x7f10cc1035a0 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10cbfff640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc106160 0x7f10cc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.495+0000 7f10d162b640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 0x7f10cc1a68c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f10c4005e30 tx=0x7f10c4032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.496+0000 7f10c9ffb640 1 -- 172.21.15.5:0/876148721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4041020 con 0x7f10cc1035a0 2024-04-22T13:37:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.496+0000 7f10c9ffb640 1 -- 172.21.15.5:0/876148721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c4032970 con 0x7f10cc1035a0 2024-04-22T13:37:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.496+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc1a7250 con 0x7f10cc1035a0 2024-04-22T13:37:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.496+0000 7f10c9ffb640 1 -- 172.21.15.5:0/876148721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4039040 con 0x7f10cc1035a0 2024-04-22T13:37:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.496+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a7790 con 0x7f10cc1035a0 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10c9ffb640 1 -- 172.21.15.5:0/876148721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f10c4044020 con 0x7f10cc1035a0 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 Processor -- start 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 -- start start 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65ae80 0x560f3b663430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 0x560f3b665e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560f3b5bf1f0 con 0x560f3b663a00 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f3b55c4b0 con 0x560f3b65ae80 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.498+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f3b5bb1b0 con 0x560f3b65a4d0 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 0x560f3b665e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 0x560f3b665e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42024/0 (socket says 172.21.15.5:42024) 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10c9ffb640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f10b80a3dd0 0x7f10b80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 -- 172.21.15.5:0/3810020166 learned_addr learned my addr 172.21.15.5:0/3810020166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65ae80 0x560f3b663430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10d162b640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f10b80a3dd0 0x7f10b80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10c9ffb640 1 -- 172.21.15.5:0/876148721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f10c40f2550 con 0x7f10cc1035a0 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 -- 172.21.15.5:0/3810020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 msgr2=0x560f3b65a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 -- 172.21.15.5:0/3810020166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65ae80 msgr2=0x560f3b663430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65ae80 0x560f3b663430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 -- 172.21.15.5:0/3810020166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3b54e1b0 con 0x560f3b663a00 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 0x560f3b665e50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0149b0 tx=0x7f10bc00c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f15b9359a3587c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10d162b640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f10b80a3dd0 0x7f10b80a6290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f10c40baa00 tx=0x7f10c402c0f0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10c2ffd640 1 -- 172.21.15.5:0/3810020166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc0142d0 con 0x560f3b663a00 2024-04-22T13:37:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10c2ffd640 1 -- 172.21.15.5:0/3810020166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc00fc10 con 0x560f3b663a00 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.499+0000 7f10c2ffd640 1 -- 172.21.15.5:0/3810020166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc012ba0 con 0x560f3b663a00 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3810020166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 msgr2=0x560f3b665e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 0x560f3b665e50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0149b0 tx=0x7f10bc00c4d0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3810020166 shutdown_connections 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b663a00 0x560f3b665e50 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65ae80 0x560f3b663430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3810020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3810020166 >> 172.21.15.5:0/3810020166 conn(0x560f3b654ba0 msgr2=0x560f3b654f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3810020166 shutdown_connections 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.500+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3810020166 wait complete. 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 -- start start 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 Processor -- start 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b6632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b660650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b660bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560f3b666910 con 0x560f3b65ae80 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560f3b578500 con 0x560f3b663a00 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f3b666a90 con 0x560f3b65a4d0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d5c7d740 10 client.0 _ll_register_callbacks cb 0x560f3b546b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b660650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b660650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42038/0 (socket says 172.21.15.5:42038) 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 -- 172.21.15.5:0/3267459661 learned_addr learned my addr 172.21.15.5:0/3267459661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b6632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b660bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 -- 172.21.15.5:0/3267459661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 msgr2=0x560f3b6632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b6632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 -- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 msgr2=0x560f3b660bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b660bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.501+0000 7f10d0e2a640 1 -- 172.21.15.5:0/3267459661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3b2d13f0 con 0x560f3b65ae80 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b660650 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009de0 tx=0x7f10b40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729f99f04dea4ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10abfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10abfff640 1 -- 172.21.15.5:0/3267459661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4012860 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10abfff640 1 -- 172.21.15.5:0/3267459661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f10b4012a00 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10bc00fc10 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10abfff640 1 -- 172.21.15.5:0/3267459661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10b4012c00 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 10 client.18978 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f3b39b7d0 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10abfff640 1 -- 172.21.15.5:0/3267459661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc00fc10 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10abfff640 1 -- 172.21.15.5:0/3267459661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f10b400f040 con 0x560f3b65ae80 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 10 client.18978 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 10 client.18978 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 20 client.18978 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b673ca0 0x560f3b6760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 4 client.18978 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.502+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3b55e930 con 0x560f3b673ca0 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.503+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b673ca0 0x560f3b6760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.503+0000 7f10cbfff640 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b673ca0 0x560f3b6760c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd5d4536176406c0 server_cookie=7759b7c968fcff6a in_seq=0 out_seq=0 2024-04-22T13:37:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.503+0000 7f10abfff640 10 client.18978 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10abfff640 1 -- 172.21.15.5:0/3267459661 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f3b55e930 con 0x560f3b673ca0 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10abfff640 10 client.18978 handle_command_reply: tid=0 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 client.18978 shutdown 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 msgr2=0x560f3b660650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b660650 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009de0 tx=0x7f10b40117b0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 shutdown_connections 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b673ca0 0x560f3b6760c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b660650 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b660bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10abfff640 0 client.18978 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/3267459661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b6632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 >> 172.21.15.5:0/3267459661 conn(0x560f3b654ba0 msgr2=0x560f3b6568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 shutdown_connections 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 1 -- 172.21.15.5:0/3267459661 wait complete. 2024-04-22T13:37:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.504+0000 7f10d5c7d740 20 client.18978 trim_cache size 0 max 16384 2024-04-22T13:37:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.517+0000 7f10d5c7d740 1 Processor -- start 2024-04-22T13:37:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.517+0000 7f10d5c7d740 1 -- start start 2024-04-22T13:37:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65a4d0 0x560f3b653a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b662d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b663a00 0x560f3b6632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1afde0 con 0x560f3b65ae80 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1b1920 con 0x560f3b65a4d0 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1b04b0 con 0x560f3b663a00 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b662d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b662d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42048/0 (socket says 172.21.15.5:42048) 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 -- 172.21.15.5:0/2953964938 learned_addr learned my addr 172.21.15.5:0/2953964938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10cbfff640 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b663a00 0x560f3b6632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10cbfff640 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65a4d0 0x560f3b653a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 -- 172.21.15.5:0/2953964938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b663a00 msgr2=0x560f3b6632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b663a00 0x560f3b6632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 -- 172.21.15.5:0/2953964938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65a4d0 msgr2=0x560f3b653a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65a4d0 0x560f3b653a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 -- 172.21.15.5:0/2953964938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3b65eba0 con 0x560f3b65ae80 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10cbfff640 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65a4d0 0x560f3b653a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b662d80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f10b4001eb0 tx=0x7f10b4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9440e9f4134521 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10c27fc640 1 -- 172.21.15.5:0/2953964938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4000e60 con 0x560f3b65ae80 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10c27fc640 1 -- 172.21.15.5:0/2953964938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b4005850 con 0x560f3b65ae80 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.518+0000 7f10c27fc640 1 -- 172.21.15.5:0/2953964938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4019510 con 0x560f3b65ae80 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 -- 172.21.15.5:0/2953964938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 msgr2=0x560f3b662d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b662d80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f10b4001eb0 tx=0x7f10b4009670 comp rx=0 tx=0).stop 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 -- 172.21.15.5:0/2953964938 shutdown_connections 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b662d80 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b663a00 0x560f3b6632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/2953964938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b65a4d0 0x560f3b653a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 -- 172.21.15.5:0/2953964938 >> 172.21.15.5:0/2953964938 conn(0x560f3b654ba0 msgr2=0x560f3b653370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 -- 172.21.15.5:0/2953964938 shutdown_connections 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 -- 172.21.15.5:0/2953964938 wait complete. 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 -- start start 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.519+0000 7f10d5c7d740 1 Processor -- start 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b65da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b65dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1b1250 con 0x560f3b65ae80 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1b0b80 con 0x560f3b663a00 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1adde0 con 0x560f3b65a4d0 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d5c7d740 10 client.0 _ll_register_callbacks cb 0x7f10b40099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b65da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b65da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42064/0 (socket says 172.21.15.5:42064) 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 -- 172.21.15.5:0/899071544 learned_addr learned my addr 172.21.15.5:0/899071544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10cbfff640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 -- 172.21.15.5:0/899071544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 msgr2=0x560f3b65d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 -- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 msgr2=0x560f3b65dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b65dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 -- 172.21.15.5:0/899071544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3b6d93e0 con 0x560f3b65ae80 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10cbfff640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10d0e2a640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b65da70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f10b40096e0 tx=0x7f10b401a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a53713e4f2e499 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.520+0000 7f10abfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10abfff640 1 -- 172.21.15.5:0/899071544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4039030 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10abfff640 1 -- 172.21.15.5:0/899071544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f10b4012660 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b4005850 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10abfff640 1 -- 172.21.15.5:0/899071544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10b4012860 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 10 client.18984 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f3b6da610 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10abfff640 1 -- 172.21.15.5:0/899071544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b4005850 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10abfff640 1 -- 172.21.15.5:0/899071544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f10b4023460 con 0x560f3b65ae80 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 10 client.18984 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 10 client.18984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 20 client.18984 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b6dac10 0x560f3b6dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 4 client.18984 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.521+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f3b5ccc10 con 0x560f3b6dac10 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10cbfff640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b6dac10 0x560f3b6dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10cbfff640 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b6dac10 0x560f3b6dd000 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6ecf795a333da8a server_cookie=af8c6930186cb922 in_seq=0 out_seq=0 2024-04-22T13:37:04.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10abfff640 10 client.18984 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10abfff640 1 -- 172.21.15.5:0/899071544 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f3b5ccc10 con 0x560f3b6dac10 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10abfff640 10 client.18984 handle_command_reply: tid=0 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 client.18984 shutdown 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 msgr2=0x560f3b65da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b65da70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f10b40096e0 tx=0x7f10b401a660 comp rx=0 tx=0).stop 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 shutdown_connections 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x560f3b6dac10 0x560f3b6dd000 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560f3b65ae80 0x560f3b65da70 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10abfff640 0 client.18984 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560f3b663a00 0x560f3b65dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 --2- 172.21.15.5:0/899071544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f3b65a4d0 0x560f3b65d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.522+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 >> 172.21.15.5:0/899071544 conn(0x560f3b654ba0 msgr2=0x560f3b6567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 shutdown_connections 2024-04-22T13:37:04.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d5c7d740 1 -- 172.21.15.5:0/899071544 wait complete. 2024-04-22T13:37:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d5c7d740 20 client.18984 trim_cache size 0 max 16384 2024-04-22T13:37:04.524 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f10b80a3dd0 msgr2=0x7f10b80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f10b80a3dd0 0x7f10b80a6290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f10c40baa00 tx=0x7f10c402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 msgr2=0x7f10cc1a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 0x7f10cc1a68c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f10c4005e30 tx=0x7f10c4032420 comp rx=0 tx=0).stop 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 shutdown_connections 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1035a0 0x7f10cc1a68c0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f10b80a3dd0 0x7f10b80a6290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc106160 0x7f10cc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 --2- 172.21.15.5:0/876148721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10cc101090 0x7f10cc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 >> 172.21.15.5:0/876148721 conn(0x7f10cc078000 msgr2=0x7f10cc0695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 shutdown_connections 2024-04-22T13:37:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.523+0000 7f10d308e640 1 -- 172.21.15.5:0/876148721 wait complete. 2024-04-22T13:37:04.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 -- 172.21.15.5:0/286269205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d8102180 msgr2=0x7fb2d8102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 --2- 172.21.15.5:0/286269205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d8102180 0x7fb2d8102560 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009900 tx=0x7fb2c802bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 -- 172.21.15.5:0/286269205 shutdown_connections 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 --2- 172.21.15.5:0/286269205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8107250 0x7fb2d810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 --2- 172.21.15.5:0/286269205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8102b30 0x7fb2d8106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 --2- 172.21.15.5:0/286269205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d8102180 0x7fb2d8102560 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 -- 172.21.15.5:0/286269205 >> 172.21.15.5:0/286269205 conn(0x7fb2d80fda00 msgr2=0x7fb2d80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 -- 172.21.15.5:0/286269205 shutdown_connections 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 -- 172.21.15.5:0/286269205 wait complete. 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.638+0000 7fb2de104640 1 Processor -- start 2024-04-22T13:37:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 -- start start 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8102180 0x7fb2d8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8107250 0x7fb2d8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 0x7fb2d819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8112190 con 0x7fb2d819cfa0 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8112010 con 0x7fb2d8102180 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2de104640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8112310 con 0x7fb2d8107250 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 0x7fb2d819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 0x7fb2d819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42076/0 (socket says 172.21.15.5:42076) 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 -- 172.21.15.5:0/264002400 learned_addr learned my addr 172.21.15.5:0/264002400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8107250 0x7fb2d8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8102180 0x7fb2d8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 -- 172.21.15.5:0/264002400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8107250 msgr2=0x7fb2d8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8107250 0x7fb2d8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 -- 172.21.15.5:0/264002400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8102180 msgr2=0x7fb2d8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8102180 0x7fb2d8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 -- 172.21.15.5:0/264002400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d819da90 con 0x7fb2d819cfa0 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8102180 0x7fb2d8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8107250 0x7fb2d8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.639+0000 7fb2d7fff640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 0x7fb2d819d450 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000eb10 tx=0x7fb2d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.640+0000 7fb2d4ff9640 1 -- 172.21.15.5:0/264002400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004b20 con 0x7fb2d819cfa0 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.640+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d819dd80 con 0x7fb2d819cfa0 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.640+0000 7fb2d4ff9640 1 -- 172.21.15.5:0/264002400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d0004cc0 con 0x7fb2d819cfa0 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.640+0000 7fb2d4ff9640 1 -- 172.21.15.5:0/264002400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0010670 con 0x7fb2d819cfa0 2024-04-22T13:37:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.640+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81a5520 con 0x7fb2d819cfa0 2024-04-22T13:37:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2d4ff9640 1 -- 172.21.15.5:0/264002400 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb2d0005020 con 0x7fb2d819cfa0 2024-04-22T13:37:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 Processor -- start 2024-04-22T13:37:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 -- start start 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dfe58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfe5e80 0x555b5dfee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 0x555b5dff0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b5df4a1f0 con 0x555b5dfeea00 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b5dee74b0 con 0x555b5dfe5e80 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.642+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555b5df461b0 con 0x555b5dfe54d0 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 0x555b5dff0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 0x555b5dff0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42080/0 (socket says 172.21.15.5:42080) 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2170706167 learned_addr learned my addr 172.21.15.5:0/2170706167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d4ff9640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb2c00a3d20 0x7fb2c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dfe58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfe5e80 0x555b5dfee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d7fff640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb2c00a3d20 0x7fb2c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2170706167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 msgr2=0x555b5dfe58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dfe58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2170706167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfe5e80 msgr2=0x555b5dfee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfe5e80 0x555b5dfee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2170706167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b5ded91b0 con 0x555b5dfeea00 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d4ff9640 1 -- 172.21.15.5:0/264002400 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb2d00d0e70 con 0x7fb2d819cfa0 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 0x555b5dff0e50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc012550 tx=0x7fb2cc00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacac82ea3116bea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2bdffb640 1 -- 172.21.15.5:0/2170706167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc018070 con 0x555b5dfeea00 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2bdffb640 1 -- 172.21.15.5:0/2170706167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc0054d0 con 0x555b5dfeea00 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2bdffb640 1 -- 172.21.15.5:0/2170706167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc004b70 con 0x555b5dfeea00 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.643+0000 7fb2d7fff640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb2c00a3d20 0x7fb2c00a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00bbfd0 tx=0x7fb2d00bd040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2170706167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 msgr2=0x555b5dff0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 0x555b5dff0e50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc012550 tx=0x7fb2cc00b6a0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2170706167 shutdown_connections 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfeea00 0x555b5dff0e50 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfe5e80 0x555b5dfee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2170706167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dfe58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2170706167 >> 172.21.15.5:0/2170706167 conn(0x555b5dfdfba0 msgr2=0x555b5dfdff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2170706167 shutdown_connections 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2170706167 wait complete. 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.644+0000 7fb2e0dd5740 1 -- start start 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 Processor -- start 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5dfee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5dfeb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5dfebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b5dff1970 con 0x555b5dfe54d0 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555b5df03500 con 0x555b5dfeea00 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555b5dff1af0 con 0x555b5dfe5e80 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2e0dd5740 10 client.0 _ll_register_callbacks cb 0x555b5ded1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5dfee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5dfee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42084/0 (socket says 172.21.15.5:42084) 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2519317586 learned_addr learned my addr 172.21.15.5:0/2519317586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5dfeb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 msgr2=0x555b5dfeb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5dfeb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2519317586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 msgr2=0x555b5dfebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5dfebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2519317586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5dc5c3f0 con 0x555b5dfe54d0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5dfeb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.645+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5dfee300 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b990 tx=0x7fb2cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424d73a0cc92aab server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2519317586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc018040 con 0x555b5dfe54d0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2519317586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb2cc004d80 con 0x555b5dfe54d0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2cc0054d0 con 0x555b5dfe54d0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2519317586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc010720 con 0x555b5dfe54d0 2024-04-22T13:37:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 10 client.18996 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b5dd267d0 con 0x555b5dfe54d0 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2519317586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc0054d0 con 0x555b5dfe54d0 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2519317586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb2cc031960 con 0x555b5dfe54d0 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 10 client.18996 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 10 client.18996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 20 client.18996 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5dffea80 0x555b5e000e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 4 client.18996 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.646+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b5dee9930 con 0x555b5dffea80 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.647+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5dffea80 0x555b5e000e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.647+0000 7fb2b6ffd640 10 client.18996 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.647+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5dffea80 0x555b5e000e70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c72c2e3fae7339a5 server_cookie=15c4a748ec730e15 in_seq=0 out_seq=0 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2519317586 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b5dee9930 con 0x555b5dffea80 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2b6ffd640 10 client.18996 handle_command_reply: tid=0 2024-04-22T13:37:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 client.18996 shutdown 2024-04-22T13:37:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 msgr2=0x555b5dfee300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5dfee300 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b990 tx=0x7fb2cc0046c0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 shutdown_connections 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5dffea80 0x555b5e000e70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5dfebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2b6ffd640 0 client.18996 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5dfeb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2519317586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5dfee300 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 >> 172.21.15.5:0/2519317586 conn(0x555b5dfdfba0 msgr2=0x555b5dfe18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.648+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 shutdown_connections 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.649+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2519317586 wait complete. 2024-04-22T13:37:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.649+0000 7fb2e0dd5740 20 client.18996 trim_cache size 0 max 16384 2024-04-22T13:37:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 Processor -- start 2024-04-22T13:37:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 -- start start 2024-04-22T13:37:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dffa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 0x555b5dfeddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5dfee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81adbc0 con 0x555b5dfe5e80 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81af700 con 0x555b5dfeea00 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81ae290 con 0x555b5dfe54d0 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 0x555b5dfeddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 0x555b5dfeddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42090/0 (socket says 172.21.15.5:42090) 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2d77fe640 1 -- 172.21.15.5:0/1653417859 learned_addr learned my addr 172.21.15.5:0/1653417859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.662+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dffa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2d77fe640 1 -- 172.21.15.5:0/1653417859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 msgr2=0x555b5dffa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dffa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2d77fe640 1 -- 172.21.15.5:0/1653417859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 msgr2=0x555b5dfee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5dfee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2d77fe640 1 -- 172.21.15.5:0/1653417859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b5dfe1e90 con 0x555b5dfe5e80 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 0x555b5dfeddc0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8005ab0 tx=0x7fb2c802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f921cf1c50c5456c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2bd7fa640 1 -- 172.21.15.5:0/1653417859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80043c0 con 0x555b5dfe5e80 2024-04-22T13:37:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2bd7fa640 1 -- 172.21.15.5:0/1653417859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c802cc90 con 0x555b5dfe5e80 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2bd7fa640 1 -- 172.21.15.5:0/1653417859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8004990 con 0x555b5dfe5e80 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/1653417859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 msgr2=0x555b5dfeddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 0x555b5dfeddc0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8005ab0 tx=0x7fb2c802c660 comp rx=0 tx=0).stop 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/1653417859 shutdown_connections 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe5e80 0x555b5dfeddc0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5dfee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/1653417859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe54d0 0x555b5dffa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.663+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/1653417859 >> 172.21.15.5:0/1653417859 conn(0x555b5dfdfba0 msgr2=0x555b5dfe6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/1653417859 shutdown_connections 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/1653417859 wait complete. 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 -- start start 2024-04-22T13:37:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 Processor -- start 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5e061900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5e061e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5e05bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81af030 con 0x555b5dfe54d0 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81ae960 con 0x555b5dfeea00 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81abbc0 con 0x555b5dfe5e80 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2e0dd5740 10 client.0 _ll_register_callbacks cb 0x7fb2cc00b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5e061900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5e061900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42096/0 (socket says 172.21.15.5:42096) 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.664+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2780326482 learned_addr learned my addr 172.21.15.5:0/2780326482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5e05bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5e061e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 msgr2=0x555b5e061e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5e061e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2780326482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 msgr2=0x555b5e05bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5e05bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 -- 172.21.15.5:0/2780326482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5e05c400 con 0x555b5dfe54d0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2d6ffd640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5e061900 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0056d0 tx=0x7fb2cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8031e09495a48a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2780326482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc018030 con 0x555b5dfe54d0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c802cc90 con 0x555b5dfe54d0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2780326482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb2cc004d80 con 0x555b5dfe54d0 2024-04-22T13:37:04.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2780326482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc044050 con 0x555b5dfe54d0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2e0dd5740 10 client.19002 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b5e069100 con 0x555b5dfe54d0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.665+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2780326482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c802cc90 con 0x555b5dfe54d0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2780326482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb2cc028cb0 con 0x555b5dfe54d0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2e0dd5740 10 client.19002 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2e0dd5740 10 client.19002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2e0dd5740 20 client.19002 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5e069760 0x555b5e06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2e0dd5740 4 client.19002 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b5df57c10 con 0x555b5e069760 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5e069760 0x555b5e06bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2b6ffd640 10 client.19002 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.666+0000 7fb2d77fe640 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5e069760 0x555b5e06bb80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=726045eb539d7624 server_cookie=381ceb2e04b54227 in_seq=0 out_seq=0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2b6ffd640 1 -- 172.21.15.5:0/2780326482 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b5df57c10 con 0x555b5e069760 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2b6ffd640 10 client.19002 handle_command_reply: tid=0 2024-04-22T13:37:04.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 client.19002 shutdown 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 msgr2=0x555b5e061900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5e061900 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0056d0 tx=0x7fb2cc005f90 comp rx=0 tx=0).stop 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 shutdown_connections 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x555b5e069760 0x555b5e06bb80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555b5dfeea00 0x555b5e05bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b5dfe5e80 0x555b5e061e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2b6ffd640 0 client.19002 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 --2- 172.21.15.5:0/2780326482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b5dfe54d0 0x555b5e061900 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 >> 172.21.15.5:0/2780326482 conn(0x555b5dfdfba0 msgr2=0x555b5dfe6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 shutdown_connections 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 1 -- 172.21.15.5:0/2780326482 wait complete. 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2e0dd5740 20 client.19002 trim_cache size 0 max 16384 2024-04-22T13:37:04.668 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:04.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb2c00a3d20 msgr2=0x7fb2c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2de104640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb2c00a3d20 0x7fb2c00a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00bbfd0 tx=0x7fb2d00bd040 comp rx=0 tx=0).stop 2024-04-22T13:37:04.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 msgr2=0x7fb2d819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.667+0000 7fb2de104640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 0x7fb2d819d450 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000eb10 tx=0x7fb2d000c2d0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 shutdown_connections 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d819cfa0 0x7fb2d819d450 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb2c00a3d20 0x7fb2c00a61e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2d8107250 0x7fb2d8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 --2- 172.21.15.5:0/264002400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2d8102180 0x7fb2d8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 >> 172.21.15.5:0/264002400 conn(0x7fb2d80fda00 msgr2=0x7fb2d80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 shutdown_connections 2024-04-22T13:37:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.668+0000 7fb2de104640 1 -- 172.21.15.5:0/264002400 wait complete. 2024-04-22T13:37:04.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 -- 172.21.15.5:0/2478041522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360103400 msgr2=0x7f636010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 --2- 172.21.15.5:0/2478041522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360103400 0x7f636010de00 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6358009cd0 tx=0x7f635802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 -- 172.21.15.5:0/2478041522 shutdown_connections 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 --2- 172.21.15.5:0/2478041522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360103400 0x7f636010de00 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 --2- 172.21.15.5:0/2478041522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6360102a60 0x7f6360102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 --2- 172.21.15.5:0/2478041522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6360101180 0x7f6360101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 -- 172.21.15.5:0/2478041522 >> 172.21.15.5:0/2478041522 conn(0x7f6360077f70 msgr2=0x7f6360078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 -- 172.21.15.5:0/2478041522 shutdown_connections 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.782+0000 7f6367681640 1 -- 172.21.15.5:0/2478041522 wait complete. 2024-04-22T13:37:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 Processor -- start 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 -- start start 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6360101180 0x7f6360197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 0x7f6360197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6360103400 0x7f636019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63601111a0 con 0x7f6360102a60 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6360111020 con 0x7f6360101180 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6367681640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6360111320 con 0x7f6360103400 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 0x7f6360197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 0x7f6360197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42118/0 (socket says 172.21.15.5:42118) 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 -- 172.21.15.5:0/284047402 learned_addr learned my addr 172.21.15.5:0/284047402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f636541d640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6360101180 0x7f6360197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6365c1e640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6360103400 0x7f636019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 -- 172.21.15.5:0/284047402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6360103400 msgr2=0x7f636019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6360103400 0x7f636019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 -- 172.21.15.5:0/284047402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6360101180 msgr2=0x7f6360197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6360101180 0x7f6360197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.783+0000 7f6364c1c640 1 -- 172.21.15.5:0/284047402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636019c6f0 con 0x7f6360102a60 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.784+0000 7f6364c1c640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 0x7f6360197c40 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f635000e9f0 tx=0x7f635000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.784+0000 7f63567fc640 1 -- 172.21.15.5:0/284047402 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635000cd80 con 0x7f6360102a60 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.784+0000 7f63567fc640 1 -- 172.21.15.5:0/284047402 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63500108e0 con 0x7f6360102a60 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.784+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636019c9e0 con 0x7f6360102a60 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.784+0000 7f63567fc640 1 -- 172.21.15.5:0/284047402 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350018980 con 0x7f6360102a60 2024-04-22T13:37:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.784+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63601a4270 con 0x7f6360102a60 2024-04-22T13:37:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.786+0000 7f63567fc640 1 -- 172.21.15.5:0/284047402 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6350010430 con 0x7f6360102a60 2024-04-22T13:37:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.786+0000 7f636a277740 1 Processor -- start 2024-04-22T13:37:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.786+0000 7f636a277740 1 -- start start 2024-04-22T13:37:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636a277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 0x55cbd57b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636a277740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b2090 0x55cbd57ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636a277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f63567fc640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f634c0a3da0 0x7f634c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636a277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbd57141f0 con 0x55cbd57b16e0 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636a277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cbd56b14b0 con 0x55cbd57bac10 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636a277740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cbd57101b0 con 0x55cbd57b2090 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 0x55cbd57b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f63567fc640 1 -- 172.21.15.5:0/284047402 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f63500d09d0 con 0x7f6360102a60 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 0x55cbd57b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42120/0 (socket says 172.21.15.5:42120) 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 -- 172.21.15.5:0/885594853 learned_addr learned my addr 172.21.15.5:0/885594853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f634c0a3da0 0x7f634c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f636541d640 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b2090 0x55cbd57ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6364c1c640 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 -- 172.21.15.5:0/885594853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b2090 msgr2=0x55cbd57ba640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b2090 0x55cbd57ba640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 -- 172.21.15.5:0/885594853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 msgr2=0x55cbd57bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.787+0000 7f6365c1e640 1 -- 172.21.15.5:0/885594853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd56a31b0 con 0x55cbd57b16e0 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f6365c1e640 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 0x55cbd57b1ac0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6358004af0 tx=0x7f6358032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f709e6517b29c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f6365c1e640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f634c0a3da0 0x7f634c0a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f635802c700 tx=0x7f6358042000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f634b7fe640 1 -- 172.21.15.5:0/885594853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358051070 con 0x55cbd57b16e0 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f634b7fe640 1 -- 172.21.15.5:0/885594853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63580327c0 con 0x55cbd57b16e0 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f634b7fe640 1 -- 172.21.15.5:0/885594853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358039040 con 0x55cbd57b16e0 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 -- 172.21.15.5:0/885594853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 msgr2=0x55cbd57b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 0x55cbd57b1ac0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6358004af0 tx=0x7f6358032f70 comp rx=0 tx=0).stop 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 -- 172.21.15.5:0/885594853 shutdown_connections 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b2090 0x55cbd57ba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 --2- 172.21.15.5:0/885594853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b16e0 0x55cbd57b1ac0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 -- 172.21.15.5:0/885594853 >> 172.21.15.5:0/885594853 conn(0x55cbd57abdb0 msgr2=0x55cbd57ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 -- 172.21.15.5:0/885594853 shutdown_connections 2024-04-22T13:37:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.788+0000 7f636a277740 1 -- 172.21.15.5:0/885594853 wait complete. 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 -- start start 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 Processor -- start 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbd57bdb20 con 0x55cbd57b2090 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55cbd56cd500 con 0x55cbd57bac10 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cbd57bdca0 con 0x55cbd57b16e0 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f636a277740 10 client.0 _ll_register_callbacks cb 0x55cbd569bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42128/0 (socket says 172.21.15.5:42128) 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.789+0000 7f6364c1c640 1 -- 172.21.15.5:0/3654364680 learned_addr learned my addr 172.21.15.5:0/3654364680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636541d640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6365c1e640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6364c1c640 1 -- 172.21.15.5:0/3654364680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 msgr2=0x55cbd57ba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6364c1c640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6364c1c640 1 -- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 msgr2=0x55cbd57b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6364c1c640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6364c1c640 1 -- 172.21.15.5:0/3654364680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd54263f0 con 0x55cbd57b2090 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636541d640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6364c1c640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57b7940 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f63500ce1d0 tx=0x7f635009c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fdf0b579984160 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6348ff9640 1 -- 172.21.15.5:0/3654364680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500a55e0 con 0x55cbd57b2090 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6348ff9640 1 -- 172.21.15.5:0/3654364680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f63500ba5d0 con 0x55cbd57b2090 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63580327c0 con 0x55cbd57b2090 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6348ff9640 1 -- 172.21.15.5:0/3654364680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63500bbb30 con 0x55cbd57b2090 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636a277740 10 client.19014 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbd54f07d0 con 0x55cbd57b2090 2024-04-22T13:37:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6348ff9640 1 -- 172.21.15.5:0/3654364680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63580327c0 con 0x55cbd57b2090 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f6348ff9640 1 -- 172.21.15.5:0/3654364680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6350018bb0 con 0x55cbd57b2090 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636a277740 10 client.19014 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.790+0000 7f636a277740 10 client.19014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f636a277740 20 client.19014 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f636a277740 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd57cadb0 0x55cbd57cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f636a277740 4 client.19014 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd56b3930 con 0x55cbd57cadb0 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f636541d640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd57cadb0 0x55cbd57cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f636541d640 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd57cadb0 0x55cbd57cd1e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2f89908d22e4a50 server_cookie=6f13824897507ca3 in_seq=0 out_seq=0 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.791+0000 7f6348ff9640 10 client.19014 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f6348ff9640 1 -- 172.21.15.5:0/3654364680 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbd56b3930 con 0x55cbd57cadb0 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f6348ff9640 10 client.19014 handle_command_reply: tid=0 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 client.19014 shutdown 2024-04-22T13:37:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 msgr2=0x55cbd57b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57b7940 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f63500ce1d0 tx=0x7f635009c470 comp rx=0 tx=0).stop 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 shutdown_connections 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd57cadb0 0x55cbd57cd1e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57b7940 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f6348ff9640 0 client.19014 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 --2- 172.21.15.5:0/3654364680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 >> 172.21.15.5:0/3654364680 conn(0x55cbd57abdb0 msgr2=0x55cbd57adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 shutdown_connections 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 1 -- 172.21.15.5:0/3654364680 wait complete. 2024-04-22T13:37:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.792+0000 7f636a277740 20 client.19014 trim_cache size 0 max 16384 2024-04-22T13:37:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.805+0000 7f636a277740 1 Processor -- start 2024-04-22T13:37:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.805+0000 7f636a277740 1 -- start start 2024-04-22T13:37:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f636a277740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f636a277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f636a277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f636a277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63601ab9d0 con 0x55cbd57b2090 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f636a277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63601ad510 con 0x55cbd57bac10 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f636a277740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63601ac0a0 con 0x55cbd57b16e0 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42138/0 (socket says 172.21.15.5:42138) 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 -- 172.21.15.5:0/2395452864 learned_addr learned my addr 172.21.15.5:0/2395452864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 -- 172.21.15.5:0/2395452864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 msgr2=0x55cbd57ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6365c1e640 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 -- 172.21.15.5:0/2395452864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 msgr2=0x55cbd57d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57d2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 -- 172.21.15.5:0/2395452864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd57bd390 con 0x55cbd57b2090 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f6364c1c640 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57a8a80 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f635009acd0 tx=0x7f6350000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb31edab0910f7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f634affd640 1 -- 172.21.15.5:0/2395452864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500870b0 con 0x55cbd57b2090 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f634affd640 1 -- 172.21.15.5:0/2395452864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6350087300 con 0x55cbd57b2090 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.806+0000 7f634affd640 1 -- 172.21.15.5:0/2395452864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350098770 con 0x55cbd57b2090 2024-04-22T13:37:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 -- 172.21.15.5:0/2395452864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 msgr2=0x55cbd57a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57a8a80 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f635009acd0 tx=0x7f6350000bc0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 -- 172.21.15.5:0/2395452864 shutdown_connections 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57a8a80 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 --2- 172.21.15.5:0/2395452864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 -- 172.21.15.5:0/2395452864 >> 172.21.15.5:0/2395452864 conn(0x55cbd57abdb0 msgr2=0x55cbd57a8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 -- 172.21.15.5:0/2395452864 shutdown_connections 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 -- 172.21.15.5:0/2395452864 wait complete. 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 -- start start 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.807+0000 7f636a277740 1 Processor -- start 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63601ace40 con 0x55cbd57b2090 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f63601ac770 con 0x55cbd57bac10 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63601a99d0 con 0x55cbd57b16e0 2024-04-22T13:37:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 10 client.0 _ll_register_callbacks cb 0x7f635009c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42146/0 (socket says 172.21.15.5:42146) 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 -- 172.21.15.5:0/1276731583 learned_addr learned my addr 172.21.15.5:0/1276731583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6365c1e640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636541d640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 -- 172.21.15.5:0/1276731583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 msgr2=0x55cbd57c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 -- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 msgr2=0x55cbd57bfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 -- 172.21.15.5:0/1276731583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd57c04b0 con 0x55cbd57b2090 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6364c1c640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57c3460 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f63500993f0 tx=0x7f63500a0ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32230af06cbaf1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6348ff9640 1 -- 172.21.15.5:0/1276731583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635009c380 con 0x55cbd57b2090 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f6348ff9640 1 -- 172.21.15.5:0/1276731583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f635009be70 con 0x55cbd57b2090 2024-04-22T13:37:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.808+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6350087300 con 0x55cbd57b2090 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f6348ff9640 1 -- 172.21.15.5:0/1276731583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f635009ca20 con 0x55cbd57b2090 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 10 client.19020 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbd5831bf0 con 0x55cbd57b2090 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f6348ff9640 1 -- 172.21.15.5:0/1276731583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6350087300 con 0x55cbd57b2090 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f6348ff9640 1 -- 172.21.15.5:0/1276731583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f63500a1eb0 con 0x55cbd57b2090 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 10 client.19020 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 10 client.19020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 20 client.19020 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd5832140 0x55cbd5834560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 4 client.19020 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbd5721c10 con 0x55cbd5832140 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.809+0000 7f636541d640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd5832140 0x55cbd5834560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636541d640 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd5832140 0x55cbd5834560 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ea1efde788ef55c server_cookie=9a9edb29fc144249 in_seq=0 out_seq=0 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f6348ff9640 10 client.19020 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f6348ff9640 1 -- 172.21.15.5:0/1276731583 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbd5721c10 con 0x55cbd5832140 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f6348ff9640 10 client.19020 handle_command_reply: tid=0 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 client.19020 shutdown 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 msgr2=0x55cbd57c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57c3460 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f63500993f0 tx=0x7f63500a0ea0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 shutdown_connections 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x55cbd5832140 0x55cbd5834560 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbd57b2090 0x55cbd57c3460 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cbd57bac10 0x55cbd57bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 --2- 172.21.15.5:0/1276731583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbd57b16e0 0x55cbd57c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f6348ff9640 0 client.19020 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 >> 172.21.15.5:0/1276731583 conn(0x55cbd57abdb0 msgr2=0x55cbd57ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 shutdown_connections 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 1 -- 172.21.15.5:0/1276731583 wait complete. 2024-04-22T13:37:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.810+0000 7f636a277740 20 client.19020 trim_cache size 0 max 16384 2024-04-22T13:37:04.812 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f634c0a3da0 msgr2=0x7f634c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f634c0a3da0 0x7f634c0a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f635802c700 tx=0x7f6358042000 comp rx=0 tx=0).stop 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 msgr2=0x7f6360197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 0x7f6360197c40 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f635000e9f0 tx=0x7f635000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 shutdown_connections 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6360102a60 0x7f6360197c40 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f634c0a3da0 0x7f634c0a6260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6360103400 0x7f636019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 --2- 172.21.15.5:0/284047402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6360101180 0x7f6360197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 >> 172.21.15.5:0/284047402 conn(0x7f6360077f70 msgr2=0x7f6360104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 shutdown_connections 2024-04-22T13:37:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.811+0000 7f6367681640 1 -- 172.21.15.5:0/284047402 wait complete. 2024-04-22T13:37:04.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:37:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.925+0000 7f24083bd640 1 -- 172.21.15.5:0/2473321854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400102a60 msgr2=0x7f2400102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.925+0000 7f24083bd640 1 --2- 172.21.15.5:0/2473321854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400102a60 0x7f2400102ec0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009930 tx=0x7f23f002bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 -- 172.21.15.5:0/2473321854 shutdown_connections 2024-04-22T13:37:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 --2- 172.21.15.5:0/2473321854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400102a60 0x7f2400102ec0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 --2- 172.21.15.5:0/2473321854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2400103400 0x7f240010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 --2- 172.21.15.5:0/2473321854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2400101180 0x7f2400101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 -- 172.21.15.5:0/2473321854 >> 172.21.15.5:0/2473321854 conn(0x7f2400077f70 msgr2=0x7f2400078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 -- 172.21.15.5:0/2473321854 shutdown_connections 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 -- 172.21.15.5:0/2473321854 wait complete. 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 Processor -- start 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 -- start start 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 0x7f24000fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2400102a60 0x7f24000fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2400103400 0x7f24000fe870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.926+0000 7f24083bd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24001111a0 con 0x7f2400101180 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f24083bd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2400111020 con 0x7f2400102a60 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f24083bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2400111320 con 0x7f2400103400 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 0x7f24000fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 0x7f24000fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42178/0 (socket says 172.21.15.5:42178) 2024-04-22T13:37:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 -- 172.21.15.5:0/1411656708 learned_addr learned my addr 172.21.15.5:0/1411656708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2405958640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2400102a60 0x7f24000fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f240695a640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2400103400 0x7f24000fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 -- 172.21.15.5:0/1411656708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2400103400 msgr2=0x7f24000fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2400103400 0x7f24000fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 -- 172.21.15.5:0/1411656708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2400102a60 msgr2=0x7f24000fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2400102a60 0x7f24000fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 -- 172.21.15.5:0/1411656708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24001a6d50 con 0x7f2400101180 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f2406159640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 0x7f24000fddf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00d950 tx=0x7f23fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f23f77fe640 1 -- 172.21.15.5:0/1411656708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc017030 con 0x7f2400101180 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24001a6fe0 con 0x7f2400101180 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f23f77fe640 1 -- 172.21.15.5:0/1411656708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc004020 con 0x7f2400101180 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f23f77fe640 1 -- 172.21.15.5:0/1411656708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0047a0 con 0x7f2400101180 2024-04-22T13:37:04.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.927+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24001a7500 con 0x7f2400101180 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.930+0000 7f23f77fe640 1 -- 172.21.15.5:0/1411656708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f23fc009a30 con 0x7f2400101180 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.930+0000 7f240afb3740 1 Processor -- start 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.930+0000 7f240afb3740 1 -- start start 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240afb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f1198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240afb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 0x55f03f122430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240afb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f124e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240afb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f03f07e1f0 con 0x55f03f119e80 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240afb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f03f01b4b0 con 0x55f03f122a00 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240afb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f03f07a1b0 con 0x55f03f1194d0 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f23f77fe640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 0x55f03f122430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240695a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f1198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 0x55f03f122430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42192/0 (socket says 172.21.15.5:42192) 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 -- 172.21.15.5:0/2548341132 learned_addr learned my addr 172.21.15.5:0/2548341132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2405958640 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f124e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f240695a640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f23f77fe640 1 -- 172.21.15.5:0/1411656708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f23fc0d00c0 con 0x7f2400101180 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 -- 172.21.15.5:0/2548341132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 msgr2=0x55f03f1198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f1198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 -- 172.21.15.5:0/2548341132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 msgr2=0x55f03f124e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f124e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 -- 172.21.15.5:0/2548341132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03f00d1b0 con 0x55f03f119e80 2024-04-22T13:37:04.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2405958640 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f124e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.931+0000 7f2406159640 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 0x55f03f122430 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0cd850 tx=0x7f23fc09bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6cb423239b4e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240695a640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005fd0 tx=0x7f23f8009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f23f4bfe640 1 -- 172.21.15.5:0/2548341132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc09c800 con 0x55f03f119e80 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f23f4bfe640 1 -- 172.21.15.5:0/2548341132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc09c100 con 0x55f03f119e80 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f23f4bfe640 1 -- 172.21.15.5:0/2548341132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0cdb30 con 0x55f03f119e80 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 -- 172.21.15.5:0/2548341132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 msgr2=0x55f03f122430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 0x55f03f122430 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0cd850 tx=0x7f23fc09bed0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 -- 172.21.15.5:0/2548341132 shutdown_connections 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f119e80 0x55f03f122430 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f124e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 --2- 172.21.15.5:0/2548341132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f1198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.932+0000 7f240afb3740 1 -- 172.21.15.5:0/2548341132 >> 172.21.15.5:0/2548341132 conn(0x55f03f113ba0 msgr2=0x55f03f113f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 -- 172.21.15.5:0/2548341132 shutdown_connections 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 -- 172.21.15.5:0/2548341132 wait complete. 2024-04-22T13:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 -- start start 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 Processor -- start 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f122360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f119e80 0x55f03f11f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f11fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f03f125910 con 0x55f03f122a00 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f03f037500 con 0x55f03f119e80 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f03f125a90 con 0x55f03f1194d0 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240afb3740 10 client.0 _ll_register_callbacks cb 0x55f03f005b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240695a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f11fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240695a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f11fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42206/0 (socket says 172.21.15.5:42206) 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f240695a640 1 -- 172.21.15.5:0/2188286810 learned_addr learned my addr 172.21.15.5:0/2188286810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.933+0000 7f2405958640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f119e80 0x55f03f11f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f2406159640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f122360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240695a640 1 -- 172.21.15.5:0/2188286810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 msgr2=0x55f03f122360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240695a640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f122360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240695a640 1 -- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f119e80 msgr2=0x55f03f11f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240695a640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f119e80 0x55f03f11f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240695a640 1 -- 172.21.15.5:0/2188286810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03ed903f0 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240695a640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f11fd00 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f23f800ff20 tx=0x7f23f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb9a67163d78595 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f23e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f23e9ffb640 1 -- 172.21.15.5:0/2188286810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f801e410 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f23e9ffb640 1 -- 172.21.15.5:0/2188286810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f23f801e5b0 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc09c100 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f23e9ffb640 1 -- 172.21.15.5:0/2188286810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23f801e7b0 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240afb3740 10 client.19032 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f03ee5a7d0 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.934+0000 7f23e9ffb640 1 -- 172.21.15.5:0/2188286810 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc09c100 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f23e9ffb640 1 -- 172.21.15.5:0/2188286810 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f23f801d070 con 0x55f03f122a00 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f240afb3740 10 client.19032 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f240afb3740 10 client.19032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f240afb3740 20 client.19032 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f240afb3740 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f132b70 0x55f03f134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f240afb3740 4 client.19032 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f03f01d930 con 0x55f03f132b70 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f2406159640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f132b70 0x55f03f134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f2406159640 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f132b70 0x55f03f134fb0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bddce25d69b21778 server_cookie=62e10307399922b1 in_seq=0 out_seq=0 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.935+0000 7f23e9ffb640 10 client.19032 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f23e9ffb640 1 -- 172.21.15.5:0/2188286810 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f03f01d930 con 0x55f03f132b70 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f23e9ffb640 10 client.19032 handle_command_reply: tid=0 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 client.19032 shutdown 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 msgr2=0x55f03f11fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f11fd00 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f23f800ff20 tx=0x7f23f8004ab0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 shutdown_connections 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f132b70 0x55f03f134fb0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f11fd00 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f119e80 0x55f03f11f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f23e9ffb640 0 client.19032 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 --2- 172.21.15.5:0/2188286810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f1194d0 0x55f03f122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 >> 172.21.15.5:0/2188286810 conn(0x55f03f113ba0 msgr2=0x55f03f1158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 shutdown_connections 2024-04-22T13:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 1 -- 172.21.15.5:0/2188286810 wait complete. 2024-04-22T13:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.936+0000 7f240afb3740 20 client.19032 trim_cache size 0 max 16384 2024-04-22T13:37:04.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 Processor -- start 2024-04-22T13:37:04.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 -- start start 2024-04-22T13:37:04.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 0x55f03f12e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f112a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f121e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24001afe80 con 0x55f03f1194d0 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f24001b19c0 con 0x55f03f122a00 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f240afb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24001b0550 con 0x55f03f119e80 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 0x55f03f12e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 0x55f03f12e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42214/0 (socket says 172.21.15.5:42214) 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 -- 172.21.15.5:0/1926050178 learned_addr learned my addr 172.21.15.5:0/1926050178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2405958640 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 -- 172.21.15.5:0/1926050178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 msgr2=0x55f03f112a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f112a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 -- 172.21.15.5:0/1926050178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 msgr2=0x55f03f121e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f121e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2406159640 1 -- 172.21.15.5:0/1926050178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03f11b2c0 con 0x55f03f1194d0 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.950+0000 7f2405958640 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f112a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f2406159640 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 0x55f03f12e800 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f23fc09be00 tx=0x7f23fc0d04a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99664129aae9bd09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f23ebfff640 1 -- 172.21.15.5:0/1926050178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0983a0 con 0x55f03f1194d0 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f23ebfff640 1 -- 172.21.15.5:0/1926050178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc09c800 con 0x55f03f1194d0 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f23ebfff640 1 -- 172.21.15.5:0/1926050178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc09a490 con 0x55f03f1194d0 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 -- 172.21.15.5:0/1926050178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 msgr2=0x55f03f12e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 0x55f03f12e800 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f23fc09be00 tx=0x7f23fc0d04a0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 -- 172.21.15.5:0/1926050178 shutdown_connections 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f122a00 0x55f03f121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f112a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 --2- 172.21.15.5:0/1926050178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f1194d0 0x55f03f12e800 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 -- 172.21.15.5:0/1926050178 >> 172.21.15.5:0/1926050178 conn(0x55f03f113ba0 msgr2=0x55f03f112370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 -- 172.21.15.5:0/1926050178 shutdown_connections 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.951+0000 7f240afb3740 1 -- 172.21.15.5:0/1926050178 wait complete. 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 -- start start 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 Processor -- start 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f1194d0 0x55f03f12b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f1263e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f126920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24001b12f0 con 0x55f03f122a00 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f24001b0c20 con 0x55f03f1194d0 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24001ade80 con 0x55f03f119e80 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240afb3740 10 client.0 _ll_register_callbacks cb 0x7f23f8009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f126920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f126920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42228/0 (socket says 172.21.15.5:42228) 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 -- 172.21.15.5:0/1670439842 learned_addr learned my addr 172.21.15.5:0/1670439842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f2405958640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f1263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 -- 172.21.15.5:0/1670439842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 msgr2=0x55f03f1263e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f1263e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f2406159640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f1194d0 0x55f03f12b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 -- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f1194d0 msgr2=0x55f03f12b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f1194d0 0x55f03f12b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 -- 172.21.15.5:0/1670439842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03f1270f0 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f240695a640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f126920 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f23f800fcc0 tx=0x7f23f802ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec6f1f99355f606 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.952+0000 7f23e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f23e9ffb640 1 -- 172.21.15.5:0/1670439842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f802a3f0 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f23e9ffb640 1 -- 172.21.15.5:0/1670439842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f23f802a590 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc09c800 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f23e9ffb640 1 -- 172.21.15.5:0/1670439842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23f8044050 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 10 client.19038 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f03f199050 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f23e9ffb640 1 -- 172.21.15.5:0/1670439842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc09c800 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f23e9ffb640 1 -- 172.21.15.5:0/1670439842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f23f803b070 con 0x55f03f122a00 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 10 client.19038 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 10 client.19038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 20 client.19038 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f199710 0x55f03f19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 4 client.19038 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f03f08bc10 con 0x55f03f199710 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.953+0000 7f2406159640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f199710 0x55f03f19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f2406159640 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f199710 0x55f03f19bb30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff407479c46626e server_cookie=4a602c48b6773958 in_seq=0 out_seq=0 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f23e9ffb640 10 client.19038 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f23e9ffb640 1 -- 172.21.15.5:0/1670439842 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f03f08bc10 con 0x55f03f199710 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f23e9ffb640 10 client.19038 handle_command_reply: tid=0 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 client.19038 shutdown 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 msgr2=0x55f03f126920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f126920 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f23f800fcc0 tx=0x7f23f802ad60 comp rx=0 tx=0).stop 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 shutdown_connections 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55f03f199710 0x55f03f19bb30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03f122a00 0x55f03f126920 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f03f119e80 0x55f03f1263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f23e9ffb640 0 client.19038 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:04.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 --2- 172.21.15.5:0/1670439842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f03f1194d0 0x55f03f12b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 >> 172.21.15.5:0/1670439842 conn(0x55f03f113ba0 msgr2=0x55f03f112c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 shutdown_connections 2024-04-22T13:37:04.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 1 -- 172.21.15.5:0/1670439842 wait complete. 2024-04-22T13:37:04.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.954+0000 7f240afb3740 20 client.19038 trim_cache size 0 max 16384 2024-04-22T13:37:04.956 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f23ec0a3d20 msgr2=0x7f23ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005fd0 tx=0x7f23f8009500 comp rx=0 tx=0).stop 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 msgr2=0x7f24000fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 0x7f24000fddf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00d950 tx=0x7f23fc00de10 comp rx=0 tx=0).stop 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 shutdown_connections 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2400103400 0x7f24000fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2400102a60 0x7f24000fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 --2- 172.21.15.5:0/1411656708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2400101180 0x7f24000fddf0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:04.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 >> 172.21.15.5:0/1411656708 conn(0x7f2400077f70 msgr2=0x7f240006aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:04.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 shutdown_connections 2024-04-22T13:37:04.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:04.955+0000 7f24083bd640 1 -- 172.21.15.5:0/1411656708 wait complete. 2024-04-22T13:37:05.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:05 smithi049 ceph-mon[31379]: pgmap v307: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:37:05.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:05 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/396797877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:37:05.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:05 smithi107 ceph-mon[31859]: pgmap v307: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:37:05.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:05 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/396797877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:37:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:05 smithi005 ceph-mon[28006]: pgmap v307: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:37:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:05 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/396797877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:37:05.962 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:37:05.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:37:05.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3321244154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 msgr2=0x7f3bd810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3321244154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd810e9a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009ca0 tx=0x7f3bd002c180 comp rx=0 tx=0).stop 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3321244154 shutdown_connections 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3321244154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd810e9a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3321244154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8103ae0 0x7f3bd810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3321244154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 0x7f3bd81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3321244154 >> 172.21.15.5:0/3321244154 conn(0x7f3bd8078000 msgr2=0x7f3bd8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3321244154 shutdown_connections 2024-04-22T13:37:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.087+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3321244154 wait complete. 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 Processor -- start 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 -- start start 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 0x7f3bd81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8103ae0 0x7f3bd8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd8112190 con 0x7f3bd810c500 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd8112010 con 0x7f3bd81031c0 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bdce1e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd8112310 con 0x7f3bd8103ae0 2024-04-22T13:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd819cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42250/0 (socket says 172.21.15.5:42250) 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bd6d9d640 1 -- 172.21.15.5:0/3254484273 learned_addr learned my addr 172.21.15.5:0/3254484273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8103ae0 0x7f3bd8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.088+0000 7f3bd659c640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 0x7f3bd81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd6d9d640 1 -- 172.21.15.5:0/3254484273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8103ae0 msgr2=0x7f3bd8198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd6d9d640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8103ae0 0x7f3bd8198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd6d9d640 1 -- 172.21.15.5:0/3254484273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 msgr2=0x7f3bd81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd6d9d640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 0x7f3bd81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd6d9d640 1 -- 172.21.15.5:0/3254484273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd819d6e0 con 0x7f3bd810c500 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd659c640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 0x7f3bd81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bd6d9d640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd819cfb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3bd002c790 tx=0x7f3bd0032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bcf7fe640 1 -- 172.21.15.5:0/3254484273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0039040 con 0x7f3bd810c500 2024-04-22T13:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bcf7fe640 1 -- 172.21.15.5:0/3254484273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd0044070 con 0x7f3bd810c500 2024-04-22T13:37:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd819d970 con 0x7f3bd810c500 2024-04-22T13:37:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bcf7fe640 1 -- 172.21.15.5:0/3254484273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0032850 con 0x7f3bd810c500 2024-04-22T13:37:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.089+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd81a5260 con 0x7f3bd810c500 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bcf7fe640 1 -- 172.21.15.5:0/3254484273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3bd00329f0 con 0x7f3bd810c500 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 Processor -- start 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 -- start start 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291537740 0x560291537b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x560291540c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x5602915416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56029149bf00 con 0x5602915380f0 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560291439590 con 0x560291541250 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560291497ec0 con 0x560291537740 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x560291540c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x560291540c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42254/0 (socket says 172.21.15.5:42254) 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bd659c640 1 -- 172.21.15.5:0/90816131 learned_addr learned my addr 172.21.15.5:0/90816131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.092+0000 7f3bcf7fe640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bc40a3d20 0x7f3bc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291537740 0x560291537b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd6d9d640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bc40a3d20 0x7f3bc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x5602915416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd659c640 1 -- 172.21.15.5:0/90816131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291537740 msgr2=0x560291537b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd659c640 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291537740 0x560291537b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd659c640 1 -- 172.21.15.5:0/90816131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 msgr2=0x5602915416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd659c640 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x5602915416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd659c640 1 -- 172.21.15.5:0/90816131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602912767e0 con 0x5602915380f0 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x5602915416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bcf7fe640 1 -- 172.21.15.5:0/3254484273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3bd00f2b70 con 0x7f3bd810c500 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd659c640 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x560291540c30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009d50 tx=0x7f3bc80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109bfbad1ee099b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bd6d9d640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bc40a3d20 0x7f3bc40a61e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00f0480 tx=0x7f3bd00baf90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bccff9640 1 -- 172.21.15.5:0/90816131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8012650 con 0x5602915380f0 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bccff9640 1 -- 172.21.15.5:0/90816131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc8009e40 con 0x5602915380f0 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.093+0000 7f3bccff9640 1 -- 172.21.15.5:0/90816131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800fb90 con 0x5602915380f0 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 -- 172.21.15.5:0/90816131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 msgr2=0x560291540c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x560291540c30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009d50 tx=0x7f3bc80117b0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 -- 172.21.15.5:0/90816131 shutdown_connections 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x560291540c30 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x5602915416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/90816131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291537740 0x560291537b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 -- 172.21.15.5:0/90816131 >> 172.21.15.5:0/90816131 conn(0x560291531fb0 msgr2=0x560291532390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 -- 172.21.15.5:0/90816131 shutdown_connections 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 -- 172.21.15.5:0/90816131 wait complete. 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 -- start start 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.094+0000 7f3bdfa05740 1 Processor -- start 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x560291544cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x5602915451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x560291545730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602915481d0 con 0x5602915380f0 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x560291455580 con 0x560291537740 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560291548350 con 0x560291541250 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bdfa05740 10 client.0 _ll_register_callbacks cb 0x56029145b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x5602915451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x5602915451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42270/0 (socket says 172.21.15.5:42270) 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 -- 172.21.15.5:0/2462594104 learned_addr learned my addr 172.21.15.5:0/2462594104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x560291545730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x560291544cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 -- 172.21.15.5:0/2462594104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 msgr2=0x560291545730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x560291545730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 -- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 msgr2=0x560291544cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x560291544cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 -- 172.21.15.5:0/2462594104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602912d6f70 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x560291545730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bd659c640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x5602915451f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3bc801c040 tx=0x7f3bc800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d16d518b7a5d421 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.095+0000 7f3bbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bbdffb640 1 -- 172.21.15.5:0/2462594104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8028020 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602914a9920 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bbdffb640 1 -- 172.21.15.5:0/2462594104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3bc800fce0 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bbdffb640 1 -- 172.21.15.5:0/2462594104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3bc8033050 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 10 client.19050 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56029116e170 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bbdffb640 1 -- 172.21.15.5:0/2462594104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5602914a9920 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bbdffb640 1 -- 172.21.15.5:0/2462594104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc80124e0 con 0x5602915380f0 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 10 client.19050 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 10 client.19050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 20 client.19050 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560291555630 0x560291557a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 4 client.19050 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.096+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602914a3790 con 0x560291555630 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.097+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560291555630 0x560291557a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.097+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560291555630 0x560291557a20 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be66261e4f0b70cf server_cookie=3abbf44ba8584184 in_seq=0 out_seq=0 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.097+0000 7f3bbdffb640 10 client.19050 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bbdffb640 1 -- 172.21.15.5:0/2462594104 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602914a3790 con 0x560291555630 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bbdffb640 10 client.19050 handle_command_reply: tid=0 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 client.19050 shutdown 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 msgr2=0x5602915451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x5602915451f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3bc801c040 tx=0x7f3bc800f670 comp rx=0 tx=0).stop 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 shutdown_connections 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x560291555630 0x560291557a20 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x5602915451f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bbdffb640 0 client.19050 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x560291545730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/2462594104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x560291544cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 >> 172.21.15.5:0/2462594104 conn(0x560291531fb0 msgr2=0x560291533d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 shutdown_connections 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 1 -- 172.21.15.5:0/2462594104 wait complete. 2024-04-22T13:37:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.098+0000 7f3bdfa05740 20 client.19050 trim_cache size 0 max 16384 2024-04-22T13:37:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.116+0000 7f3bdfa05740 1 Processor -- start 2024-04-22T13:37:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.116+0000 7f3bdfa05740 1 -- start start 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 0x5602915473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602915380f0 0x560291551100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x560291542640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd81ab9c0 con 0x560291537740 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd81ad500 con 0x560291541250 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd81ac090 con 0x5602915380f0 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 0x5602915473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 0x5602915473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42286/0 (socket says 172.21.15.5:42286) 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 -- 172.21.15.5:0/3884942004 learned_addr learned my addr 172.21.15.5:0/3884942004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 -- 172.21.15.5:0/3884942004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602915380f0 msgr2=0x560291551100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd659c640 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602915380f0 0x560291551100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602915380f0 0x560291551100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 -- 172.21.15.5:0/3884942004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 msgr2=0x560291542640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x560291542640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 -- 172.21.15.5:0/3884942004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560291537e90 con 0x560291537740 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd659c640 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602915380f0 0x560291551100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 0x5602915473e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3bd81008b0 tx=0x7f3bc0004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c6b68288d6e2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bbffff640 1 -- 172.21.15.5:0/3884942004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0005220 con 0x560291537740 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bbffff640 1 -- 172.21.15.5:0/3884942004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc0023070 con 0x560291537740 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.117+0000 7f3bbffff640 1 -- 172.21.15.5:0/3884942004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00038e0 con 0x560291537740 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 -- 172.21.15.5:0/3884942004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 msgr2=0x5602915473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 0x5602915473e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3bd81008b0 tx=0x7f3bc0004580 comp rx=0 tx=0).stop 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 -- 172.21.15.5:0/3884942004 shutdown_connections 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291541250 0x560291542640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602915380f0 0x560291551100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/3884942004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560291537740 0x5602915473e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 -- 172.21.15.5:0/3884942004 >> 172.21.15.5:0/3884942004 conn(0x560291531fb0 msgr2=0x560291530780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 -- 172.21.15.5:0/3884942004 shutdown_connections 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 -- 172.21.15.5:0/3884942004 wait complete. 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 -- start start 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.118+0000 7f3bdfa05740 1 Processor -- start 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x56029155c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x56029155cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x5602915ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd81ace30 con 0x5602915380f0 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd81ac760 con 0x560291537740 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd81a99c0 con 0x560291541250 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 10 client.0 _ll_register_callbacks cb 0x7f3bc8009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x56029155cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x56029155cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42302/0 (socket says 172.21.15.5:42302) 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 -- 172.21.15.5:0/1233029712 learned_addr learned my addr 172.21.15.5:0/1233029712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x56029155c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x5602915ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 -- 172.21.15.5:0/1233029712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 msgr2=0x5602915ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x5602915ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 -- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 msgr2=0x56029155c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x56029155c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 -- 172.21.15.5:0/1233029712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602915bae50 con 0x5602915380f0 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x5602915ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bd659c640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x56029155cf30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x560291544510 tx=0x7f3bc800f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc69ba77d1284c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bbdffb640 1 -- 172.21.15.5:0/1233029712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc803c060 con 0x5602915380f0 2024-04-22T13:37:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bbdffb640 1 -- 172.21.15.5:0/1233029712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3bc8012b70 con 0x5602915380f0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.119+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc0023070 con 0x5602915380f0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bbdffb640 1 -- 172.21.15.5:0/1233029712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3bc8028070 con 0x5602915380f0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 10 client.19056 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602915bbf60 con 0x5602915380f0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bbdffb640 1 -- 172.21.15.5:0/1233029712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc0023070 con 0x5602915380f0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bbdffb640 1 -- 172.21.15.5:0/1233029712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc8003790 con 0x5602915380f0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 10 client.19056 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 10 client.19056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 20 client.19056 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5602915bc6e0 0x5602915beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 4 client.19056 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56029141fc50 con 0x5602915bc6e0 2024-04-22T13:37:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.120+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5602915bc6e0 0x5602915beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.121+0000 7f3bbdffb640 10 client.19056 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.121+0000 7f3bd5d9b640 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5602915bc6e0 0x5602915beb00 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc6d79edbd11128 server_cookie=c0e210806fcd9b56 in_seq=0 out_seq=0 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bbdffb640 1 -- 172.21.15.5:0/1233029712 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56029141fc50 con 0x5602915bc6e0 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bbdffb640 10 client.19056 handle_command_reply: tid=0 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 client.19056 shutdown 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 msgr2=0x56029155cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x56029155cf30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x560291544510 tx=0x7f3bc800f6e0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 shutdown_connections 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5602915bc6e0 0x5602915beb00 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602915380f0 0x56029155cf30 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bbdffb640 0 client.19056 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560291541250 0x5602915ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 --2- 172.21.15.5:0/1233029712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560291537740 0x56029155c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 >> 172.21.15.5:0/1233029712 conn(0x560291531fb0 msgr2=0x560291531010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 shutdown_connections 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 1 -- 172.21.15.5:0/1233029712 wait complete. 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.122+0000 7f3bdfa05740 20 client.19056 trim_cache size 0 max 16384 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "dcbc3722-e886-44d1-8f4a-67ee2cfde450", 2024-04-22T13:37:06.124 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-22T13:37:06.125 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:06.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bc40a3d20 msgr2=0x7f3bc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bc40a3d20 0x7f3bc40a61e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00f0480 tx=0x7f3bd00baf90 comp rx=0 tx=0).stop 2024-04-22T13:37:06.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 msgr2=0x7f3bd819cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd819cfb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3bd002c790 tx=0x7f3bd0032430 comp rx=0 tx=0).stop 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 shutdown_connections 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bd810c500 0x7f3bd819cfb0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3bc40a3d20 0x7f3bc40a61e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8103ae0 0x7f3bd8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 --2- 172.21.15.5:0/3254484273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd81031c0 0x7f3bd81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 >> 172.21.15.5:0/3254484273 conn(0x7f3bd8078000 msgr2=0x7f3bd80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 shutdown_connections 2024-04-22T13:37:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.123+0000 7f3bdce1e640 1 -- 172.21.15.5:0/3254484273 wait complete. 2024-04-22T13:37:06.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:37:06.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:37:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.241+0000 7fb941e43640 1 -- 172.21.15.5:0/3538524867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c102160 msgr2=0x7fb93c102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.241+0000 7fb941e43640 1 --2- 172.21.15.5:0/3538524867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c102160 0x7fb93c102540 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009900 tx=0x7fb92c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 -- 172.21.15.5:0/3538524867 shutdown_connections 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 --2- 172.21.15.5:0/3538524867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb93c107230 0x7fb93c10ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 --2- 172.21.15.5:0/3538524867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102b10 0x7fb93c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 --2- 172.21.15.5:0/3538524867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c102160 0x7fb93c102540 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 -- 172.21.15.5:0/3538524867 >> 172.21.15.5:0/3538524867 conn(0x7fb93c0fd9a0 msgr2=0x7fb93c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 -- 172.21.15.5:0/3538524867 shutdown_connections 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 -- 172.21.15.5:0/3538524867 wait complete. 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 Processor -- start 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.242+0000 7fb941e43640 1 -- start start 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb941e43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102160 0x7fb93c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb941e43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 0x7fb93c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb941e43640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb941e43640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c112190 con 0x7fb93c107230 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb941e43640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c112010 con 0x7fb93c074cd0 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb941e43640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c112310 con 0x7fb93c102160 2024-04-22T13:37:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 0x7fb93c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 0x7fb93c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42332/0 (socket says 172.21.15.5:42332) 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 -- 172.21.15.5:0/386180961 learned_addr learned my addr 172.21.15.5:0/386180961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93bfff640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102160 0x7fb93c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 -- 172.21.15.5:0/386180961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102160 msgr2=0x7fb93c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102160 0x7fb93c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 -- 172.21.15.5:0/386180961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb93c074cd0 msgr2=0x7fb93c1a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 -- 172.21.15.5:0/386180961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c1a6fc0 con 0x7fb93c107230 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb93affd640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 0x7fb93c074790 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fb92400d740 tx=0x7fb92400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.243+0000 7fb938ff9640 1 -- 172.21.15.5:0/386180961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924017070 con 0x7fb93c107230 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.244+0000 7fb938ff9640 1 -- 172.21.15.5:0/386180961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92400de50 con 0x7fb93c107230 2024-04-22T13:37:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.244+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c1a72b0 con 0x7fb93c107230 2024-04-22T13:37:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.244+0000 7fb938ff9640 1 -- 172.21.15.5:0/386180961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924004ce0 con 0x7fb93c107230 2024-04-22T13:37:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.244+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c1a7800 con 0x7fb93c107230 2024-04-22T13:37:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb938ff9640 1 -- 172.21.15.5:0/386180961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb924004e80 con 0x7fb93c107230 2024-04-22T13:37:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 Processor -- start 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 -- start start 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c777092960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c77709b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c77709de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c776ff71e0 con 0x55c77709ba80 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c776f94390 con 0x55c777092580 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb944a34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c776ff31a0 con 0x55c777092ea0 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c77709de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c77709de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42344/0 (socket says 172.21.15.5:42344) 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb93affd640 1 -- 172.21.15.5:0/3705261578 learned_addr learned my addr 172.21.15.5:0/3705261578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.246+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c77709b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93bfff640 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c777092960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb938ff9640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb9280a3da0 0x7fb9280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93affd640 1 -- 172.21.15.5:0/3705261578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 msgr2=0x55c77709b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93affd640 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c77709b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93affd640 1 -- 172.21.15.5:0/3705261578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 msgr2=0x55c777092960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93affd640 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c777092960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93affd640 1 -- 172.21.15.5:0/3705261578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c776dd37d0 con 0x55c77709ba80 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93bfff640 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c777092960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93bfff640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb9280a3da0 0x7fb9280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb938ff9640 1 -- 172.21.15.5:0/386180961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7fb9240d0150 con 0x7fb93c107230 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93affd640 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c77709de70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cd7f0 tx=0x7fb9240982d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79131cb74739f7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb931ffb640 1 -- 172.21.15.5:0/3705261578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924020070 con 0x55c77709ba80 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb931ffb640 1 -- 172.21.15.5:0/3705261578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92409c7d0 con 0x55c77709ba80 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb931ffb640 1 -- 172.21.15.5:0/3705261578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92409c0d0 con 0x55c77709ba80 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.247+0000 7fb93bfff640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb9280a3da0 0x7fb9280a6260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb934007f80 tx=0x7fb934004a40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 -- 172.21.15.5:0/3705261578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 msgr2=0x55c77709de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c77709de70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cd7f0 tx=0x7fb9240982d0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 -- 172.21.15.5:0/3705261578 shutdown_connections 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c77709de70 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c77709b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 --2- 172.21.15.5:0/3705261578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c777092960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 -- 172.21.15.5:0/3705261578 >> 172.21.15.5:0/3705261578 conn(0x55c77708ccb0 msgr2=0x55c77708d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 -- 172.21.15.5:0/3705261578 shutdown_connections 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 -- 172.21.15.5:0/3705261578 wait complete. 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.248+0000 7fb944a34740 1 -- start start 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 Processor -- start 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c777098af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c777099030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c77709e7b0 con 0x55c777092ea0 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c776fb02f0 con 0x55c77709ba80 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c77709e930 con 0x55c777092580 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb944a34740 10 client.0 _ll_register_callbacks cb 0x55c776fbd580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c777099030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c777099030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42348/0 (socket says 172.21.15.5:42348) 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 -- 172.21.15.5:0/874687086 learned_addr learned my addr 172.21.15.5:0/874687086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93bfff640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c777098af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 -- 172.21.15.5:0/874687086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 msgr2=0x55c777098af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c777098af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 -- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 msgr2=0x55c777099570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 -- 172.21.15.5:0/874687086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c776d093f0 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb93affd640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c777099030 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cde80 tx=0x7fb924009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c437f6c75b92292 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.249+0000 7fb91affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb91affd640 1 -- 172.21.15.5:0/874687086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924020070 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb91affd640 1 -- 172.21.15.5:0/874687086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7fb9240a0170 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb92409c7d0 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb91affd640 1 -- 172.21.15.5:0/874687086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb92409bea0 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 10 client.19068 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c776cc90f0 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb91affd640 1 -- 172.21.15.5:0/874687086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92409c7d0 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb91affd640 1 -- 172.21.15.5:0/874687086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb924098740 con 0x55c777092ea0 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 10 client.19068 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 10 client.19068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 20 client.19068 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7770ab890 0x55c7770adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 4 client.19068 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c777004c00 con 0x55c7770ab890 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.250+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7770ab890 0x55c7770adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.251+0000 7fb91affd640 10 client.19068 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.251+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7770ab890 0x55c7770adc80 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c34be2984b3f36 server_cookie=238ab0cb3e8fa36f in_seq=0 out_seq=0 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb91affd640 1 -- 172.21.15.5:0/874687086 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c777004c00 con 0x55c7770ab890 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb91affd640 10 client.19068 handle_command_reply: tid=0 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 client.19068 shutdown 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 msgr2=0x55c777099030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c777099030 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cde80 tx=0x7fb924009e60 comp rx=0 tx=0).stop 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 shutdown_connections 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7770ab890 0x55c7770adc80 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c777099030 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb91affd640 0 client.19068 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 --2- 172.21.15.5:0/874687086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c777098af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 >> 172.21.15.5:0/874687086 conn(0x55c77708ccb0 msgr2=0x55c77708ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 shutdown_connections 2024-04-22T13:37:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 1 -- 172.21.15.5:0/874687086 wait complete. 2024-04-22T13:37:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.252+0000 7fb944a34740 20 client.19068 trim_cache size 0 max 16384 2024-04-22T13:37:06.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.269+0000 7fb944a34740 1 Processor -- start 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.269+0000 7fb944a34740 1 -- start start 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb944a34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c77709b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb944a34740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c77708e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb944a34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb944a34740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1afdb0 con 0x55c777092ea0 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb944a34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b18f0 con 0x55c77709ba80 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb944a34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b0480 con 0x55c777092580 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c77708e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c77708e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42350/0 (socket says 172.21.15.5:42350) 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 -- 172.21.15.5:0/2522322599 learned_addr learned my addr 172.21.15.5:0/2522322599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c77709b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93bfff640 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 -- 172.21.15.5:0/2522322599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 msgr2=0x55c77709b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c77709b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 -- 172.21.15.5:0/2522322599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 msgr2=0x55c777099390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 -- 172.21.15.5:0/2522322599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77709de70 con 0x55c777092ea0 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb93affd640 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c77708e990 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb924098530 tx=0x7fb92409b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71a0b3061442897 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb9317fa640 1 -- 172.21.15.5:0/2522322599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924020070 con 0x55c777092ea0 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb9317fa640 1 -- 172.21.15.5:0/2522322599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92409c7d0 con 0x55c777092ea0 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.270+0000 7fb9317fa640 1 -- 172.21.15.5:0/2522322599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240950f0 con 0x55c777092ea0 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 -- 172.21.15.5:0/2522322599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 msgr2=0x55c77708e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c77708e990 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb924098530 tx=0x7fb92409b410 comp rx=0 tx=0).stop 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 -- 172.21.15.5:0/2522322599 shutdown_connections 2024-04-22T13:37:06.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c777092ea0 0x55c77708e990 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c77709ba80 0x55c777099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 --2- 172.21.15.5:0/2522322599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092580 0x55c77709b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 -- 172.21.15.5:0/2522322599 >> 172.21.15.5:0/2522322599 conn(0x55c77708ccb0 msgr2=0x55c777097a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 -- 172.21.15.5:0/2522322599 shutdown_connections 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 -- 172.21.15.5:0/2522322599 wait complete. 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.271+0000 7fb944a34740 1 -- start start 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 Processor -- start 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c7770ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c7770aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c7770af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b1220 con 0x55c77709ba80 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b0b50 con 0x55c777092580 2024-04-22T13:37:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1addb0 con 0x55c777092ea0 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb944a34740 10 client.0 _ll_register_callbacks cb 0x7fb92c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c7770af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c7770ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c7770af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42362/0 (socket says 172.21.15.5:42362) 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 -- 172.21.15.5:0/3405904666 learned_addr learned my addr 172.21.15.5:0/3405904666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93affd640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c7770aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 -- 172.21.15.5:0/3405904666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 msgr2=0x55c7770aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c7770aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 -- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 msgr2=0x55c7770ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c7770ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 -- 172.21.15.5:0/3405904666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c777110a70 con 0x55c77709ba80 2024-04-22T13:37:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c7770ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb93bfff640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c7770af0e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb93401eef0 tx=0x7fb934004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc462338d0615ce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.272+0000 7fb91affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb91affd640 1 -- 172.21.15.5:0/3405904666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93402d060 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb91affd640 1 -- 172.21.15.5:0/3405904666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7fb93401d880 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb92409c7d0 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb91affd640 1 -- 172.21.15.5:0/3405904666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb93401da80 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 10 client.19074 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c777111cf0 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb91affd640 1 -- 172.21.15.5:0/3405904666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92409c7d0 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb91affd640 1 -- 172.21.15.5:0/3405904666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb934027030 con 0x55c77709ba80 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 10 client.19074 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 10 client.19074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 20 client.19074 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7771124a0 0x55c7771148c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 4 client.19074 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.273+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c776ffea70 con 0x55c7771124a0 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.274+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7771124a0 0x55c7771148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.274+0000 7fb93b7fe640 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7771124a0 0x55c7771148c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe2f4801abb40aa server_cookie=cc07f1c60b079702 in_seq=0 out_seq=0 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.274+0000 7fb91affd640 10 client.19074 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.274+0000 7fb91affd640 1 -- 172.21.15.5:0/3405904666 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c776ffea70 con 0x55c7771124a0 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.274+0000 7fb91affd640 10 client.19074 handle_command_reply: tid=0 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.274+0000 7fb944a34740 1 client.19074 shutdown 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 msgr2=0x55c7770af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c7770af0e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb93401eef0 tx=0x7fb934004ab0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 shutdown_connections 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55c7771124a0 0x55c7771148c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c77709ba80 0x55c7770af0e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c777092ea0 0x55c7770aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb91affd640 0 client.19074 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 --2- 172.21.15.5:0/3405904666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c777092580 0x55c7770ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 >> 172.21.15.5:0/3405904666 conn(0x55c77708ccb0 msgr2=0x55c7770a7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 shutdown_connections 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 1 -- 172.21.15.5:0/3405904666 wait complete. 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb944a34740 20 client.19074 trim_cache size 0 max 16384 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:06.277 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: "dcbc3722-e886-44d1-8f4a-67ee2cfde450": { 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "dcbc3722-e886-44d1-8f4a-67ee2cfde450", 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb9280a3da0 msgr2=0x7fb9280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb9280a3da0 0x7fb9280a6260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb934007f80 tx=0x7fb934004a40 comp rx=0 tx=0).stop 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 msgr2=0x7fb93c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 0x7fb93c074790 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fb92400d740 tx=0x7fb92400dc00 comp rx=0 tx=0).stop 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 shutdown_connections 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93c107230 0x7fb93c074790 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fb9280a3da0 0x7fb9280a6260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 --2- 172.21.15.5:0/386180961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102160 0x7fb93c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.275+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 >> 172.21.15.5:0/386180961 conn(0x7fb93c0fd9a0 msgr2=0x7fb93c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.276+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 shutdown_connections 2024-04-22T13:37:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:06.276+0000 7fb941e43640 1 -- 172.21.15.5:0/386180961 wait complete. 2024-04-22T13:37:06.285 INFO:tasks.cephfs.filesystem:scrub status for tag:dcbc3722-e886-44d1-8f4a-67ee2cfde450 - {'path': '/', 'tag': 'dcbc3722-e886-44d1-8f4a-67ee2cfde450', 'options': 'recursive,force'} 2024-04-22T13:37:07.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:07 smithi049 ceph-mon[31379]: pgmap v308: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:37:07.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:07 smithi049 ceph-mon[31379]: scrub queued for path: / 2024-04-22T13:37:07.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:07 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:37:07.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:07 smithi049 ceph-mon[31379]: scrub summary: active paths [/] 2024-04-22T13:37:07.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:07 smithi049 ceph-mon[31379]: scrub summary: idle+waiting paths [/] 2024-04-22T13:37:07.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:07 smithi049 ceph-mon[31379]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T13:37:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:07 smithi107 ceph-mon[31859]: pgmap v308: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:37:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:07 smithi107 ceph-mon[31859]: scrub queued for path: / 2024-04-22T13:37:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:07 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:37:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:07 smithi107 ceph-mon[31859]: scrub summary: active paths [/] 2024-04-22T13:37:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:07 smithi107 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-04-22T13:37:07.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:07 smithi107 ceph-mon[31859]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T13:37:07.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:07 smithi005 ceph-mon[28006]: pgmap v308: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:37:07.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:07 smithi005 ceph-mon[28006]: scrub queued for path: / 2024-04-22T13:37:07.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:07 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:37:07.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:07 smithi005 ceph-mon[28006]: scrub summary: active paths [/] 2024-04-22T13:37:07.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:07 smithi005 ceph-mon[28006]: scrub summary: idle+waiting paths [/] 2024-04-22T13:37:07.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:07 smithi005 ceph-mon[28006]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T13:37:09.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:09 smithi049 ceph-mon[31379]: pgmap v310: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:37:09.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:09 smithi107 ceph-mon[31859]: pgmap v310: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:37:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:09 smithi005 ceph-mon[28006]: pgmap v310: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:37:11.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:11 smithi049 ceph-mon[31379]: pgmap v311: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:37:11.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:11 smithi049 ceph-mon[31379]: scrub summary: idle 2024-04-22T13:37:11.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:11 smithi107 ceph-mon[31859]: pgmap v311: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:37:11.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:11 smithi107 ceph-mon[31859]: scrub summary: idle 2024-04-22T13:37:11.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:11 smithi005 ceph-mon[28006]: pgmap v311: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:37:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:11 smithi005 ceph-mon[28006]: scrub summary: idle 2024-04-22T13:37:13.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:13 smithi049 ceph-mon[31379]: pgmap v312: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:13.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:13 smithi049 ceph-mon[31379]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T13:37:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:13 smithi107 ceph-mon[31859]: pgmap v312: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:13.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:13 smithi107 ceph-mon[31859]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T13:37:13.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:13 smithi005 ceph-mon[28006]: pgmap v312: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:13.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:13 smithi005 ceph-mon[28006]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T13:37:15.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:15 smithi049 ceph-mon[31379]: pgmap v314: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-22T13:37:15.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:15 smithi107 ceph-mon[31859]: pgmap v314: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-22T13:37:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:15 smithi005 ceph-mon[28006]: pgmap v314: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-22T13:37:17.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:17 smithi049 ceph-mon[31379]: pgmap v315: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 216 B/s wr, 12 op/s 2024-04-22T13:37:17.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:17 smithi107 ceph-mon[31859]: pgmap v315: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 216 B/s wr, 12 op/s 2024-04-22T13:37:17.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:17 smithi005 ceph-mon[28006]: pgmap v315: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 216 B/s wr, 12 op/s 2024-04-22T13:37:18.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 -- 172.21.15.5:0/3813163625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f0106fb0 msgr2=0x7fd2f0107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 --2- 172.21.15.5:0/3813163625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f0106fb0 0x7fd2f0107390 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009930 tx=0x7fd2ec02bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 -- 172.21.15.5:0/3813163625 shutdown_connections 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 --2- 172.21.15.5:0/3813163625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01011d0 0x7fd2f010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 --2- 172.21.15.5:0/3813163625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0100810 0x7fd2f0100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 --2- 172.21.15.5:0/3813163625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f0106fb0 0x7fd2f0107390 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 -- 172.21.15.5:0/3813163625 >> 172.21.15.5:0/3813163625 conn(0x7fd2f0076190 msgr2=0x7fd2f00765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 -- 172.21.15.5:0/3813163625 shutdown_connections 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.964+0000 7fd2f7566640 1 -- 172.21.15.5:0/3813163625 wait complete. 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 Processor -- start 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 -- start start 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0100810 0x7fd2f0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 0x7fd2f0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0106fb0 0x7fd2f019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0112240 con 0x7fd2f01011d0 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01120c0 con 0x7fd2f0106fb0 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f7566640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01123c0 con 0x7fd2f0100810 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 0x7fd2f0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 0x7fd2f0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40132/0 (socket says 172.21.15.5:40132) 2024-04-22T13:37:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 -- 172.21.15.5:0/343661488 learned_addr learned my addr 172.21.15.5:0/343661488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f5302640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0100810 0x7fd2f0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f5b03640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0106fb0 0x7fd2f019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 -- 172.21.15.5:0/343661488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0100810 msgr2=0x7fd2f0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0100810 0x7fd2f0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 -- 172.21.15.5:0/343661488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0106fb0 msgr2=0x7fd2f019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0106fb0 0x7fd2f019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.965+0000 7fd2f4b01640 1 -- 172.21.15.5:0/343661488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f019d750 con 0x7fd2f01011d0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.966+0000 7fd2f4b01640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 0x7fd2f0198ca0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000e9f0 tx=0x7fd2e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.966+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000cd80 con 0x7fd2f01011d0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.966+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e00108e0 con 0x7fd2f01011d0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.966+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f019da40 con 0x7fd2f01011d0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.966+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0018980 con 0x7fd2f01011d0 2024-04-22T13:37:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.966+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f01a52d0 con 0x7fd2f01011d0 2024-04-22T13:37:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.967+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f0050530 con 0x7fd2f01011d0 2024-04-22T13:37:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.968+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd2e0010430 con 0x7fd2f01011d0 2024-04-22T13:37:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.969+0000 7fd2e67fc640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2dc0a3d10 0x7fd2dc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.969+0000 7fd2f5302640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2dc0a3d10 0x7fd2dc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:18.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.969+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fd2e00d0b70 con 0x7fd2f01011d0 2024-04-22T13:37:18.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.969+0000 7fd2f5302640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2dc0a3d10 0x7fd2dc0a61d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec005ab0 tx=0x7fd2ec005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:18.974+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2e009d210 con 0x7fd2f01011d0 2024-04-22T13:37:19.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.188+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd2f010a500 con 0x7fd2dc0a3d10 2024-04-22T13:37:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.197+0000 7fd2e67fc640 1 -- 172.21.15.5:0/343661488 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fd2ec0040c0 con 0x7fd2dc0a3d10 2024-04-22T13:37:19.193 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-22T00:00:00", "created": "2024-04-22T13:27:17", "first": "2024-04-22T13:28:00", "last": "2024-04-22T13:37:00", "last_pruned": "2024-04-22T13:37:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-22T13:37:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.199+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2dc0a3d10 msgr2=0x7fd2dc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.199+0000 7fd2f7566640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2dc0a3d10 0x7fd2dc0a61d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec005ab0 tx=0x7fd2ec005fd0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.199+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 msgr2=0x7fd2f0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.199+0000 7fd2f7566640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 0x7fd2f0198ca0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000e9f0 tx=0x7fd2e000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 shutdown_connections 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f01011d0 0x7fd2f0198ca0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd2dc0a3d10 0x7fd2dc0a61d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2f0106fb0 0x7fd2f019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 --2- 172.21.15.5:0/343661488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0100810 0x7fd2f0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 >> 172.21.15.5:0/343661488 conn(0x7fd2f0076190 msgr2=0x7fd2f00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 shutdown_connections 2024-04-22T13:37:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.200+0000 7fd2f7566640 1 -- 172.21.15.5:0/343661488 wait complete. 2024-04-22T13:37:19.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-22T13:37:19.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 -- 172.21.15.5:0/340020975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f1010c590 msgr2=0x7f0f1010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 --2- 172.21.15.5:0/340020975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f1010c590 0x7f0f1010e9c0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009ca0 tx=0x7f0f0802c180 comp rx=0 tx=0).stop 2024-04-22T13:37:19.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 -- 172.21.15.5:0/340020975 shutdown_connections 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 --2- 172.21.15.5:0/340020975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f1010c590 0x7f0f1010e9c0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 --2- 172.21.15.5:0/340020975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f10103ae0 0x7f0f1010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 --2- 172.21.15.5:0/340020975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f101031c0 0x7f0f101035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 -- 172.21.15.5:0/340020975 >> 172.21.15.5:0/340020975 conn(0x7f0f10077f70 msgr2=0x7f0f10078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 -- 172.21.15.5:0/340020975 shutdown_connections 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 -- 172.21.15.5:0/340020975 wait complete. 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 Processor -- start 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 -- start start 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 0x7f0f1019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.315+0000 7f0f17184640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f10103ae0 0x7f0f1019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f17184640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1010c590 0x7f0f101985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f17184640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10112230 con 0x7f0f101031c0 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f17184640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101120b0 con 0x7f0f10103ae0 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f17184640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101123b0 con 0x7f0f1010c590 2024-04-22T13:37:19.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 0x7f0f1019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 0x7f0f1019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40152/0 (socket says 172.21.15.5:40152) 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 -- 172.21.15.5:0/1314988034 learned_addr learned my addr 172.21.15.5:0/1314988034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f0ffff640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f10103ae0 0x7f0f1019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f15721640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1010c590 0x7f0f101985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 -- 172.21.15.5:0/1314988034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1010c590 msgr2=0x7f0f101985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1010c590 0x7f0f101985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 -- 172.21.15.5:0/1314988034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f10103ae0 msgr2=0x7f0f1019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f10103ae0 0x7f0f1019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 -- 172.21.15.5:0/1314988034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f10198d70 con 0x7f0f101031c0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f15721640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1010c590 0x7f0f101985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f14f20640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 0x7f0f1019e3f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d770 tx=0x7f0ef800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017070 con 0x7f0f101031c0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f10199060 con 0x7f0f101031c0 2024-04-22T13:37:19.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef8009820 con 0x7f0f101031c0 2024-04-22T13:37:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.316+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004bc0 con 0x7f0f101031c0 2024-04-22T13:37:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.317+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a52b0 con 0x7f0f101031c0 2024-04-22T13:37:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.317+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f10050530 con 0x7f0f101031c0 2024-04-22T13:37:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.318+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0ef80099d0 con 0x7f0f101031c0 2024-04-22T13:37:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.319+0000 7f0f0dffb640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0efc0a3d10 0x7f0efc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.319+0000 7f0f0ffff640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0efc0a3d10 0x7f0efc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.319+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f0ef80d02d0 con 0x7f0f101031c0 2024-04-22T13:37:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.320+0000 7f0f0ffff640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0efc0a3d10 0x7f0efc0a61d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f1010e460 tx=0x7f0f00009110 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:19.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.323+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ef809c970 con 0x7f0f101031c0 2024-04-22T13:37:19.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:19 smithi049 ceph-mon[31379]: pgmap v316: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.533+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 --> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0f101076f0 con 0x7f0efc0a3d10 2024-04-22T13:37:19.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:19 smithi107 ceph-mon[31859]: pgmap v316: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.541+0000 7f0f0dffb640 1 -- 172.21.15.5:0/1314988034 <== mgr.15354 v2:172.21.15.5:6800/751347710 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0f000095e0 con 0x7f0efc0a3d10 2024-04-22T13:37:19.536 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-04-22T13:37:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.542+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0efc0a3d10 msgr2=0x7f0efc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.542+0000 7f0f17184640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0efc0a3d10 0x7f0efc0a61d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f1010e460 tx=0x7f0f00009110 comp rx=0 tx=0).stop 2024-04-22T13:37:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.542+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 msgr2=0x7f0f1019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.542+0000 7f0f17184640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 0x7f0f1019e3f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d770 tx=0x7f0ef800db70 comp rx=0 tx=0).stop 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 shutdown_connections 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f0efc0a3d10 0x7f0efc0a61d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1010c590 0x7f0f101985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f10103ae0 0x7f0f1019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 --2- 172.21.15.5:0/1314988034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f101031c0 0x7f0f1019e3f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 >> 172.21.15.5:0/1314988034 conn(0x7f0f10077f70 msgr2=0x7f0f100fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 shutdown_connections 2024-04-22T13:37:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:19.543+0000 7f0f17184640 1 -- 172.21.15.5:0/1314988034 wait complete. 2024-04-22T13:37:19.548 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:19 smithi005 ceph-mon[28006]: pgmap v316: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:19.549 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-22T13:37:19.560 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-22T13:37:20.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:20 smithi049 ceph-mon[31379]: from='client.19080 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:37:20.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:20 smithi107 ceph-mon[31859]: from='client.19080 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:37:20.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:20 smithi005 ceph-mon[28006]: from='client.19080 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:37:21.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:21 smithi049 ceph-mon[31379]: from='client.19086 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:37:21.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:21 smithi049 ceph-mon[31379]: pgmap v317: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:21.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:21 smithi107 ceph-mon[31859]: from='client.19086 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:37:21.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:21 smithi107 ceph-mon[31859]: pgmap v317: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:21 smithi005 ceph-mon[28006]: from='client.19086 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:37:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:21 smithi005 ceph-mon[28006]: pgmap v317: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:37:23.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:23 smithi049 ceph-mon[31379]: pgmap v318: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:37:23.533 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:23 smithi107 ceph-mon[31859]: pgmap v318: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:37:23.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:23 smithi005 ceph-mon[28006]: pgmap v318: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:37:24.284 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:24 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:37:24.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:24 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:37:24.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:24 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:37:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: pgmap v319: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 91 B/s wr, 11 op/s 2024-04-22T13:37:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:25 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: pgmap v319: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 91 B/s wr, 11 op/s 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:25 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: pgmap v319: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 91 B/s wr, 11 op/s 2024-04-22T13:37:25.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:25.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:25 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:26.520 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-22T13:37:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-22T13:37:26.521 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: pgmap v320: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: Unable to set osd_memory_target on smithi005 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-22T13:37:26.522 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:37:26.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:37:26.523 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:26 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-22T13:37:26.534 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: pgmap v320: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: Unable to set osd_memory_target on smithi005 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:37:26.535 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:37:26.536 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:26 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: pgmap v320: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: Unable to set osd_memory_target on smithi005 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:37:26.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:26 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' 2024-04-22T13:37:29.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:28 smithi049 ceph-mon[31379]: pgmap v321: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T13:37:29.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:28 smithi107 ceph-mon[31859]: pgmap v321: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T13:37:29.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:28 smithi005 ceph-mon[28006]: pgmap v321: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T13:37:31.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:30 smithi049 ceph-mon[31379]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T13:37:31.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:30 smithi107 ceph-mon[31859]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T13:37:31.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:30 smithi005 ceph-mon[28006]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T13:37:33.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:32 smithi049 ceph-mon[31379]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-22T13:37:33.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:32 smithi107 ceph-mon[31859]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-22T13:37:33.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:32 smithi005 ceph-mon[28006]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-22T13:37:35.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:34 smithi049 ceph-mon[31379]: pgmap v324: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:35.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:34 smithi107 ceph-mon[31859]: pgmap v324: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:35.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:34 smithi005 ceph-mon[28006]: pgmap v324: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:36.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:37:36.286 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:37:36.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 -- 172.21.15.5:0/2627661809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074ca0 msgr2=0x7f815010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 --2- 172.21.15.5:0/2627661809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074ca0 0x7f815010e230 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8140009900 tx=0x7f814002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 -- 172.21.15.5:0/2627661809 shutdown_connections 2024-04-22T13:37:36.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 --2- 172.21.15.5:0/2627661809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074ca0 0x7f815010e230 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 --2- 172.21.15.5:0/2627661809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815010e770 0x7f8150110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 --2- 172.21.15.5:0/2627661809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8150074380 0x7f8150074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 -- 172.21.15.5:0/2627661809 >> 172.21.15.5:0/2627661809 conn(0x7f81500fda20 msgr2=0x7f81500ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 -- 172.21.15.5:0/2627661809 shutdown_connections 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 -- 172.21.15.5:0/2627661809 wait complete. 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 Processor -- start 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.410+0000 7f8157985640 1 -- start start 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 0x7f81501a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8150074ca0 0x7f81501a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815010e770 0x7f815019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8150114390 con 0x7f8150074380 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8150114210 con 0x7f8150074ca0 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8150114510 con 0x7f815010e770 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 0x7f81501a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 0x7f81501a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44552/0 (socket says 172.21.15.5:44552) 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 -- 172.21.15.5:0/3215578279 learned_addr learned my addr 172.21.15.5:0/3215578279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8154f20640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8150074ca0 0x7f81501a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155f22640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815010e770 0x7f815019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 -- 172.21.15.5:0/3215578279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815010e770 msgr2=0x7f815019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815010e770 0x7f815019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 -- 172.21.15.5:0/3215578279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8150074ca0 msgr2=0x7f81501a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8150074ca0 0x7f81501a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 -- 172.21.15.5:0/3215578279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815019afd0 con 0x7f8150074380 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8155721640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 0x7f81501a0600 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d950 tx=0x7f814c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f81467fc640 1 -- 172.21.15.5:0/3215578279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c017070 con 0x7f8150074380 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815019b2c0 con 0x7f8150074380 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f81467fc640 1 -- 172.21.15.5:0/3215578279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814c009c30 con 0x7f8150074380 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.411+0000 7f81467fc640 1 -- 172.21.15.5:0/3215578279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00f620 con 0x7f8150074380 2024-04-22T13:37:36.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.412+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501a7580 con 0x7f8150074380 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.414+0000 7f81467fc640 1 -- 172.21.15.5:0/3215578279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f814c010460 con 0x7f8150074380 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.414+0000 7f815a575740 1 Processor -- start 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.414+0000 7f815a575740 1 -- start start 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f815a575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x558281443960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f815a575740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x55828144c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f81467fc640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f813c0a3d20 0x7f813c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f815a575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828144ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f815a575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5582813a81e0 con 0x558281443580 2024-04-22T13:37:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f815a575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x558281345390 con 0x55828144ca80 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f815a575740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5582813a41a0 con 0x558281443ea0 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x558281443960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8154f20640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f813c0a3d20 0x7f813c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x558281443960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44560/0 (socket says 172.21.15.5:44560) 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 -- 172.21.15.5:0/946315216 learned_addr learned my addr 172.21.15.5:0/946315216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f81467fc640 1 -- 172.21.15.5:0/3215578279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f814c0d0460 con 0x7f8150074380 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155721640 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x55828144c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828144ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 -- 172.21.15.5:0/946315216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 msgr2=0x55828144c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x55828144c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 -- 172.21.15.5:0/946315216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 msgr2=0x55828144ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828144ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 -- 172.21.15.5:0/946315216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582811847d0 con 0x558281443580 2024-04-22T13:37:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8155f22640 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x558281443960 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8148012a90 tx=0x7f81480075b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef365bebf6ef745 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.415+0000 7f8154f20640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f813c0a3d20 0x7f813c0a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8140006010 tx=0x7f8140005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f813b7fe640 1 -- 172.21.15.5:0/946315216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148004d20 con 0x558281443580 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f813b7fe640 1 -- 172.21.15.5:0/946315216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8148007af0 con 0x558281443580 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f813b7fe640 1 -- 172.21.15.5:0/946315216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148005810 con 0x558281443580 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 -- 172.21.15.5:0/946315216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 msgr2=0x558281443960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x558281443960 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8148012a90 tx=0x7f81480075b0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 -- 172.21.15.5:0/946315216 shutdown_connections 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828144ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x55828144c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 --2- 172.21.15.5:0/946315216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x558281443960 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 -- 172.21.15.5:0/946315216 >> 172.21.15.5:0/946315216 conn(0x55828143dcb0 msgr2=0x55828143e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 -- 172.21.15.5:0/946315216 shutdown_connections 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.416+0000 7f815a575740 1 -- 172.21.15.5:0/946315216 wait complete. 2024-04-22T13:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 -- start start 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 Processor -- start 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281449b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 0x55828144a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828144ca80 0x55828144a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55828144f870 con 0x558281443ea0 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5582813612f0 con 0x558281443580 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55828144f9f0 con 0x55828144ca80 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f815a575740 10 client.0 _ll_register_callbacks cb 0x55828136e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f8154f20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 0x55828144a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f8154f20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 0x55828144a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44576/0 (socket says 172.21.15.5:44576) 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.417+0000 7f8154f20640 1 -- 172.21.15.5:0/322359917 learned_addr learned my addr 172.21.15.5:0/322359917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8155721640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281449b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8155f22640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828144ca80 0x55828144a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8154f20640 1 -- 172.21.15.5:0/322359917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828144ca80 msgr2=0x55828144a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8154f20640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828144ca80 0x55828144a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8154f20640 1 -- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 msgr2=0x558281449b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8154f20640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281449b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8154f20640 1 -- 172.21.15.5:0/322359917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582810ba3f0 con 0x558281443ea0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8155f22640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828144ca80 0x55828144a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8154f20640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 0x55828144a080 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8140009900 tx=0x7f8140009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300989d8444c56a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8138ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8138ff9640 1 -- 172.21.15.5:0/322359917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140047040 con 0x558281443ea0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8138ff9640 1 -- 172.21.15.5:0/322359917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f8140005520 con 0x558281443ea0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8148007af0 con 0x558281443ea0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8138ff9640 1 -- 172.21.15.5:0/322359917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81400046c0 con 0x558281443ea0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f815a575740 10 client.19098 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55828107a0f0 con 0x558281443ea0 2024-04-22T13:37:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.418+0000 7f8138ff9640 1 -- 172.21.15.5:0/322359917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8148007af0 con 0x558281443ea0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f8138ff9640 1 -- 172.21.15.5:0/322359917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8140053030 con 0x558281443ea0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f815a575740 10 client.19098 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f815a575740 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f815a575740 20 client.19098 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f815a575740 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55828145cba0 0x55828145f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f815a575740 4 client.19098 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582813b5c00 con 0x55828145cba0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.419+0000 7f8155721640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55828145cba0 0x55828145f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f8138ff9640 10 client.19098 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f8155721640 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55828145cba0 0x55828145f000 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a272eaafc4f310 server_cookie=1b78154be39c69cc in_seq=0 out_seq=0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f8138ff9640 1 -- 172.21.15.5:0/322359917 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582813b5c00 con 0x55828145cba0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f8138ff9640 10 client.19098 handle_command_reply: tid=0 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 client.19098 shutdown 2024-04-22T13:37:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 msgr2=0x55828144a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 0x55828144a080 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8140009900 tx=0x7f8140009260 comp rx=0 tx=0).stop 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 shutdown_connections 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x55828145cba0 0x55828145f000 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443ea0 0x55828144a080 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828144ca80 0x55828144a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 --2- 172.21.15.5:0/322359917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281449b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f8138ff9640 0 client.19098 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.420+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 >> 172.21.15.5:0/322359917 conn(0x55828143dcb0 msgr2=0x55828143fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.421+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 shutdown_connections 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.421+0000 7f815a575740 1 -- 172.21.15.5:0/322359917 wait complete. 2024-04-22T13:37:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.421+0000 7f815a575740 20 client.19098 trim_cache size 0 max 16384 2024-04-22T13:37:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.438+0000 7f815a575740 1 Processor -- start 2024-04-22T13:37:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.438+0000 7f815a575740 1 -- start start 2024-04-22T13:37:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f815a575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x55828144c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f815a575740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x558281458730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f815a575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828143cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f815a575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81501afde0 con 0x558281443580 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f815a575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b1920 con 0x55828144ca80 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f815a575740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b04b0 con 0x558281443ea0 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x55828144c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x55828144c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44578/0 (socket says 172.21.15.5:44578) 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 -- 172.21.15.5:0/840731146 learned_addr learned my addr 172.21.15.5:0/840731146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 -- 172.21.15.5:0/840731146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 msgr2=0x558281458730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x558281458730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 -- 172.21.15.5:0/840731146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 msgr2=0x55828143cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828143cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 -- 172.21.15.5:0/840731146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558281442900 con 0x558281443580 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f8155721640 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x55828144c270 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f814c0cdc60 tx=0x7f814c09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff2c6dc17f2d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f813affd640 1 -- 172.21.15.5:0/840731146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c020040 con 0x558281443580 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f813affd640 1 -- 172.21.15.5:0/840731146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814c09c400 con 0x558281443580 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.439+0000 7f813affd640 1 -- 172.21.15.5:0/840731146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c0baaf0 con 0x558281443580 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 -- 172.21.15.5:0/840731146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 msgr2=0x55828144c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x55828144c270 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f814c0cdc60 tx=0x7f814c09f040 comp rx=0 tx=0).stop 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 -- 172.21.15.5:0/840731146 shutdown_connections 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55828144ca80 0x55828143cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x558281458730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 --2- 172.21.15.5:0/840731146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558281443580 0x55828144c270 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 -- 172.21.15.5:0/840731146 >> 172.21.15.5:0/840731146 conn(0x55828143dcb0 msgr2=0x55828143c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 -- 172.21.15.5:0/840731146 shutdown_connections 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 -- 172.21.15.5:0/840731146 wait complete. 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 -- start start 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.440+0000 7f815a575740 1 Processor -- start 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281446ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x5582814473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 0x5582814c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b1250 con 0x55828144ca80 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b0b80 con 0x558281443580 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81501adde0 con 0x558281443ea0 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f815a575740 10 client.0 _ll_register_callbacks cb 0x7f8140044040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 0x5582814c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 0x5582814c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44594/0 (socket says 172.21.15.5:44594) 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 -- 172.21.15.5:0/2021896400 learned_addr learned my addr 172.21.15.5:0/2021896400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155721640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281446ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8154f20640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x5582814473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 -- 172.21.15.5:0/2021896400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 msgr2=0x5582814473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x5582814473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 -- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 msgr2=0x558281446ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281446ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 -- 172.21.15.5:0/2021896400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582814c2680 con 0x55828144ca80 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155721640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281446ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8154f20640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x5582814473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8155f22640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 0x5582814c2140 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8148012a90 tx=0x7f81480124a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9331066022715a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8138ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.441+0000 7f8138ff9640 1 -- 172.21.15.5:0/2021896400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148003b70 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f8138ff9640 1 -- 172.21.15.5:0/2021896400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f8148003d10 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c09c400 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f8138ff9640 1 -- 172.21.15.5:0/2021896400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8148007400 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 10 client.19104 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582814c38f0 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f8138ff9640 1 -- 172.21.15.5:0/2021896400 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814c09c400 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f8138ff9640 1 -- 172.21.15.5:0/2021896400 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8148025070 con 0x55828144ca80 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 10 client.19104 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 10 client.19104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 20 client.19104 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5582814c3f10 0x5582814c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 4 client.19104 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.442+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582813afa70 con 0x5582814c3f10 2024-04-22T13:37:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f8155721640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5582814c3f10 0x5582814c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f8138ff9640 10 client.19104 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f8155721640 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5582814c3f10 0x5582814c6330 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf17ba350128573c server_cookie=f58a6e90d09bf6ea in_seq=0 out_seq=0 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f8138ff9640 1 -- 172.21.15.5:0/2021896400 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582813afa70 con 0x5582814c3f10 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f8138ff9640 10 client.19104 handle_command_reply: tid=0 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 client.19104 shutdown 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 msgr2=0x5582814c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 0x5582814c2140 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8148012a90 tx=0x7f81480124a0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 shutdown_connections 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x5582814c3f10 0x5582814c6330 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55828144ca80 0x5582814c2140 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558281443ea0 0x5582814473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f8138ff9640 0 client.19104 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 --2- 172.21.15.5:0/2021896400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558281443580 0x558281446ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 >> 172.21.15.5:0/2021896400 conn(0x55828143dcb0 msgr2=0x55828143c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 shutdown_connections 2024-04-22T13:37:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.443+0000 7f815a575740 1 -- 172.21.15.5:0/2021896400 wait complete. 2024-04-22T13:37:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f815a575740 20 client.19104 trim_cache size 0 max 16384 2024-04-22T13:37:36.449 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:36.449 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-22T13:37:36.449 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-22T13:37:36.449 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f813c0a3d20 msgr2=0x7f813c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f813c0a3d20 0x7f813c0a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8140006010 tx=0x7f8140005e40 comp rx=0 tx=0).stop 2024-04-22T13:37:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 msgr2=0x7f81501a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 0x7f81501a0600 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d950 tx=0x7f814c00de10 comp rx=0 tx=0).stop 2024-04-22T13:37:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 shutdown_connections 2024-04-22T13:37:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f813c0a3d20 0x7f813c0a61e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815010e770 0x7f815019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8150074ca0 0x7f81501a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 --2- 172.21.15.5:0/3215578279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150074380 0x7f81501a0600 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 >> 172.21.15.5:0/3215578279 conn(0x7f81500fda20 msgr2=0x7f81500ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 shutdown_connections 2024-04-22T13:37:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.444+0000 7f8157985640 1 -- 172.21.15.5:0/3215578279 wait complete. 2024-04-22T13:37:36.452 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:37:36.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:37:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/1178749747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf28106160 msgr2=0x7fdf2810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/1178749747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf28106160 0x7fdf2810dd30 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009ca0 tx=0x7fdf2002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/1178749747 shutdown_connections 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/1178749747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf28106160 0x7fdf2810dd30 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/1178749747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf28101a40 0x7fdf28105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/1178749747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf28101090 0x7fdf28101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/1178749747 >> 172.21.15.5:0/1178749747 conn(0x7fdf280771d0 msgr2=0x7fdf280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/1178749747 shutdown_connections 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.557+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/1178749747 wait complete. 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.558+0000 7fdf2d1ef640 1 Processor -- start 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.558+0000 7fdf2d1ef640 1 -- start start 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf28101090 0x7fdf28198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf28106160 0x7fdf28198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 0x7fdf2819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281121e0 con 0x7fdf2819cec0 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28112060 con 0x7fdf28101090 2024-04-22T13:37:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28112360 con 0x7fdf28106160 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 0x7fdf2819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 0x7fdf2819d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44598/0 (socket says 172.21.15.5:44598) 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 -- 172.21.15.5:0/2319307358 learned_addr learned my addr 172.21.15.5:0/2319307358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2659c640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf28106160 0x7fdf28198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf26d9d640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf28101090 0x7fdf28198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 -- 172.21.15.5:0/2319307358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf28106160 msgr2=0x7fdf28198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf28106160 0x7fdf28198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 -- 172.21.15.5:0/2319307358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf28101090 msgr2=0x7fdf28198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf28101090 0x7fdf28198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 -- 172.21.15.5:0/2319307358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2819d9b0 con 0x7fdf2819cec0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2759e640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 0x7fdf2819d370 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005e30 tx=0x7fdf20034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20039040 con 0x7fdf2819cec0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2819dc40 con 0x7fdf2819cec0 2024-04-22T13:37:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf20044070 con 0x7fdf2819cec0 2024-04-22T13:37:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20004040 con 0x7fdf2819cec0 2024-04-22T13:37:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.559+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281a5460 con 0x7fdf2819cec0 2024-04-22T13:37:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.560+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf28102560 con 0x7fdf2819cec0 2024-04-22T13:37:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.561+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fdf200324c0 con 0x7fdf2819cec0 2024-04-22T13:37:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.562+0000 7fdf0ffff640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdf100a3d20 0x7fdf100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.562+0000 7fdf26d9d640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdf100a3d20 0x7fdf100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.562+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fdf200f28c0 con 0x7fdf2819cec0 2024-04-22T13:37:36.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.563+0000 7fdf26d9d640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdf100a3d20 0x7fdf100a61e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800abe0 tx=0x7fdf1800a2d0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.567+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf200bf070 con 0x7fdf2819cec0 2024-04-22T13:37:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.791+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf28199950 con 0x7fdf2819cec0 2024-04-22T13:37:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.792+0000 7fdf0ffff640 1 -- 172.21.15.5:0/2319307358 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7fdf200be970 con 0x7fdf2819cec0 2024-04-22T13:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:37:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdf100a3d20 msgr2=0x7fdf100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdf100a3d20 0x7fdf100a61e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800abe0 tx=0x7fdf1800a2d0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 msgr2=0x7fdf2819d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 0x7fdf2819d370 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005e30 tx=0x7fdf20034760 comp rx=0 tx=0).stop 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 shutdown_connections 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf2819cec0 0x7fdf2819d370 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdf100a3d20 0x7fdf100a61e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf28106160 0x7fdf28198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 --2- 172.21.15.5:0/2319307358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf28101090 0x7fdf28198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 >> 172.21.15.5:0/2319307358 conn(0x7fdf280771d0 msgr2=0x7fdf280fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 shutdown_connections 2024-04-22T13:37:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.794+0000 7fdf2d1ef640 1 -- 172.21.15.5:0/2319307358 wait complete. 2024-04-22T13:37:36.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:37:36.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 -- 172.21.15.5:0/3245360472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 msgr2=0x7fde4410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 --2- 172.21.15.5:0/3245360472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde4410eef0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009cd0 tx=0x7fde3c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 -- 172.21.15.5:0/3245360472 shutdown_connections 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 --2- 172.21.15.5:0/3245360472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde4410eef0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 --2- 172.21.15.5:0/3245360472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde44103b50 0x7fde44103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 --2- 172.21.15.5:0/3245360472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde44102270 0x7fde44102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 -- 172.21.15.5:0/3245360472 >> 172.21.15.5:0/3245360472 conn(0x7fde440fda10 msgr2=0x7fde440ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 -- 172.21.15.5:0/3245360472 shutdown_connections 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 -- 172.21.15.5:0/3245360472 wait complete. 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.910+0000 7fde4974a640 1 Processor -- start 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 -- start start 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde44102270 0x7fde4419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde44103b50 0x7fde4419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde441985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde44112290 con 0x7fde441044f0 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde44112110 con 0x7fde44102270 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde4974a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde44112410 con 0x7fde44103b50 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde441985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde441985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44614/0 (socket says 172.21.15.5:44614) 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 -- 172.21.15.5:0/1321458874 learned_addr learned my addr 172.21.15.5:0/1321458874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde42ffd640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde44102270 0x7fde4419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde427fc640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde44103b50 0x7fde4419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 -- 172.21.15.5:0/1321458874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde44103b50 msgr2=0x7fde4419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde44103b50 0x7fde4419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 -- 172.21.15.5:0/1321458874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde44102270 msgr2=0x7fde4419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde44102270 0x7fde4419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 -- 172.21.15.5:0/1321458874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde44198d70 con 0x7fde441044f0 2024-04-22T13:37:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.911+0000 7fde437fe640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde441985f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098d0 tx=0x7fde3c032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.912+0000 7fde2bfff640 1 -- 172.21.15.5:0/1321458874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c041020 con 0x7fde441044f0 2024-04-22T13:37:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.912+0000 7fde2bfff640 1 -- 172.21.15.5:0/1321458874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde3c032970 con 0x7fde441044f0 2024-04-22T13:37:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.912+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde44199000 con 0x7fde441044f0 2024-04-22T13:37:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.912+0000 7fde2bfff640 1 -- 172.21.15.5:0/1321458874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c039040 con 0x7fde441044f0 2024-04-22T13:37:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.912+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441a5300 con 0x7fde441044f0 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde2bfff640 1 -- 172.21.15.5:0/1321458874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fde3c044020 con 0x7fde441044f0 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 Processor -- start 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 -- start start 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904b7e80 0x55c2904c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904c0a00 0x55c2904c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c29041c1f0 con 0x55c2904b74d0 2024-04-22T13:37:36.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c2903b94b0 con 0x55c2904c0a00 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.914+0000 7fde4c40e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c2904181b0 con 0x55c2904b7e80 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44628/0 (socket says 172.21.15.5:44628) 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde427fc640 1 -- 172.21.15.5:0/3864024222 learned_addr learned my addr 172.21.15.5:0/3864024222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904b7e80 0x55c2904c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde427fc640 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904c0a00 0x55c2904c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde2bfff640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 -- 172.21.15.5:0/3864024222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904c0a00 msgr2=0x55c2904c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904c0a00 0x55c2904c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 -- 172.21.15.5:0/3864024222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 msgr2=0x55c2904b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 -- 172.21.15.5:0/3864024222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2903ab1b0 con 0x55c2904b7e80 2024-04-22T13:37:36.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde437fe640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde2bfff640 1 -- 172.21.15.5:0/1321458874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fde3c0f2480 con 0x7fde441044f0 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde42ffd640 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904b7e80 0x55c2904c0430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde34004630 tx=0x7fde3400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda7c45a2bfbe164 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde437fe640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0efd90 tx=0x7fde3c02e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde297fa640 1 -- 172.21.15.5:0/3864024222 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34003bd0 con 0x55c2904b7e80 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde297fa640 1 -- 172.21.15.5:0/3864024222 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde34015070 con 0x55c2904b7e80 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.915+0000 7fde297fa640 1 -- 172.21.15.5:0/3864024222 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34004c60 con 0x55c2904b7e80 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 -- 172.21.15.5:0/3864024222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904b7e80 msgr2=0x55c2904c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904b7e80 0x55c2904c0430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde34004630 tx=0x7fde3400bb20 comp rx=0 tx=0).stop 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 -- 172.21.15.5:0/3864024222 shutdown_connections 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904c0a00 0x55c2904c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904b7e80 0x55c2904c0430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 --2- 172.21.15.5:0/3864024222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 -- 172.21.15.5:0/3864024222 >> 172.21.15.5:0/3864024222 conn(0x55c2904b1ba0 msgr2=0x55c2904b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 -- 172.21.15.5:0/3864024222 shutdown_connections 2024-04-22T13:37:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 -- 172.21.15.5:0/3864024222 wait complete. 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.916+0000 7fde4c40e740 1 -- start start 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 Processor -- start 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c2904c3970 con 0x55c2904b74d0 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55c2903d5500 con 0x55c2904b7e80 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c2904c3af0 con 0x55c2904c0a00 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde4c40e740 10 client.0 _ll_register_callbacks cb 0x55c2903a3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44638/0 (socket says 172.21.15.5:44638) 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 -- 172.21.15.5:0/453766107 learned_addr learned my addr 172.21.15.5:0/453766107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde42ffd640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 -- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 msgr2=0x55c2904bdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904bdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 -- 172.21.15.5:0/453766107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 msgr2=0x55c2904bd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 -- 172.21.15.5:0/453766107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29012e3f0 con 0x55c2904b74d0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde42ffd640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde427fc640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c0290 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fde38004650 tx=0x7fde38005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a630f75519de8a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.917+0000 7fde26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde26ff5640 1 -- 172.21.15.5:0/453766107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38013a20 con 0x55c2904b74d0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde26ff5640 1 -- 172.21.15.5:0/453766107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fde38013bc0 con 0x55c2904b74d0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde34015070 con 0x55c2904b74d0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde26ff5640 1 -- 172.21.15.5:0/453766107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde38029050 con 0x55c2904b74d0 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 10 client.19122 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2901f87d0 con 0x55c2904b74d0 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde26ff5640 1 -- 172.21.15.5:0/453766107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde34015070 con 0x55c2904b74d0 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde26ff5640 1 -- 172.21.15.5:0/453766107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fde3800f420 con 0x55c2904b74d0 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 10 client.19122 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 10 client.19122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 20 client.19122 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c2904d0b00 0x55c2904d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 4 client.19122 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2903bb930 con 0x55c2904d0b00 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.918+0000 7fde42ffd640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c2904d0b00 0x55c2904d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.919+0000 7fde42ffd640 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c2904d0b00 0x55c2904d2f10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=faa4bd1f40699cd7 server_cookie=b2fd0dd8bb3ed084 in_seq=0 out_seq=0 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.919+0000 7fde26ff5640 10 client.19122 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde26ff5640 1 -- 172.21.15.5:0/453766107 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2903bb930 con 0x55c2904d0b00 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde26ff5640 10 client.19122 handle_command_reply: tid=0 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 client.19122 shutdown 2024-04-22T13:37:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 msgr2=0x55c2904c0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c0290 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fde38004650 tx=0x7fde38005f90 comp rx=0 tx=0).stop 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 shutdown_connections 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904bdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c2904d0b00 0x55c2904d2f10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 --2- 172.21.15.5:0/453766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c0290 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde26ff5640 0 client.19122 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 >> 172.21.15.5:0/453766107 conn(0x55c2904b1ba0 msgr2=0x55c2904b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 shutdown_connections 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 1 -- 172.21.15.5:0/453766107 wait complete. 2024-04-22T13:37:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.920+0000 7fde4c40e740 20 client.19122 trim_cache size 0 max 16384 2024-04-22T13:37:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 Processor -- start 2024-04-22T13:37:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 -- start start 2024-04-22T13:37:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde441adb50 con 0x55c2904b74d0 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde441af690 con 0x55c2904b7e80 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde4c40e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde441ae220 con 0x55c2904c0a00 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44652/0 (socket says 172.21.15.5:44652) 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.933+0000 7fde427fc640 1 -- 172.21.15.5:0/1557603392 learned_addr learned my addr 172.21.15.5:0/1557603392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde42ffd640 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde427fc640 1 -- 172.21.15.5:0/1557603392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 msgr2=0x55c2904c0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde427fc640 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde427fc640 1 -- 172.21.15.5:0/1557603392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 msgr2=0x55c2904bfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde427fc640 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde427fc640 1 -- 172.21.15.5:0/1557603392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2904b67f0 con 0x55c2904b74d0 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde427fc640 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904cc860 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fde380092e0 tx=0x7fde38009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818098e6bd6c7682 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde28ff9640 1 -- 172.21.15.5:0/1557603392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38026020 con 0x55c2904b74d0 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde28ff9640 1 -- 172.21.15.5:0/1557603392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde38003f90 con 0x55c2904b74d0 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde28ff9640 1 -- 172.21.15.5:0/1557603392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3800fc40 con 0x55c2904b74d0 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde4c40e740 1 -- 172.21.15.5:0/1557603392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 msgr2=0x55c2904cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.934+0000 7fde4c40e740 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904cc860 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fde380092e0 tx=0x7fde38009ce0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- 172.21.15.5:0/1557603392 shutdown_connections 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904c0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904bfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 --2- 172.21.15.5:0/1557603392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904cc860 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- 172.21.15.5:0/1557603392 >> 172.21.15.5:0/1557603392 conn(0x55c2904b1ba0 msgr2=0x55c2904b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- 172.21.15.5:0/1557603392 shutdown_connections 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- 172.21.15.5:0/1557603392 wait complete. 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- start start 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 Processor -- start 2024-04-22T13:37:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde441aefc0 con 0x55c2904b74d0 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fde441ae8f0 con 0x55c2904b7e80 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde441abb50 con 0x55c2904c0a00 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde4c40e740 10 client.0 _ll_register_callbacks cb 0x7fde38004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44662/0 (socket says 172.21.15.5:44662) 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde427fc640 1 -- 172.21.15.5:0/2905601214 learned_addr learned my addr 172.21.15.5:0/2905601214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde42ffd640 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde427fc640 1 -- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 msgr2=0x55c2904c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.935+0000 7fde427fc640 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde427fc640 1 -- 172.21.15.5:0/2905601214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 msgr2=0x55c2904c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde427fc640 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde427fc640 1 -- 172.21.15.5:0/2905601214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2904c4640 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde427fc640 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c8a10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fde3800f600 tx=0x7fde38013300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b885cb9e4de1c1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde26ff5640 1 -- 172.21.15.5:0/2905601214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3803a060 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde38003f90 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde26ff5640 1 -- 172.21.15.5:0/2905601214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fde38013a20 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde26ff5640 1 -- 172.21.15.5:0/2905601214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde38026030 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 10 client.19128 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c290536ab0 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde26ff5640 1 -- 172.21.15.5:0/2905601214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde38003f90 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde26ff5640 1 -- 172.21.15.5:0/2905601214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fde38004690 con 0x55c2904b74d0 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 10 client.19128 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 10 client.19128 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-22T13:37:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 20 client.19128 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c290537150 0x55c290539570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 4 client.19128 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.936+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 --> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c290429c10 con 0x55c290537150 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde42ffd640 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c290537150 0x55c290539570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde42ffd640 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c290537150 0x55c290539570 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b84bc6740c87ed5 server_cookie=4e0ade369edb75e7 in_seq=0 out_seq=0 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde26ff5640 10 client.19128 ms_handle_connect on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde26ff5640 1 -- 172.21.15.5:0/2905601214 <== mds.4 v2:172.21.15.107:6836/207360580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c290429c10 con 0x55c290537150 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde26ff5640 10 client.19128 handle_command_reply: tid=0 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 client.19128 shutdown 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 msgr2=0x55c2904c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c8a10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fde3800f600 tx=0x7fde38013300 comp rx=0 tx=0).stop 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 shutdown_connections 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c2904c0a00 0x55c2904c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c2904b7e80 0x55c2904c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.107:6836/207360580,v1:172.21.15.107:6837/207360580] conn(0x55c290537150 0x55c290539570 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 --2- 172.21.15.5:0/2905601214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2904b74d0 0x55c2904c8a10 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde26ff5640 0 client.19128 ms_handle_reset on v2:172.21.15.107:6836/207360580 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.937+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 >> 172.21.15.5:0/2905601214 conn(0x55c2904b1ba0 msgr2=0x55c2904b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 shutdown_connections 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4c40e740 1 -- 172.21.15.5:0/2905601214 wait complete. 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4c40e740 20 client.19128 trim_cache size 0 max 16384 2024-04-22T13:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fde2c0a3d20 msgr2=0x7fde2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0efd90 tx=0x7fde3c02e040 comp rx=0 tx=0).stop 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 msgr2=0x7fde441985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde441985f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098d0 tx=0x7fde3c032420 comp rx=0 tx=0).stop 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 shutdown_connections 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde441044f0 0x7fde441985f0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde44103b50 0x7fde4419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 --2- 172.21.15.5:0/1321458874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde44102270 0x7fde4419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 >> 172.21.15.5:0/1321458874 conn(0x7fde440fda10 msgr2=0x7fde440fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 shutdown_connections 2024-04-22T13:37:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:36.938+0000 7fde4974a640 1 -- 172.21.15.5:0/1321458874 wait complete. 2024-04-22T13:37:36.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:36 smithi005 ceph-mon[28006]: pgmap v325: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:36.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:37:37.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:36 smithi049 ceph-mon[31379]: pgmap v325: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:37.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:36 smithi107 ceph-mon[31859]: pgmap v325: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 -- 172.21.15.5:0/682529533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948103b50 msgr2=0x7f6948103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 --2- 172.21.15.5:0/682529533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948103b50 0x7f6948103fb0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6938009930 tx=0x7f693802bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 -- 172.21.15.5:0/682529533 shutdown_connections 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 --2- 172.21.15.5:0/682529533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948103b50 0x7f6948103fb0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 --2- 172.21.15.5:0/682529533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69481044f0 0x7f694810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 --2- 172.21.15.5:0/682529533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6948102270 0x7f6948102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 -- 172.21.15.5:0/682529533 >> 172.21.15.5:0/682529533 conn(0x7f69480fda10 msgr2=0x7f69480ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 -- 172.21.15.5:0/682529533 shutdown_connections 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.054+0000 7f695054a640 1 -- 172.21.15.5:0/682529533 wait complete. 2024-04-22T13:37:37.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 Processor -- start 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 -- start start 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 0x7f694819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6948103b50 0x7f694819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69481044f0 0x7f69481985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6948112290 con 0x7f6948102270 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6948112110 con 0x7f69481044f0 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f695054a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6948112410 con 0x7f6948103b50 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 0x7f694819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 0x7f694819e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44674/0 (socket says 172.21.15.5:44674) 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 -- 172.21.15.5:0/2435326657 learned_addr learned my addr 172.21.15.5:0/2435326657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694eae7640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69481044f0 0x7f69481985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694dae5640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6948103b50 0x7f694819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 -- 172.21.15.5:0/2435326657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6948103b50 msgr2=0x7f694819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6948103b50 0x7f694819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 -- 172.21.15.5:0/2435326657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69481044f0 msgr2=0x7f69481985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69481044f0 0x7f69481985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 -- 172.21.15.5:0/2435326657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6948198d70 con 0x7f6948102270 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.055+0000 7f694e2e6640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 0x7f694819e440 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f694400e9f0 tx=0x7f694400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.056+0000 7f693f7fe640 1 -- 172.21.15.5:0/2435326657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694400cd80 con 0x7f6948102270 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.056+0000 7f693f7fe640 1 -- 172.21.15.5:0/2435326657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69440108e0 con 0x7f6948102270 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.056+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6948199060 con 0x7f6948102270 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.056+0000 7f693f7fe640 1 -- 172.21.15.5:0/2435326657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944018980 con 0x7f6948102270 2024-04-22T13:37:37.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.056+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481a5300 con 0x7f6948102270 2024-04-22T13:37:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.058+0000 7f695320e740 1 Processor -- start 2024-04-22T13:37:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.058+0000 7f693f7fe640 1 -- 172.21.15.5:0/2435326657 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6944010430 con 0x7f6948102270 2024-04-22T13:37:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.058+0000 7f695320e740 1 -- start start 2024-04-22T13:37:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f695320e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed4d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f695320e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4de80 0x5595aed56430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f695320e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f695320e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595aecb21f0 con 0x5595aed56a00 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f695320e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5595aec4f4b0 con 0x5595aed4d4d0 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f695320e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595aecae1b0 con 0x5595aed4de80 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed4d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694dae5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed4d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:37968/0 (socket says 172.21.15.5:37968) 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 -- 172.21.15.5:0/1816119080 learned_addr learned my addr 172.21.15.5:0/1816119080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694e2e6640 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4de80 0x5595aed56430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 -- 172.21.15.5:0/1816119080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4de80 msgr2=0x5595aed56430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4de80 0x5595aed56430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 -- 172.21.15.5:0/1816119080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 msgr2=0x5595aed58e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed58e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 -- 172.21.15.5:0/1816119080 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595aec411b0 con 0x5595aed4d4d0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f693f7fe640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f69340a3d20 0x7f69340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694eae7640 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed4d8b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6940010370 tx=0x7f6940010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8064f17ec4ff57c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f694e2e6640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f69340a3d20 0x7f69340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f693f7fe640 1 -- 172.21.15.5:0/2435326657 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f69440d0ae0 con 0x7f6948102270 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.059+0000 7f693cbfe640 1 -- 172.21.15.5:0/1816119080 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940016070 con 0x5595aed4d4d0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f693cbfe640 1 -- 172.21.15.5:0/1816119080 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6940004d20 con 0x5595aed4d4d0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f693cbfe640 1 -- 172.21.15.5:0/1816119080 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940005420 con 0x5595aed4d4d0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 -- 172.21.15.5:0/1816119080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 msgr2=0x5595aed4d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f694e2e6640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f69340a3d20 0x7f69340a61e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f69440ce2e0 tx=0x7f6944018420 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed4d8b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6940010370 tx=0x7f6940010830 comp rx=0 tx=0).stop 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 -- 172.21.15.5:0/1816119080 shutdown_connections 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4de80 0x5595aed56430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 --2- 172.21.15.5:0/1816119080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed4d8b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 -- 172.21.15.5:0/1816119080 >> 172.21.15.5:0/1816119080 conn(0x5595aed47ba0 msgr2=0x5595aed47f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 -- 172.21.15.5:0/1816119080 shutdown_connections 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.060+0000 7f695320e740 1 -- 172.21.15.5:0/1816119080 wait complete. 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 -- start start 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 Processor -- start 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 0x5595aed53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed53c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595aed59970 con 0x5595aed4de80 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5595aec6b500 con 0x5595aed4d4d0 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595aed59af0 con 0x5595aed56a00 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f695320e740 10 client.0 _ll_register_callbacks cb 0x5595aec39b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 0x5595aed53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694eae7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 0x5595aed53680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44704/0 (socket says 172.21.15.5:44704) 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 -- 172.21.15.5:0/2312274425 learned_addr learned my addr 172.21.15.5:0/2312274425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694eae7640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed53c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 -- 172.21.15.5:0/2312274425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 msgr2=0x5595aed53c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed53c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 -- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 msgr2=0x5595aed562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694dae5640 1 -- 172.21.15.5:0/2312274425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ae9c43f0 con 0x5595aed4de80 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.061+0000 7f694eae7640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f694dae5640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 0x5595aed53680 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6938009900 tx=0x7f693802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dadd0db264930d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f6931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f6931ffb640 1 -- 172.21.15.5:0/2312274425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693802c5d0 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f6931ffb640 1 -- 172.21.15.5:0/2312274425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f6938005030 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6940004d20 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f6931ffb640 1 -- 172.21.15.5:0/2312274425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6938005230 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 10 client.19140 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595aea8e7d0 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f6931ffb640 1 -- 172.21.15.5:0/2312274425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6940004d20 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f6931ffb640 1 -- 172.21.15.5:0/2312274425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f69380035f0 con 0x5595aed4de80 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 10 client.19140 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 10 client.19140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 20 client.19140 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aed66b70 0x5595aed68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 4 client.19140 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.062+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595aec51930 con 0x5595aed66b70 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.063+0000 7f694eae7640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aed66b70 0x5595aed68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.063+0000 7f694eae7640 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aed66b70 0x5595aed68ff0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65ce14916219d595 server_cookie=9d8553c3c269efc7 in_seq=0 out_seq=0 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.063+0000 7f6931ffb640 10 client.19140 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f6931ffb640 1 -- 172.21.15.5:0/2312274425 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595aec51930 con 0x5595aed66b70 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f6931ffb640 10 client.19140 handle_command_reply: tid=0 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 client.19140 shutdown 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 msgr2=0x5595aed53680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 0x5595aed53680 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6938009900 tx=0x7f693802ca10 comp rx=0 tx=0).stop 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 shutdown_connections 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aed66b70 0x5595aed68ff0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4de80 0x5595aed53680 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed53c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 --2- 172.21.15.5:0/2312274425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4d4d0 0x5595aed562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f6931ffb640 0 client.19140 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 >> 172.21.15.5:0/2312274425 conn(0x5595aed47ba0 msgr2=0x5595aed498f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 shutdown_connections 2024-04-22T13:37:37.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 1 -- 172.21.15.5:0/2312274425 wait complete. 2024-04-22T13:37:37.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.064+0000 7f695320e740 20 client.19140 trim_cache size 0 max 16384 2024-04-22T13:37:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 Processor -- start 2024-04-22T13:37:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 -- start start 2024-04-22T13:37:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4d4d0 0x5595aed62830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed46a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69481adbc0 con 0x5595aed56a00 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69481af700 con 0x5595aed4de80 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.077+0000 7f695320e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69481ae290 con 0x5595aed4d4d0 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed46a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed46a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44706/0 (socket says 172.21.15.5:44706) 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 -- 172.21.15.5:0/3422662986 learned_addr learned my addr 172.21.15.5:0/3422662986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694dae5640 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4d4d0 0x5595aed62830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 -- 172.21.15.5:0/3422662986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4d4d0 msgr2=0x5595aed62830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4d4d0 0x5595aed62830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 -- 172.21.15.5:0/3422662986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 msgr2=0x5595aed527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 -- 172.21.15.5:0/3422662986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595aed51bd0 con 0x5595aed56a00 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f694eae7640 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed46a80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x5595aed47530 tx=0x7f6940009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b53cf2da93f7f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f6933fff640 1 -- 172.21.15.5:0/3422662986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940016040 con 0x5595aed56a00 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f6933fff640 1 -- 172.21.15.5:0/3422662986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6940012510 con 0x5595aed56a00 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.078+0000 7f6933fff640 1 -- 172.21.15.5:0/3422662986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694002c2b0 con 0x5595aed56a00 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- 172.21.15.5:0/3422662986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 msgr2=0x5595aed46a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed46a80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x5595aed47530 tx=0x7f6940009600 comp rx=0 tx=0).stop 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- 172.21.15.5:0/3422662986 shutdown_connections 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed56a00 0x5595aed46a80 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- 172.21.15.5:0/3422662986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed4d4d0 0x5595aed62830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- 172.21.15.5:0/3422662986 >> 172.21.15.5:0/3422662986 conn(0x5595aed47ba0 msgr2=0x5595aed46370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- 172.21.15.5:0/3422662986 shutdown_connections 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- 172.21.15.5:0/3422662986 wait complete. 2024-04-22T13:37:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- start start 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 Processor -- start 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 0x5595aed5ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed5f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69481af030 con 0x5595aed4d4d0 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69481ae960 con 0x5595aed4de80 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.079+0000 7f695320e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69481abbc0 con 0x5595aed56a00 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f695320e740 10 client.0 _ll_register_callbacks cb 0x7f69380044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 0x5595aed5ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 0x5595aed5ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44722/0 (socket says 172.21.15.5:44722) 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 -- 172.21.15.5:0/2698746830 learned_addr learned my addr 172.21.15.5:0/2698746830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694dae5640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed5f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 -- 172.21.15.5:0/2698746830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 msgr2=0x5595aed4fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed4fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 -- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 msgr2=0x5595aed5f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed5f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 -- 172.21.15.5:0/2698746830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595aed50690 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f694eae7640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 0x5595aed5ed40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6940000f80 tx=0x7f6940010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5583ae1a1a28ffd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f6931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f6931ffb640 1 -- 172.21.15.5:0/2698746830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940016030 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f6931ffb640 1 -- 172.21.15.5:0/2698746830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f69400126c0 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6940012510 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f6931ffb640 1 -- 172.21.15.5:0/2698746830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69400128c0 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f695320e740 10 client.19146 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595aedccd30 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.080+0000 7f6931ffb640 1 -- 172.21.15.5:0/2698746830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6940012510 con 0x5595aed4d4d0 2024-04-22T13:37:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f6931ffb640 1 -- 172.21.15.5:0/2698746830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6940037040 con 0x5595aed4d4d0 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 10 client.19146 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 10 client.19146 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 20 client.19146 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aedcd1a0 0x5595aedcf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 4 client.19146 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 --> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595aecbfc10 con 0x5595aedcd1a0 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f694dae5640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aedcd1a0 0x5595aedcf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f694dae5640 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aedcd1a0 0x5595aedcf590 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ff96c25807127d9 server_cookie=44f096438a714753 in_seq=0 out_seq=0 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f6931ffb640 10 client.19146 ms_handle_connect on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f6931ffb640 1 -- 172.21.15.5:0/2698746830 <== mds.3 v2:172.21.15.49:6834/827214764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595aecbfc10 con 0x5595aedcd1a0 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f6931ffb640 10 client.19146 handle_command_reply: tid=0 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.081+0000 7f695320e740 1 client.19146 shutdown 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 msgr2=0x5595aed5ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 0x5595aed5ed40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6940000f80 tx=0x7f6940010d30 comp rx=0 tx=0).stop 2024-04-22T13:37:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 shutdown_connections 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:6834/827214764,v1:172.21.15.49:6835/827214764] conn(0x5595aedcd1a0 0x5595aedcf590 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595aed56a00 0x5595aed4fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f6931ffb640 0 client.19146 ms_handle_reset on v2:172.21.15.49:6834/827214764 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5595aed4de80 0x5595aed5f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 --2- 172.21.15.5:0/2698746830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595aed4d4d0 0x5595aed5ed40 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 >> 172.21.15.5:0/2698746830 conn(0x5595aed47ba0 msgr2=0x5595aed52660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 shutdown_connections 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 1 -- 172.21.15.5:0/2698746830 wait complete. 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695320e740 20 client.19146 trim_cache size 0 max 16384 2024-04-22T13:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f69340a3d20 msgr2=0x7f69340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f69340a3d20 0x7f69340a61e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f69440ce2e0 tx=0x7f6944018420 comp rx=0 tx=0).stop 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 msgr2=0x7f694819e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 0x7f694819e440 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f694400e9f0 tx=0x7f694400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 shutdown_connections 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f69340a3d20 0x7f69340a61e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69481044f0 0x7f69481985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6948103b50 0x7f694819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 --2- 172.21.15.5:0/2435326657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6948102270 0x7f694819e440 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 >> 172.21.15.5:0/2435326657 conn(0x7f69480fda10 msgr2=0x7f69480fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 shutdown_connections 2024-04-22T13:37:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.082+0000 7f695054a640 1 -- 172.21.15.5:0/2435326657 wait complete. 2024-04-22T13:37:37.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:37:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 -- 172.21.15.5:0/3510322758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c103b70 msgr2=0x7fbb1c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 --2- 172.21.15.5:0/3510322758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c103b70 0x7fbb1c10bfc0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 -- 172.21.15.5:0/3510322758 shutdown_connections 2024-04-22T13:37:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 --2- 172.21.15.5:0/3510322758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c103b70 0x7fbb1c10bfc0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 --2- 172.21.15.5:0/3510322758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb1c10c6d0 0x7fbb1c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 --2- 172.21.15.5:0/3510322758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c1031c0 0x7fbb1c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.197+0000 7fbb24443640 1 -- 172.21.15.5:0/3510322758 >> 172.21.15.5:0/3510322758 conn(0x7fbb1c077f70 msgr2=0x7fbb1c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 -- 172.21.15.5:0/3510322758 shutdown_connections 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 -- 172.21.15.5:0/3510322758 wait complete. 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 Processor -- start 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 -- start start 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb1c1031c0 0x7fbb1c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c10c6d0 0x7fbb1c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 0x7fbb1c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c112330 con 0x7fbb1c198720 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1121b0 con 0x7fbb1c1031c0 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.198+0000 7fbb24443640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1124b0 con 0x7fbb1c10c6d0 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 0x7fbb1c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 0x7fbb1c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44736/0 (socket says 172.21.15.5:44736) 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 -- 172.21.15.5:0/3220975904 learned_addr learned my addr 172.21.15.5:0/3220975904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb221df640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb1c1031c0 0x7fbb1c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb219de640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c10c6d0 0x7fbb1c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 -- 172.21.15.5:0/3220975904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c10c6d0 msgr2=0x7fbb1c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c10c6d0 0x7fbb1c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 -- 172.21.15.5:0/3220975904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb1c1031c0 msgr2=0x7fbb1c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb1c1031c0 0x7fbb1c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 -- 172.21.15.5:0/3220975904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c199320 con 0x7fbb1c198720 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb219de640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c10c6d0 0x7fbb1c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb229e0640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 0x7fbb1c198ba0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400eb10 tx=0x7fbb1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb137fe640 1 -- 172.21.15.5:0/3220975904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004b20 con 0x7fbb1c198720 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c1a5400 con 0x7fbb1c198720 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb137fe640 1 -- 172.21.15.5:0/3220975904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb14004cc0 con 0x7fbb1c198720 2024-04-22T13:37:37.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb137fe640 1 -- 172.21.15.5:0/3220975904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14010670 con 0x7fbb1c198720 2024-04-22T13:37:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.199+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1a57b0 con 0x7fbb1c198720 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.202+0000 7fbb137fe640 1 -- 172.21.15.5:0/3220975904 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbb14005020 con 0x7fbb1c198720 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.202+0000 7fbb27035740 1 Processor -- start 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 -- start start 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d43e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43ee80 0x55af1d447430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d447a00 0x55af1d449e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55af1d3a31f0 con 0x55af1d43e4d0 2024-04-22T13:37:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55af1d3404b0 con 0x55af1d43ee80 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb27035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55af1d39f1b0 con 0x55af1d447a00 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d43e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d43e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44744/0 (socket says 172.21.15.5:44744) 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb219de640 1 -- 172.21.15.5:0/3516784499 learned_addr learned my addr 172.21.15.5:0/3516784499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb221df640 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43ee80 0x55af1d447430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb137fe640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbb080a3d20 0x7fbb080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb219de640 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d447a00 0x55af1d449e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb221df640 1 -- 172.21.15.5:0/3516784499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d447a00 msgr2=0x55af1d449e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb221df640 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d447a00 0x55af1d449e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb221df640 1 -- 172.21.15.5:0/3516784499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 msgr2=0x55af1d43e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb221df640 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d43e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb221df640 1 -- 172.21.15.5:0/3516784499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1d3321b0 con 0x55af1d43ee80 2024-04-22T13:37:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb219de640 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d43e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb229e0640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbb080a3d20 0x7fbb080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.203+0000 7fbb137fe640 1 -- 172.21.15.5:0/3220975904 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fbb140d0df0 con 0x7fbb1c198720 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb221df640 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43ee80 0x55af1d447430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005fd0 tx=0x7fbb180107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07754e772f47827 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb10bfe640 1 -- 172.21.15.5:0/3516784499 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1801c480 con 0x55af1d43ee80 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb10bfe640 1 -- 172.21.15.5:0/3516784499 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1801cad0 con 0x55af1d43ee80 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb10bfe640 1 -- 172.21.15.5:0/3516784499 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18011af0 con 0x55af1d43ee80 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb27035740 1 -- 172.21.15.5:0/3516784499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43ee80 msgr2=0x55af1d447430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb229e0640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbb080a3d20 0x7fbb080a61e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb14099700 tx=0x7fbb1400efd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb27035740 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43ee80 0x55af1d447430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005fd0 tx=0x7fbb180107a0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb27035740 1 -- 172.21.15.5:0/3516784499 shutdown_connections 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.204+0000 7fbb27035740 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d447a00 0x55af1d449e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43ee80 0x55af1d447430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 --2- 172.21.15.5:0/3516784499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d43e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- 172.21.15.5:0/3516784499 >> 172.21.15.5:0/3516784499 conn(0x55af1d438ba0 msgr2=0x55af1d438f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- 172.21.15.5:0/3516784499 shutdown_connections 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- 172.21.15.5:0/3516784499 wait complete. 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- start start 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 Processor -- start 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d4472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d444680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d444c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55af1d44a970 con 0x55af1d43e4d0 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55af1d35c500 con 0x55af1d447a00 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55af1d44aaf0 con 0x55af1d43ee80 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb27035740 10 client.0 _ll_register_callbacks cb 0x55af1d32ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d4472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d4472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44758/0 (socket says 172.21.15.5:44758) 2024-04-22T13:37:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb219de640 1 -- 172.21.15.5:0/2460447106 learned_addr learned my addr 172.21.15.5:0/2460447106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.205+0000 7fbb219de640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d444c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb221df640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d444680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb219de640 1 -- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 msgr2=0x55af1d444680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb219de640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d444680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb219de640 1 -- 172.21.15.5:0/2460447106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 msgr2=0x55af1d444c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb219de640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d444c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb219de640 1 -- 172.21.15.5:0/2460447106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1d0b53f0 con 0x55af1d43e4d0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb219de640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d4472f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c036040 tx=0x7fbb0c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d066e618658072d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb05ffb640 1 -- 172.21.15.5:0/2460447106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c005460 con 0x55af1d43e4d0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1801cad0 con 0x55af1d43e4d0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb05ffb640 1 -- 172.21.15.5:0/2460447106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fbb0c0046c0 con 0x55af1d43e4d0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb05ffb640 1 -- 172.21.15.5:0/2460447106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c053050 con 0x55af1d43e4d0 2024-04-22T13:37:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb27035740 10 client.19158 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af1d17f7d0 con 0x55af1d43e4d0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.206+0000 7fbb05ffb640 1 -- 172.21.15.5:0/2460447106 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1801cad0 con 0x55af1d43e4d0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb05ffb640 1 -- 172.21.15.5:0/2460447106 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb0c003710 con 0x55af1d43e4d0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb27035740 10 client.19158 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb27035740 10 client.19158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb27035740 20 client.19158 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb27035740 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d457bf0 0x55af1d459fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb27035740 4 client.19158 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af1d342930 con 0x55af1d457bf0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb221df640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d457bf0 0x55af1d459fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb221df640 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d457bf0 0x55af1d459fe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d529442dbbd4aa70 server_cookie=e7862451915ff3ae in_seq=0 out_seq=0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.207+0000 7fbb05ffb640 10 client.19158 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb05ffb640 1 -- 172.21.15.5:0/2460447106 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af1d342930 con 0x55af1d457bf0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb05ffb640 10 client.19158 handle_command_reply: tid=0 2024-04-22T13:37:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 client.19158 shutdown 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 msgr2=0x55af1d4472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d4472f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c036040 tx=0x7fbb0c0058b0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 shutdown_connections 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d457bf0 0x55af1d459fe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d444c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb05ffb640 0 client.19158 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d444680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 --2- 172.21.15.5:0/2460447106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d4472f0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 >> 172.21.15.5:0/2460447106 conn(0x55af1d438ba0 msgr2=0x55af1d43a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 shutdown_connections 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 1 -- 172.21.15.5:0/2460447106 wait complete. 2024-04-22T13:37:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.208+0000 7fbb27035740 20 client.19158 trim_cache size 0 max 16384 2024-04-22T13:37:37.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 Processor -- start 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 -- start start 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43e4d0 0x55af1d453830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d446db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 0x55af1d4472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1adc40 con 0x55af1d447a00 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1af780 con 0x55af1d43e4d0 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb27035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1ae310 con 0x55af1d43ee80 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 0x55af1d4472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 0x55af1d4472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44766/0 (socket says 172.21.15.5:44766) 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb219de640 1 -- 172.21.15.5:0/2112543708 learned_addr learned my addr 172.21.15.5:0/2112543708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb219de640 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43e4d0 0x55af1d453830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.222+0000 7fbb221df640 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d446db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb219de640 1 -- 172.21.15.5:0/2112543708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 msgr2=0x55af1d446db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb219de640 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d446db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb219de640 1 -- 172.21.15.5:0/2112543708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43e4d0 msgr2=0x55af1d453830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb219de640 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43e4d0 0x55af1d453830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb219de640 1 -- 172.21.15.5:0/2112543708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1d45a350 con 0x55af1d447a00 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb219de640 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 0x55af1d4472f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005ec0 tx=0x7fbb0c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30ce5b8db4ad754 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb07fff640 1 -- 172.21.15.5:0/2112543708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003eb0 con 0x55af1d447a00 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb07fff640 1 -- 172.21.15.5:0/2112543708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c0035f0 con 0x55af1d447a00 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb07fff640 1 -- 172.21.15.5:0/2112543708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004ab0 con 0x55af1d447a00 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb27035740 1 -- 172.21.15.5:0/2112543708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 msgr2=0x55af1d4472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb27035740 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 0x55af1d4472f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005ec0 tx=0x7fbb0c005a20 comp rx=0 tx=0).stop 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.223+0000 7fbb27035740 1 -- 172.21.15.5:0/2112543708 shutdown_connections 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d447a00 0x55af1d4472f0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d446db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 --2- 172.21.15.5:0/2112543708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d43e4d0 0x55af1d453830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- 172.21.15.5:0/2112543708 >> 172.21.15.5:0/2112543708 conn(0x55af1d438ba0 msgr2=0x55af1d437c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- 172.21.15.5:0/2112543708 shutdown_connections 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- 172.21.15.5:0/2112543708 wait complete. 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- start start 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 Processor -- start 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d450030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d440f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d4414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1af0b0 con 0x55af1d43e4d0 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1ae9e0 con 0x55af1d447a00 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1abc40 con 0x55af1d43ee80 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb27035740 10 client.0 _ll_register_callbacks cb 0x7fbb0c02c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d450030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb219de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d450030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44780/0 (socket says 172.21.15.5:44780) 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb219de640 1 -- 172.21.15.5:0/1718739316 learned_addr learned my addr 172.21.15.5:0/1718739316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb221df640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d440f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb219de640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d4414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.224+0000 7fbb219de640 1 -- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 msgr2=0x55af1d440f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb219de640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d440f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb219de640 1 -- 172.21.15.5:0/1718739316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 msgr2=0x55af1d4414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb219de640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d4414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb219de640 1 -- 172.21.15.5:0/1718739316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1d444f70 con 0x55af1d43e4d0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb219de640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d450030 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c02c860 tx=0x7fbb0c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d358d6878d78c2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb05ffb640 1 -- 172.21.15.5:0/1718739316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0055b0 con 0x55af1d43e4d0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb05ffb640 1 -- 172.21.15.5:0/1718739316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fbb0c003cb0 con 0x55af1d43e4d0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c0035f0 con 0x55af1d43e4d0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb05ffb640 1 -- 172.21.15.5:0/1718739316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c003eb0 con 0x55af1d43e4d0 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 10 client.19164 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af1d4c2210 con 0x55af1d43e4d0 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb05ffb640 1 -- 172.21.15.5:0/1718739316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c0035f0 con 0x55af1d43e4d0 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb05ffb640 1 -- 172.21.15.5:0/1718739316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb0c034710 con 0x55af1d43e4d0 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 10 client.19164 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 10 client.19164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 20 client.19164 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d4c2810 0x55af1d4c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 4 client.19164 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.225+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 --> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af1d3b0c10 con 0x55af1d4c2810 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb221df640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d4c2810 0x55af1d4c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb221df640 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d4c2810 0x55af1d4c4c30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82f27097e9b6f4b server_cookie=2a41176d05ce3b26 in_seq=0 out_seq=0 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb05ffb640 10 client.19164 ms_handle_connect on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb05ffb640 1 -- 172.21.15.5:0/1718739316 <== mds.2 v2:172.21.15.107:6838/3107882391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af1d3b0c10 con 0x55af1d4c2810 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb05ffb640 10 client.19164 handle_command_reply: tid=0 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb27035740 1 client.19164 shutdown 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 msgr2=0x55af1d450030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.226+0000 7fbb27035740 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d450030 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c02c860 tx=0x7fbb0c005bd0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 shutdown_connections 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:6838/3107882391,v1:172.21.15.107:6839/3107882391] conn(0x55af1d4c2810 0x55af1d4c4c30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af1d447a00 0x55af1d4414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af1d43ee80 0x55af1d440f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 --2- 172.21.15.5:0/1718739316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af1d43e4d0 0x55af1d450030 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb05ffb640 0 client.19164 ms_handle_reset on v2:172.21.15.107:6838/3107882391 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 >> 172.21.15.5:0/1718739316 conn(0x55af1d438ba0 msgr2=0x55af1d437370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 shutdown_connections 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 1 -- 172.21.15.5:0/1718739316 wait complete. 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb27035740 20 client.19164 trim_cache size 0 max 16384 2024-04-22T13:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbb080a3d20 msgr2=0x7fbb080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbb080a3d20 0x7fbb080a61e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb14099700 tx=0x7fbb1400efd0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 msgr2=0x7fbb1c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 0x7fbb1c198ba0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400eb10 tx=0x7fbb1400c2d0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 shutdown_connections 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb1c198720 0x7fbb1c198ba0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fbb080a3d20 0x7fbb080a61e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb1c10c6d0 0x7fbb1c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 --2- 172.21.15.5:0/3220975904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb1c1031c0 0x7fbb1c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 >> 172.21.15.5:0/3220975904 conn(0x7fbb1c077f70 msgr2=0x7fbb1c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 shutdown_connections 2024-04-22T13:37:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.227+0000 7fbb24443640 1 -- 172.21.15.5:0/3220975904 wait complete. 2024-04-22T13:37:37.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:37:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 -- 172.21.15.5:0/356810944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698410c6d0 msgr2=0x7f698410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 --2- 172.21.15.5:0/356810944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698410c6d0 0x7f698410eb60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 -- 172.21.15.5:0/356810944 shutdown_connections 2024-04-22T13:37:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 --2- 172.21.15.5:0/356810944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698410c6d0 0x7f698410eb60 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 --2- 172.21.15.5:0/356810944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6984103b70 0x7f698410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 --2- 172.21.15.5:0/356810944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69841031c0 0x7f69841035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 -- 172.21.15.5:0/356810944 >> 172.21.15.5:0/356810944 conn(0x7f6984077f70 msgr2=0x7f6984078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 -- 172.21.15.5:0/356810944 shutdown_connections 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 -- 172.21.15.5:0/356810944 wait complete. 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.342+0000 7f698b19d640 1 Processor -- start 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 -- start start 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 0x7f698419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410c6d0 0x7f698419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6984198720 0x7f6984198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69841123a0 con 0x7f69841031c0 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6984112220 con 0x7f6984198720 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698b19d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6984112520 con 0x7f698410c6d0 2024-04-22T13:37:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 0x7f698419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 0x7f698419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44798/0 (socket says 172.21.15.5:44798) 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 -- 172.21.15.5:0/3276646337 learned_addr learned my addr 172.21.15.5:0/3276646337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f698973a640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6984198720 0x7f6984198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6983fff640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410c6d0 0x7f698419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 -- 172.21.15.5:0/3276646337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410c6d0 msgr2=0x7f698419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410c6d0 0x7f698419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 -- 172.21.15.5:0/3276646337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6984198720 msgr2=0x7f6984198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6984198720 0x7f6984198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 -- 172.21.15.5:0/3276646337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6984199320 con 0x7f69841031c0 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.343+0000 7f6988f39640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 0x7f698419e660 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f697400e9f0 tx=0x7f697400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.344+0000 7f6981ffb640 1 -- 172.21.15.5:0/3276646337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400cd80 con 0x7f69841031c0 2024-04-22T13:37:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.344+0000 7f6981ffb640 1 -- 172.21.15.5:0/3276646337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69740108e0 con 0x7f69841031c0 2024-04-22T13:37:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.344+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841a5400 con 0x7f69841031c0 2024-04-22T13:37:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.344+0000 7f6981ffb640 1 -- 172.21.15.5:0/3276646337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974018980 con 0x7f69841031c0 2024-04-22T13:37:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.344+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a5860 con 0x7f69841031c0 2024-04-22T13:37:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f6981ffb640 1 -- 172.21.15.5:0/3276646337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6974010430 con 0x7f69841031c0 2024-04-22T13:37:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 Processor -- start 2024-04-22T13:37:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 -- start start 2024-04-22T13:37:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 0x562ff373b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373be80 0x562ff3744430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff3746e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ff36a01f0 con 0x562ff373b4d0 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ff363d4b0 con 0x562ff373be80 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698dd8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ff369c1b0 con 0x562ff3744a00 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698973a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 0x562ff373b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698973a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 0x562ff373b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44814/0 (socket says 172.21.15.5:44814) 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f698973a640 1 -- 172.21.15.5:0/2687471480 learned_addr learned my addr 172.21.15.5:0/2687471480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.347+0000 7f6988f39640 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373be80 0x562ff3744430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f6983fff640 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff3746e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 -- 172.21.15.5:0/2687471480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 msgr2=0x562ff3746e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff3746e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f6981ffb640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f696c0a3da0 0x7f696c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 -- 172.21.15.5:0/2687471480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373be80 msgr2=0x562ff3744430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373be80 0x562ff3744430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 -- 172.21.15.5:0/2687471480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff362f1b0 con 0x562ff373b4d0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f6981ffb640 1 -- 172.21.15.5:0/3276646337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f69740d0b80 con 0x7f69841031c0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f696c0a3da0 0x7f696c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f698973a640 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 0x562ff373b8b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900666c9e87573ab server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f696affd640 1 -- 172.21.15.5:0/2687471480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c039040 con 0x562ff373b4d0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.348+0000 7f696affd640 1 -- 172.21.15.5:0/2687471480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697c043050 con 0x562ff373b4d0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f696affd640 1 -- 172.21.15.5:0/2687471480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c051070 con 0x562ff373b4d0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 -- 172.21.15.5:0/2687471480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 msgr2=0x562ff373b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698973a640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f696c0a3da0 0x7f696c0a6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f697c04a000 tx=0x7f697c04b000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 0x562ff373b8b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c005ef0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 -- 172.21.15.5:0/2687471480 shutdown_connections 2024-04-22T13:37:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff3746e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373be80 0x562ff3744430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2687471480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373b4d0 0x562ff373b8b0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 -- 172.21.15.5:0/2687471480 >> 172.21.15.5:0/2687471480 conn(0x562ff3735ba0 msgr2=0x562ff3735f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 -- 172.21.15.5:0/2687471480 shutdown_connections 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 -- 172.21.15.5:0/2687471480 wait complete. 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.349+0000 7f698dd8f740 1 -- start start 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 Processor -- start 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff37442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3741670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3741bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ff3747970 con 0x562ff373be80 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x562ff3659500 con 0x562ff3744a00 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ff3747af0 con 0x562ff373b4d0 2024-04-22T13:37:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698dd8f740 10 client.0 _ll_register_callbacks cb 0x562ff3627b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3741670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3741670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44830/0 (socket says 172.21.15.5:44830) 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 -- 172.21.15.5:0/2054901959 learned_addr learned my addr 172.21.15.5:0/2054901959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f698973a640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3741bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6988f39640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff37442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 -- 172.21.15.5:0/2054901959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 msgr2=0x562ff37442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff37442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 -- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 msgr2=0x562ff3741bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3741bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.350+0000 7f6983fff640 1 -- 172.21.15.5:0/2054901959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff33b23f0 con 0x562ff373be80 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6983fff640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3741670 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f6978013710 tx=0x7f6978013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106c29511868b6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6963fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6963fff640 1 -- 172.21.15.5:0/2054901959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978004460 con 0x562ff373be80 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6963fff640 1 -- 172.21.15.5:0/2054901959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f69780119c0 con 0x562ff373be80 2024-04-22T13:37:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697c043050 con 0x562ff373be80 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6963fff640 1 -- 172.21.15.5:0/2054901959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6978011bc0 con 0x562ff373be80 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 10 client.19176 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ff347c7d0 con 0x562ff373be80 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6963fff640 1 -- 172.21.15.5:0/2054901959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697c043050 con 0x562ff373be80 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f6963fff640 1 -- 172.21.15.5:0/2054901959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f697801a070 con 0x562ff373be80 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 10 client.19176 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 10 client.19176 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 20 client.19176 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff3754c80 0x562ff37570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 4 client.19176 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.351+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ff363f930 con 0x562ff3754c80 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.352+0000 7f6988f39640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff3754c80 0x562ff37570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.352+0000 7f6963fff640 10 client.19176 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.352+0000 7f6988f39640 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff3754c80 0x562ff37570c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cac4e5c15e7fceea server_cookie=fe45dfd5c1c10cd1 in_seq=0 out_seq=0 2024-04-22T13:37:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f6963fff640 1 -- 172.21.15.5:0/2054901959 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ff363f930 con 0x562ff3754c80 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f6963fff640 10 client.19176 handle_command_reply: tid=0 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 client.19176 shutdown 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 msgr2=0x562ff3741670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3741670 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f6978013710 tx=0x7f6978013bd0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 shutdown_connections 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff3754c80 0x562ff37570c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3741670 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f6963fff640 0 client.19176 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3741bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 --2- 172.21.15.5:0/2054901959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff37442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 >> 172.21.15.5:0/2054901959 conn(0x562ff3735ba0 msgr2=0x562ff37378f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 shutdown_connections 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 1 -- 172.21.15.5:0/2054901959 wait complete. 2024-04-22T13:37:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.353+0000 7f698dd8f740 20 client.19176 trim_cache size 0 max 16384 2024-04-22T13:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 Processor -- start 2024-04-22T13:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 -- start start 2024-04-22T13:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373b4d0 0x562ff3750830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3743da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff37442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69841adce0 con 0x562ff373be80 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69841af820 con 0x562ff373b4d0 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.366+0000 7f698dd8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69841ae3b0 con 0x562ff3744a00 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3743da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3743da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44842/0 (socket says 172.21.15.5:44842) 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 -- 172.21.15.5:0/1020742199 learned_addr learned my addr 172.21.15.5:0/1020742199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6988f39640 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373b4d0 0x562ff3750830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f698973a640 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff37442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 -- 172.21.15.5:0/1020742199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 msgr2=0x562ff37442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff37442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 -- 172.21.15.5:0/1020742199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373b4d0 msgr2=0x562ff3750830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373b4d0 0x562ff3750830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 -- 172.21.15.5:0/1020742199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff373fc00 con 0x562ff373be80 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f698973a640 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff37442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f6983fff640 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3743da0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6978001eb0 tx=0x7f69780096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad26ae62d88a05c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f696a7fc640 1 -- 172.21.15.5:0/1020742199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978031070 con 0x562ff373be80 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f696a7fc640 1 -- 172.21.15.5:0/1020742199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6978011d60 con 0x562ff373be80 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.367+0000 7f696a7fc640 1 -- 172.21.15.5:0/1020742199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978003bc0 con 0x562ff373be80 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 -- 172.21.15.5:0/1020742199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 msgr2=0x562ff3743da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3743da0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6978001eb0 tx=0x7f69780096c0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 -- 172.21.15.5:0/1020742199 shutdown_connections 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff3743da0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff3744a00 0x562ff37442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 --2- 172.21.15.5:0/1020742199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff373b4d0 0x562ff3750830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 -- 172.21.15.5:0/1020742199 >> 172.21.15.5:0/1020742199 conn(0x562ff3735ba0 msgr2=0x562ff373cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 -- 172.21.15.5:0/1020742199 shutdown_connections 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 -- 172.21.15.5:0/1020742199 wait complete. 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 -- start start 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.368+0000 7f698dd8f740 1 Processor -- start 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff374cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff374d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3749ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69841af150 con 0x562ff373be80 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69841aea80 con 0x562ff3744a00 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69841abce0 con 0x562ff373b4d0 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 10 client.0 _ll_register_callbacks cb 0x7f6978009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff374d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff374d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44844/0 (socket says 172.21.15.5:44844) 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 -- 172.21.15.5:0/871273217 learned_addr learned my addr 172.21.15.5:0/871273217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698973a640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3749ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 -- 172.21.15.5:0/871273217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 msgr2=0x562ff374cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6988f39640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff374cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff374cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 -- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 msgr2=0x562ff3749ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3749ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 -- 172.21.15.5:0/871273217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff374a6c0 con 0x562ff373be80 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6983fff640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff374d280 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6978009c00 tx=0x7f6978003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc88b30e7351cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6963fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6963fff640 1 -- 172.21.15.5:0/871273217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978031040 con 0x562ff373be80 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6978011d60 con 0x562ff373be80 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.369+0000 7f6963fff640 1 -- 172.21.15.5:0/871273217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f6978003b40 con 0x562ff373be80 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f6963fff640 1 -- 172.21.15.5:0/871273217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6978003d60 con 0x562ff373be80 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 10 client.19182 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ff37c0230 con 0x562ff373be80 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f6963fff640 1 -- 172.21.15.5:0/871273217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6978011d60 con 0x562ff373be80 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f6963fff640 1 -- 172.21.15.5:0/871273217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6978034070 con 0x562ff373be80 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 10 client.19182 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 10 client.19182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 20 client.19182 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff37c0860 0x562ff37c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 4 client.19182 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 --> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ff36adc10 con 0x562ff37c0860 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.370+0000 7f6988f39640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff37c0860 0x562ff37c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f6988f39640 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff37c0860 0x562ff37c2c80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a8b7a240bd31de1 server_cookie=d1ac023220dba003 in_seq=0 out_seq=0 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f6963fff640 10 client.19182 ms_handle_connect on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f6963fff640 1 -- 172.21.15.5:0/871273217 <== mds.1 v2:172.21.15.49:6836/164218196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ff36adc10 con 0x562ff37c0860 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f6963fff640 10 client.19182 handle_command_reply: tid=0 2024-04-22T13:37:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 client.19182 shutdown 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 msgr2=0x562ff374d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff374d280 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6978009c00 tx=0x7f6978003820 comp rx=0 tx=0).stop 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 shutdown_connections 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:6836/164218196,v1:172.21.15.49:6837/164218196] conn(0x562ff37c0860 0x562ff37c2c80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ff373be80 0x562ff374d280 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562ff3744a00 0x562ff3749ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 --2- 172.21.15.5:0/871273217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff373b4d0 0x562ff374cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f6963fff640 0 client.19182 ms_handle_reset on v2:172.21.15.49:6836/164218196 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 >> 172.21.15.5:0/871273217 conn(0x562ff3735ba0 msgr2=0x562ff3736f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 shutdown_connections 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 1 -- 172.21.15.5:0/871273217 wait complete. 2024-04-22T13:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.371+0000 7f698dd8f740 20 client.19182 trim_cache size 0 max 16384 2024-04-22T13:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f696c0a3da0 msgr2=0x7f696c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f696c0a3da0 0x7f696c0a6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f697c04a000 tx=0x7f697c04b000 comp rx=0 tx=0).stop 2024-04-22T13:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 msgr2=0x7f698419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 0x7f698419e660 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f697400e9f0 tx=0x7f697400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 shutdown_connections 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f696c0a3da0 0x7f696c0a6260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6984198720 0x7f6984198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410c6d0 0x7f698419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 --2- 172.21.15.5:0/3276646337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69841031c0 0x7f698419e660 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 >> 172.21.15.5:0/3276646337 conn(0x7f6984077f70 msgr2=0x7f69840fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 shutdown_connections 2024-04-22T13:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.372+0000 7f698b19d640 1 -- 172.21.15.5:0/3276646337 wait complete. 2024-04-22T13:37:37.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:37:37.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 -- 172.21.15.5:0/3732513684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d010c500 msgr2=0x7f29d010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 --2- 172.21.15.5:0/3732513684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d010c500 0x7f29d010e9a0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009ca0 tx=0x7f29c802c180 comp rx=0 tx=0).stop 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 -- 172.21.15.5:0/3732513684 shutdown_connections 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 --2- 172.21.15.5:0/3732513684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d010c500 0x7f29d010e9a0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 --2- 172.21.15.5:0/3732513684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29d0103ae0 0x7f29d010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 --2- 172.21.15.5:0/3732513684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d01031c0 0x7f29d01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 -- 172.21.15.5:0/3732513684 >> 172.21.15.5:0/3732513684 conn(0x7f29d0078000 msgr2=0x7f29d0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 -- 172.21.15.5:0/3732513684 shutdown_connections 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.486+0000 7f29d5723640 1 -- 172.21.15.5:0/3732513684 wait complete. 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 Processor -- start 2024-04-22T13:37:37.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 -- start start 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 0x7f29d0100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29d0103ae0 0x7f29d0101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d010c500 0x7f29d0101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0112170 con 0x7f29d01031c0 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0111ff0 con 0x7f29d010c500 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29d5723640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01122f0 con 0x7f29d0103ae0 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 0x7f29d0100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 0x7f29d0100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44864/0 (socket says 172.21.15.5:44864) 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29ceffd640 1 -- 172.21.15.5:0/992374967 learned_addr learned my addr 172.21.15.5:0/992374967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d010c500 0x7f29d0101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.487+0000 7f29c7fff640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29d0103ae0 0x7f29d0101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ceffd640 1 -- 172.21.15.5:0/992374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29d0103ae0 msgr2=0x7f29d0101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ceffd640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29d0103ae0 0x7f29d0101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ceffd640 1 -- 172.21.15.5:0/992374967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d010c500 msgr2=0x7f29d0101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ceffd640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d010c500 0x7f29d0101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ceffd640 1 -- 172.21.15.5:0/992374967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d01a6b90 con 0x7f29d01031c0 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ceffd640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 0x7f29d0100e10 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e9f0 tx=0x7f29c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ccff9640 1 -- 172.21.15.5:0/992374967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c000cd80 con 0x7f29d01031c0 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ccff9640 1 -- 172.21.15.5:0/992374967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c00108e0 con 0x7f29d01031c0 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d01a6e80 con 0x7f29d01031c0 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29ccff9640 1 -- 172.21.15.5:0/992374967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0018980 con 0x7f29d01031c0 2024-04-22T13:37:37.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.488+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01a7340 con 0x7f29d01031c0 2024-04-22T13:37:37.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.490+0000 7f29ccff9640 1 -- 172.21.15.5:0/992374967 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f29c0010430 con 0x7f29d01031c0 2024-04-22T13:37:37.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.490+0000 7f29d83ec740 1 Processor -- start 2024-04-22T13:37:37.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.490+0000 7f29d83ec740 1 -- start start 2024-04-22T13:37:37.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babfef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babffae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29d83ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556babf541f0 con 0x556babff8a00 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29d83ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556babef14b0 con 0x556babfef4d0 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29d83ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556babf501b0 con 0x556babfefe80 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babffae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babfef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babffae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44880/0 (socket says 172.21.15.5:44880) 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babfef8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.5:38158/0 (socket says 172.21.15.5:38158) 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 -- 172.21.15.5:0/3189023907 learned_addr learned my addr 172.21.15.5:0/3189023907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29ceffd640 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29ccff9640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f29bc0a3d20 0x7f29bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 -- 172.21.15.5:0/3189023907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 msgr2=0x556babff8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 -- 172.21.15.5:0/3189023907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 msgr2=0x556babfef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babfef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29c7fff640 1 -- 172.21.15.5:0/3189023907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556babee31b0 con 0x556babff8a00 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f29bc0a3d20 0x7f29bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.491+0000 7f29ccff9640 1 -- 172.21.15.5:0/992374967 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f29c00d0ac0 con 0x7f29d01031c0 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29c7fff640 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babffae50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f29b800bc10 tx=0x7f29b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e27d9007bbbe41 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29c57fa640 1 -- 172.21.15.5:0/3189023907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8005020 con 0x556babff8a00 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29c57fa640 1 -- 172.21.15.5:0/3189023907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80047a0 con 0x556babff8a00 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29c57fa640 1 -- 172.21.15.5:0/3189023907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80116a0 con 0x556babff8a00 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29d83ec740 1 -- 172.21.15.5:0/3189023907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 msgr2=0x556babffae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f29bc0a3d20 0x7f29bc0a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005e30 tx=0x7f29c802e040 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.492+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babffae50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f29b800bc10 tx=0x7f29b80107a0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- 172.21.15.5:0/3189023907 shutdown_connections 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babffae50 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3189023907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babfef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- 172.21.15.5:0/3189023907 >> 172.21.15.5:0/3189023907 conn(0x556babfe9ba0 msgr2=0x556babfe9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- 172.21.15.5:0/3189023907 shutdown_connections 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- 172.21.15.5:0/3189023907 wait complete. 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- start start 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 Processor -- start 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babff82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556babffb970 con 0x556babff8a00 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556babf0d500 con 0x556babfef4d0 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556babffbaf0 con 0x556babfefe80 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29d83ec740 10 client.0 _ll_register_callbacks cb 0x556babedbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44882/0 (socket says 172.21.15.5:44882) 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babff82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29cf7fe640 1 -- 172.21.15.5:0/3155904852 learned_addr learned my addr 172.21.15.5:0/3155904852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.493+0000 7f29c7fff640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29ceffd640 1 -- 172.21.15.5:0/3155904852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 msgr2=0x556babff5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29ceffd640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29ceffd640 1 -- 172.21.15.5:0/3155904852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 msgr2=0x556babff5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29ceffd640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29ceffd640 1 -- 172.21.15.5:0/3155904852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556babc663f0 con 0x556babfef4d0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29ceffd640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babff82e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f29c00ce210 tx=0x7f29c000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad1834c96c2d605 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29b67fc640 1 -- 172.21.15.5:0/3155904852 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c009c470 con 0x556babfef4d0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29b67fc640 1 -- 172.21.15.5:0/3155904852 <== mon.2 v2:172.21.15.49:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f29c009bf60 con 0x556babfef4d0 2024-04-22T13:37:37.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b80047a0 con 0x556babfef4d0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29b67fc640 1 -- 172.21.15.5:0/3155904852 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29c00bd0e0 con 0x556babfef4d0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29d83ec740 10 client.27908 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556babd307d0 con 0x556babfef4d0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.494+0000 7f29b67fc640 1 -- 172.21.15.5:0/3155904852 <== mon.2 v2:172.21.15.49:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80047a0 con 0x556babfef4d0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29b67fc640 1 -- 172.21.15.5:0/3155904852 <== mon.2 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f29c009b9d0 con 0x556babfef4d0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29d83ec740 10 client.27908 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29d83ec740 10 client.27908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29d83ec740 20 client.27908 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac008b70 0x556bac00af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29d83ec740 4 client.27908 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556babef3930 con 0x556bac008b70 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29c7fff640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac008b70 0x556bac00af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29c7fff640 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac008b70 0x556bac00af80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8152805c99c11df server_cookie=9c2f94a47ed390cd in_seq=0 out_seq=0 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.495+0000 7f29b67fc640 10 client.27908 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:37.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29b67fc640 1 -- 172.21.15.5:0/3155904852 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556babef3930 con 0x556bac008b70 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29b67fc640 10 client.27908 handle_command_reply: tid=0 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 client.27908 shutdown 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 msgr2=0x556babff82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babff82e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f29c00ce210 tx=0x7f29c000cf20 comp rx=0 tx=0).stop 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 shutdown_connections 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac008b70 0x556bac00af80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babff5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29b67fc640 0 client.27908 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 --2- 172.21.15.5:0/3155904852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556babff82e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 >> 172.21.15.5:0/3155904852 conn(0x556babfe9ba0 msgr2=0x556babfeb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 shutdown_connections 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 1 -- 172.21.15.5:0/3155904852 wait complete. 2024-04-22T13:37:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.496+0000 7f29d83ec740 20 client.27908 trim_cache size 0 max 16384 2024-04-22T13:37:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.509+0000 7f29d83ec740 1 Processor -- start 2024-04-22T13:37:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.509+0000 7f29d83ec740 1 -- start start 2024-04-22T13:37:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556bac004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babfe8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29d83ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01afd20 con 0x556babff8a00 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29d83ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b1860 con 0x556babfef4d0 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29d83ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b03f0 con 0x556babfefe80 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44896/0 (socket says 172.21.15.5:44896) 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556bac004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 -- 172.21.15.5:0/2578419425 learned_addr learned my addr 172.21.15.5:0/2578419425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29c7fff640 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babfe8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 -- 172.21.15.5:0/2578419425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 msgr2=0x556babfe8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babfe8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 -- 172.21.15.5:0/2578419425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 msgr2=0x556bac004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556bac004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 -- 172.21.15.5:0/2578419425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556babff5f40 con 0x556babff8a00 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29ceffd640 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556bac004830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.510+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff7da0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f29c802c790 tx=0x7f29c8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b85e8be7b0ba58 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29c4ff9640 1 -- 172.21.15.5:0/2578419425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8039040 con 0x556babff8a00 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29c4ff9640 1 -- 172.21.15.5:0/2578419425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c8054070 con 0x556babff8a00 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29c4ff9640 1 -- 172.21.15.5:0/2578419425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8034bd0 con 0x556babff8a00 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 -- 172.21.15.5:0/2578419425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 msgr2=0x556babff7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff7da0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f29c802c790 tx=0x7f29c8041000 comp rx=0 tx=0).stop 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 -- 172.21.15.5:0/2578419425 shutdown_connections 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babff8a00 0x556babff7da0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfefe80 0x556babfe8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2578419425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babfef4d0 0x556bac004830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 -- 172.21.15.5:0/2578419425 >> 172.21.15.5:0/2578419425 conn(0x556babfe9ba0 msgr2=0x556babfe8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 -- 172.21.15.5:0/2578419425 shutdown_connections 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.511+0000 7f29d83ec740 1 -- 172.21.15.5:0/2578419425 wait complete. 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 -- start start 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 Processor -- start 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfef4d0 0x556bac000d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 0x556bac001240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babff8a00 0x556babffc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b1190 con 0x556babfefe80 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b0ac0 con 0x556babff8a00 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01add20 con 0x556babfef4d0 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29d83ec740 10 client.0 _ll_register_callbacks cb 0x7f29c0098db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 0x556bac001240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 0x556bac001240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44898/0 (socket says 172.21.15.5:44898) 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 -- 172.21.15.5:0/2392815073 learned_addr learned my addr 172.21.15.5:0/2392815073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29cf7fe640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babff8a00 0x556babffc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29ceffd640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfef4d0 0x556bac000d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 -- 172.21.15.5:0/2392815073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfef4d0 msgr2=0x556bac000d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfef4d0 0x556bac000d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 -- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babff8a00 msgr2=0x556babffc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babff8a00 0x556babffc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.512+0000 7f29c7fff640 1 -- 172.21.15.5:0/2392815073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556babffcb10 con 0x556babfefe80 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29c7fff640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 0x556bac001240 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f29b8000c00 tx=0x7f29b800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9edfb8506828f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29b67fc640 1 -- 172.21.15.5:0/2392815073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b802e060 con 0x556babfefe80 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29b67fc640 1 -- 172.21.15.5:0/2392815073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f29b8005450 con 0x556babfefe80 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c8054070 con 0x556babfefe80 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29b67fc640 1 -- 172.21.15.5:0/2392815073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29b8005650 con 0x556babfefe80 2024-04-22T13:37:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 10 client.19200 fetch_fsmap learned FSMap version 28 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bac06eca0 con 0x556babfefe80 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29b67fc640 1 -- 172.21.15.5:0/2392815073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c8054070 con 0x556babfefe80 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29b67fc640 1 -- 172.21.15.5:0/2392815073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f29b8029310 con 0x556babfefe80 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 10 client.19200 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 10 client.19200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 20 client.19200 populate_metadata read hostname 'smithi005' 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac06f140 0x556bac071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 4 client.19200 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.513+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 --> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556babf61c10 con 0x556bac06f140 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29ceffd640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac06f140 0x556bac071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29ceffd640 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac06f140 0x556bac071530 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cb1e629345e633 server_cookie=d651b0cf411ab6cf in_seq=0 out_seq=0 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29b67fc640 10 client.19200 ms_handle_connect on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29b67fc640 1 -- 172.21.15.5:0/2392815073 <== mds.0 v2:172.21.15.49:6838/1393810832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556babf61c10 con 0x556bac06f140 2024-04-22T13:37:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29b67fc640 10 client.19200 handle_command_reply: tid=0 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 client.19200 shutdown 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 msgr2=0x556bac001240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 0x556bac001240 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f29b8000c00 tx=0x7f29b800b7a0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 shutdown_connections 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:6838/1393810832,v1:172.21.15.49:6839/1393810832] conn(0x556bac06f140 0x556bac071530 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556babfefe80 0x556bac001240 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29b67fc640 0 client.19200 ms_handle_reset on v2:172.21.15.49:6838/1393810832 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556babff8a00 0x556babffc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 --2- 172.21.15.5:0/2392815073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556babfef4d0 0x556bac000d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 >> 172.21.15.5:0/2392815073 conn(0x556babfe9ba0 msgr2=0x556babfe8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 shutdown_connections 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 1 -- 172.21.15.5:0/2392815073 wait complete. 2024-04-22T13:37:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.514+0000 7f29d83ec740 20 client.19200 trim_cache size 0 max 16384 2024-04-22T13:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-22T13:37:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f29bc0a3d20 msgr2=0x7f29bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f29bc0a3d20 0x7f29bc0a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005e30 tx=0x7f29c802e040 comp rx=0 tx=0).stop 2024-04-22T13:37:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 msgr2=0x7f29d0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 0x7f29d0100e10 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e9f0 tx=0x7f29c000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 shutdown_connections 2024-04-22T13:37:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f29bc0a3d20 0x7f29bc0a61e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d010c500 0x7f29d0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29d0103ae0 0x7f29d0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 --2- 172.21.15.5:0/992374967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d01031c0 0x7f29d0100e10 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 >> 172.21.15.5:0/992374967 conn(0x7f29d0078000 msgr2=0x7f29d00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 shutdown_connections 2024-04-22T13:37:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.515+0000 7f29d5723640 1 -- 172.21.15.5:0/992374967 wait complete. 2024-04-22T13:37:37.523 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-22T13:37:37.524 INFO:tasks.fwd_scrub:done joining 2024-04-22T13:37:37.524 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-22T13:37:37.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 -- 172.21.15.5:0/1352951347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06a640 msgr2=0x7f504c06aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 --2- 172.21.15.5:0/1352951347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06a640 0x7f504c06aac0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5040009930 tx=0x7f504002bc30 comp rx=0 tx=0).stop 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 -- 172.21.15.5:0/1352951347 shutdown_connections 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 --2- 172.21.15.5:0/1352951347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06a640 0x7f504c06aac0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 --2- 172.21.15.5:0/1352951347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f504c06b000 0x7f504c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 --2- 172.21.15.5:0/1352951347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 0x7f504c108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 -- 172.21.15.5:0/1352951347 >> 172.21.15.5:0/1352951347 conn(0x7f504c077c40 msgr2=0x7f504c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 -- 172.21.15.5:0/1352951347 shutdown_connections 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.641+0000 7f5051f7e640 1 -- 172.21.15.5:0/1352951347 wait complete. 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 Processor -- start 2024-04-22T13:37:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 -- start start 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f504c06a640 0x7f504c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 0x7f504c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 0x7f504c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1121a0 con 0x7f504c06b000 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f504c112020 con 0x7f504c06a640 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f5051f7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f504c112320 con 0x7f504c108060 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f504affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 0x7f504c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f504affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 0x7f504c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44914/0 (socket says 172.21.15.5:44914) 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.642+0000 7f504affd640 1 -- 172.21.15.5:0/4097583454 learned_addr learned my addr 172.21.15.5:0/4097583454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504b7fe640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f504c06a640 0x7f504c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504bfff640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 0x7f504c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504affd640 1 -- 172.21.15.5:0/4097583454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 msgr2=0x7f504c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504affd640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 0x7f504c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504affd640 1 -- 172.21.15.5:0/4097583454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f504c06a640 msgr2=0x7f504c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504affd640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f504c06a640 0x7f504c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504affd640 1 -- 172.21.15.5:0/4097583454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c19d690 con 0x7f504c06b000 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504bfff640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 0x7f504c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f504affd640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 0x7f504c198be0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5040009230 tx=0x7f504002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040043020 con 0x7f504c06b000 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c19d920 con 0x7f504c06b000 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f504002caf0 con 0x7f504c06b000 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040003af0 con 0x7f504c06b000 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.643+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c1a5210 con 0x7f504c06b000 2024-04-22T13:37:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.644+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c103650 con 0x7f504c06b000 2024-04-22T13:37:37.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.645+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5040046020 con 0x7f504c06b000 2024-04-22T13:37:37.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.646+0000 7f5048ff9640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50340a3d20 0x7f50340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.646+0000 7f504b7fe640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50340a3d20 0x7f50340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.646+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f50400f2c00 con 0x7f504c06b000 2024-04-22T13:37:37.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.647+0000 7f504b7fe640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50340a3d20 0x7f50340a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f503c005fd0 tx=0x7f503c00b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.650+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f504004d020 con 0x7f504c06b000 2024-04-22T13:37:37.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:37 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2319307358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:37:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.838+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f504c1999f0 con 0x7f504c06b000 2024-04-22T13:37:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.839+0000 7f5048ff9640 1 -- 172.21.15.5:0/4097583454 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23014 (secure 0 0 0) 0x7f50400bf3b0 con 0x7f504c06b000 2024-04-22T13:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":101,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:37:12.386618+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50340a3d20 msgr2=0x7f50340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50340a3d20 0x7f50340a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f503c005fd0 tx=0x7f503c00b390 comp rx=0 tx=0).stop 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 msgr2=0x7f504c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 0x7f504c198be0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5040009230 tx=0x7f504002c710 comp rx=0 tx=0).stop 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 shutdown_connections 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f504c06b000 0x7f504c198be0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f50340a3d20 0x7f50340a61e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f504c108060 0x7f504c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 --2- 172.21.15.5:0/4097583454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f504c06a640 0x7f504c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 >> 172.21.15.5:0/4097583454 conn(0x7f504c077c40 msgr2=0x7f504c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 shutdown_connections 2024-04-22T13:37:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.841+0000 7f5051f7e640 1 -- 172.21.15.5:0/4097583454 wait complete. 2024-04-22T13:37:37.849 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:37:37.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 -- 172.21.15.5:0/2696152072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08103400 msgr2=0x7f7f0810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 --2- 172.21.15.5:0/2696152072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08103400 0x7f7f0810de00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009cd0 tx=0x7f7f0002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 -- 172.21.15.5:0/2696152072 shutdown_connections 2024-04-22T13:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 --2- 172.21.15.5:0/2696152072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08103400 0x7f7f0810de00 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 --2- 172.21.15.5:0/2696152072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f08102a60 0x7f7f08102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 --2- 172.21.15.5:0/2696152072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08101180 0x7f7f08101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 -- 172.21.15.5:0/2696152072 >> 172.21.15.5:0/2696152072 conn(0x7f7f08077f70 msgr2=0x7f7f08078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 -- 172.21.15.5:0/2696152072 shutdown_connections 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.958+0000 7f7f0df15640 1 -- 172.21.15.5:0/2696152072 wait complete. 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 Processor -- start 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 -- start start 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 0x7f7f0819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08102a60 0x7f7f0819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f08103400 0x7f7f08198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08111180 con 0x7f7f08101180 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08111000 con 0x7f7f08102a60 2024-04-22T13:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f0df15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08111300 con 0x7f7f08103400 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 0x7f7f0819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 0x7f7f0819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44932/0 (socket says 172.21.15.5:44932) 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f077fe640 1 -- 172.21.15.5:0/4258680323 learned_addr learned my addr 172.21.15.5:0/4258680323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f06ffd640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08102a60 0x7f7f0819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.959+0000 7f7f07fff640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f08103400 0x7f7f08198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f06ffd640 1 -- 172.21.15.5:0/4258680323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f08103400 msgr2=0x7f7f08198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f06ffd640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f08103400 0x7f7f08198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f06ffd640 1 -- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 msgr2=0x7f7f0819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f06ffd640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 0x7f7f0819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f06ffd640 1 -- 172.21.15.5:0/4258680323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f08198d10 con 0x7f7f08102a60 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f077fe640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 0x7f7f0819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f06ffd640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08102a60 0x7f7f0819e920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005fd0 tx=0x7f7efc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc009c30 con 0x7f7f08102a60 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7efc009dd0 con 0x7f7f08102a60 2024-04-22T13:37:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f08199000 con 0x7f7f08102a60 2024-04-22T13:37:37.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc003e20 con 0x7f7f08102a60 2024-04-22T13:37:37.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.960+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f081a52a0 con 0x7f7f08102a60 2024-04-22T13:37:37.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.961+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f08069250 con 0x7f7f08102a60 2024-04-22T13:37:37.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.962+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7efc004000 con 0x7f7f08102a60 2024-04-22T13:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.963+0000 7f7f04ff9640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7ef00a3da0 0x7f7ef00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.963+0000 7f7f077fe640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7ef00a3da0 0x7f7ef00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.963+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f7efc0d0230 con 0x7f7f08102a60 2024-04-22T13:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.963+0000 7f7f077fe640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7ef00a3da0 0x7f7ef00a6260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80095f0 tx=0x7f7ef8005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:37.967+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7efc017020 con 0x7f7f08102a60 2024-04-22T13:37:38.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:37 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2319307358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:37:38.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:37 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2319307358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:37:38.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.154+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f0810a460 con 0x7f7f08102a60 2024-04-22T13:37:38.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.156+0000 7f7f04ff9640 1 -- 172.21.15.5:0/4258680323 <== mon.2 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f7efc09c300 con 0x7f7f08102a60 2024-04-22T13:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:37:38.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7ef00a3da0 msgr2=0x7f7ef00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7ef00a3da0 0x7f7ef00a6260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80095f0 tx=0x7f7ef8005f90 comp rx=0 tx=0).stop 2024-04-22T13:37:38.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08102a60 msgr2=0x7f7f0819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08102a60 0x7f7f0819e920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005fd0 tx=0x7f7efc00dde0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 shutdown_connections 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f7ef00a3da0 0x7f7ef00a6260 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f08103400 0x7f7f08198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08102a60 0x7f7f0819e920 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 --2- 172.21.15.5:0/4258680323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f08101180 0x7f7f0819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 >> 172.21.15.5:0/4258680323 conn(0x7f7f08077f70 msgr2=0x7f7f0806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 shutdown_connections 2024-04-22T13:37:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.157+0000 7f7f0df15640 1 -- 172.21.15.5:0/4258680323 wait complete. 2024-04-22T13:37:38.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 -- 172.21.15.5:0/3488576956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b0103ae0 msgr2=0x7f53b010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/3488576956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b0103ae0 0x7f53b010bfc0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5398009900 tx=0x7f539802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 -- 172.21.15.5:0/3488576956 shutdown_connections 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/3488576956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b0103ae0 0x7f53b010bfc0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/3488576956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b010c590 0x7f53b010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/3488576956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53b01031c0 0x7f53b01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 -- 172.21.15.5:0/3488576956 >> 172.21.15.5:0/3488576956 conn(0x7f53b0077f40 msgr2=0x7f53b0078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 -- 172.21.15.5:0/3488576956 shutdown_connections 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.273+0000 7f53b4a4a640 1 -- 172.21.15.5:0/3488576956 wait complete. 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 Processor -- start 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 -- start start 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 0x7f53b019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53b0103ae0 0x7f53b019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b010c590 0x7f53b0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53b01121b0 con 0x7f53b01031c0 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53b0112030 con 0x7f53b0103ae0 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53b4a4a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53b0112330 con 0x7f53b010c590 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53ae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 0x7f53b019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53ae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 0x7f53b019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44948/0 (socket says 172.21.15.5:44948) 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53ae59c640 1 -- 172.21.15.5:0/1708764941 learned_addr learned my addr 172.21.15.5:0/1708764941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.274+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53b0103ae0 0x7f53b019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53aed9d640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b010c590 0x7f53b0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53ae59c640 1 -- 172.21.15.5:0/1708764941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b010c590 msgr2=0x7f53b0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53ae59c640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b010c590 0x7f53b0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53ae59c640 1 -- 172.21.15.5:0/1708764941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53b0103ae0 msgr2=0x7f53b019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53ae59c640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53b0103ae0 0x7f53b019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53ae59c640 1 -- 172.21.15.5:0/1708764941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b0198d10 con 0x7f53b01031c0 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53ae59c640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 0x7f53b019e3e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f53a000d950 tx=0x7f53a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0017070 con 0x7f53b01031c0 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b0199000 con 0x7f53b01031c0 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a0009c30 con 0x7f53b01031c0 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a000f620 con 0x7f53b01031c0 2024-04-22T13:37:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.275+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b01a52a0 con 0x7f53b01031c0 2024-04-22T13:37:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.276+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b0068570 con 0x7f53b01031c0 2024-04-22T13:37:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.277+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f53a0010460 con 0x7f53b01031c0 2024-04-22T13:37:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.278+0000 7f53a77fe640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f539c0a3d20 0x7f539c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.278+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f53a00d03f0 con 0x7f53b01031c0 2024-04-22T13:37:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.278+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f539c0a3d20 0x7f539c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.279+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f539c0a3d20 0x7f539c0a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5398006010 tx=0x7f5398005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.283+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53a0009de0 con 0x7f53b01031c0 2024-04-22T13:37:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.469+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53b01076e0 con 0x7f53b01031c0 2024-04-22T13:37:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.470+0000 7f53a77fe640 1 -- 172.21.15.5:0/1708764941 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f53a009c3e0 con 0x7f53b01031c0 2024-04-22T13:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:37:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f539c0a3d20 msgr2=0x7f539c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f539c0a3d20 0x7f539c0a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5398006010 tx=0x7f5398005e40 comp rx=0 tx=0).stop 2024-04-22T13:37:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 msgr2=0x7f53b019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 0x7f53b019e3e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f53a000d950 tx=0x7f53a000de10 comp rx=0 tx=0).stop 2024-04-22T13:37:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 shutdown_connections 2024-04-22T13:37:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f539c0a3d20 0x7f539c0a61e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b010c590 0x7f53b0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53b0103ae0 0x7f53b019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.471+0000 7f53b4a4a640 1 --2- 172.21.15.5:0/1708764941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b01031c0 0x7f53b019e3e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.472+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 >> 172.21.15.5:0/1708764941 conn(0x7f53b0077f40 msgr2=0x7f53b00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.472+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 shutdown_connections 2024-04-22T13:37:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.472+0000 7f53b4a4a640 1 -- 172.21.15.5:0/1708764941 wait complete. 2024-04-22T13:37:38.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 -- 172.21.15.5:0/3545492911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c100810 msgr2=0x7f1c0c100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- 172.21.15.5:0/3545492911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c100810 0x7f1c0c100c90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009930 tx=0x7f1bf802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 -- 172.21.15.5:0/3545492911 shutdown_connections 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- 172.21.15.5:0/3545492911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c100810 0x7f1c0c100c90 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- 172.21.15.5:0/3545492911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c0c1011d0 0x7f1c0c10edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- 172.21.15.5:0/3545492911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c108150 0x7f1c0c108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 -- 172.21.15.5:0/3545492911 >> 172.21.15.5:0/3545492911 conn(0x7f1c0c076170 msgr2=0x7f1c0c076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 -- 172.21.15.5:0/3545492911 shutdown_connections 2024-04-22T13:37:38.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 -- 172.21.15.5:0/3545492911 wait complete. 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 Processor -- start 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 -- start start 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c0c100810 0x7f1c0c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c1011d0 0x7f1c0c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.588+0000 7f1c13088640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 0x7f1c0c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c13088640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c112220 con 0x7f1c0c108150 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c13088640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1120a0 con 0x7f1c0c100810 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c13088640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1123a0 con 0x7f1c0c1011d0 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 0x7f1c0c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 0x7f1c0c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44966/0 (socket says 172.21.15.5:44966) 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 -- 172.21.15.5:0/2864614290 learned_addr learned my addr 172.21.15.5:0/2864614290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:38.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c10e24640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c0c100810 0x7f1c0c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c0bfff640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c1011d0 0x7f1c0c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 -- 172.21.15.5:0/2864614290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c1011d0 msgr2=0x7f1c0c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c1011d0 0x7f1c0c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 -- 172.21.15.5:0/2864614290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c0c100810 msgr2=0x7f1c0c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c0c100810 0x7f1c0c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 -- 172.21.15.5:0/2864614290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c198d40 con 0x7f1c0c108150 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c0bfff640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c1011d0 0x7f1c0c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c11625640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 0x7f1c0c1985c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400eaa0 tx=0x7f1c0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400cdb0 con 0x7f1c0c108150 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c04005020 con 0x7f1c0c108150 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c199030 con 0x7f1c0c108150 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.589+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c040116d0 con 0x7f1c0c108150 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.590+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c101920 con 0x7f1c0c108150 2024-04-22T13:37:38.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.590+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c069250 con 0x7f1c0c108150 2024-04-22T13:37:38.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.591+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1c040051d0 con 0x7f1c0c108150 2024-04-22T13:37:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.592+0000 7f1c09ffb640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.592+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f1c040d0a70 con 0x7f1c0c108150 2024-04-22T13:37:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.592+0000 7f1c10e24640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.592+0000 7f1c10e24640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c0000b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:38.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.597+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c04016030 con 0x7f1c0c108150 2024-04-22T13:37:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.783+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c0c10a400 con 0x7f1c0c108150 2024-04-22T13:37:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.784+0000 7f1c09ffb640 1 -- 172.21.15.5:0/2864614290 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f1c0409d0b0 con 0x7f1c0c108150 2024-04-22T13:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bfc0a3d20 msgr2=0x7f1bfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c0000b390 comp rx=0 tx=0).stop 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 msgr2=0x7f1c0c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 0x7f1c0c1985c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400eaa0 tx=0x7f1c0400ef60 comp rx=0 tx=0).stop 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 shutdown_connections 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c108150 0x7f1c0c1985c0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c0c1011d0 0x7f1c0c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 --2- 172.21.15.5:0/2864614290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c0c100810 0x7f1c0c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 >> 172.21.15.5:0/2864614290 conn(0x7f1c0c076170 msgr2=0x7f1c0c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 shutdown_connections 2024-04-22T13:37:38.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.786+0000 7f1c13088640 1 -- 172.21.15.5:0/2864614290 wait complete. 2024-04-22T13:37:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:38 smithi005 ceph-mon[28006]: pgmap v326: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:38 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4097583454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:37:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:38 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4258680323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:37:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:38 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1708764941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:37:38.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-22T13:37:38.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.898+0000 7f8f877ce640 1 -- 172.21.15.5:0/746711327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f80107270 msgr2=0x7f8f8010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.898+0000 7f8f877ce640 1 --2- 172.21.15.5:0/746711327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f80107270 0x7f8f8010ee40 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009ca0 tx=0x7f8f7802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- 172.21.15.5:0/746711327 shutdown_connections 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 --2- 172.21.15.5:0/746711327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f80107270 0x7f8f8010ee40 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 --2- 172.21.15.5:0/746711327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f80102b50 0x7f8f80106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 --2- 172.21.15.5:0/746711327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f801021a0 0x7f8f80102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- 172.21.15.5:0/746711327 >> 172.21.15.5:0/746711327 conn(0x7f8f800fda40 msgr2=0x7f8f800ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- 172.21.15.5:0/746711327 shutdown_connections 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- 172.21.15.5:0/746711327 wait complete. 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 Processor -- start 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- start start 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f801021a0 0x7f8f80198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f80107270 0x7f8f80198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 0x7f8f8019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f801121e0 con 0x7f8f8019d000 2024-04-22T13:37:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80112060 con 0x7f8f80107270 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.899+0000 7f8f877ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80112360 con 0x7f8f801021a0 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 0x7f8f8019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 0x7f8f8019d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59810/0 (socket says 172.21.15.5:59810) 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 -- 172.21.15.5:0/624668379 learned_addr learned my addr 172.21.15.5:0/624668379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f84d69640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f80107270 0x7f8f80198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f8556a640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f801021a0 0x7f8f80198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 -- 172.21.15.5:0/624668379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f801021a0 msgr2=0x7f8f80198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f801021a0 0x7f8f80198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 -- 172.21.15.5:0/624668379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f80107270 msgr2=0x7f8f80198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f80107270 0x7f8f80198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 -- 172.21.15.5:0/624668379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8019daf0 con 0x7f8f8019d000 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f8556a640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f801021a0 0x7f8f80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f84d69640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f80107270 0x7f8f80198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:37:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f85d6b640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 0x7f8f8019d4b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005e30 tx=0x7f8f78034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:38.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78039040 con 0x7f8f8019d000 2024-04-22T13:37:38.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8019dd80 con 0x7f8f8019d000 2024-04-22T13:37:38.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f78044070 con 0x7f8f8019d000 2024-04-22T13:37:38.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78003e60 con 0x7f8f8019d000 2024-04-22T13:37:38.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.900+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f801a55a0 con 0x7f8f8019d000 2024-04-22T13:37:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.901+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f80069250 con 0x7f8f8019d000 2024-04-22T13:37:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.903+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8f780324c0 con 0x7f8f8019d000 2024-04-22T13:37:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.904+0000 7f8f767fc640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:37:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.904+0000 7f8f8556a640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:37:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.904+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f8f780f34a0 con 0x7f8f8019d000 2024-04-22T13:37:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.904+0000 7f8f8556a640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009b40 tx=0x7f8f7c005e50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:37:38.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:38.908+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f780bfc50 con 0x7f8f8019d000 2024-04-22T13:37:39.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:38 smithi049 ceph-mon[31379]: pgmap v326: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:39.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:38 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4097583454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:37:39.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:38 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4258680323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:37:39.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:38 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1708764941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:37:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:38 smithi107 ceph-mon[31859]: pgmap v326: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:38 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4097583454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:37:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:38 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4258680323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:37:39.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:38 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1708764941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:37:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.146+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8f80102b50 con 0x7f8f8019d000 2024-04-22T13:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.154+0000 7f8f767fc640 1 -- 172.21.15.5:0/624668379 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746048 (secure 0 0 0) 0x7f8f78051e10 con 0x7f8f8019d000 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 15354, 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-04-22T13:37:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 751347710 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 751347710 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/751347710", 2024-04-22T13:37:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-04-22T13:27:09.442209+0000", 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 15342, 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-04-22T13:37:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-04-22T13:37:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-22T13:37:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T13:37:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-04-22T13:37:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-22T13:37:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T13:37:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T13:37:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T13:37:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-04-22T13:37:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T13:37:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-04-22T13:37:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T13:37:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.233 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T13:37:39.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-04-22T13:37:39.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-04-22T13:37:39.243 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T13:37:39.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.249 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-04-22T13:37:39.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-04-22T13:37:39.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-04-22T13:37:39.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.264 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.265 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T13:37:39.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.277 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T13:37:39.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-04-22T13:37:39.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-04-22T13:37:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-04-22T13:37:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T13:37:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-04-22T13:37:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-22T13:37:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T13:37:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-04-22T13:37:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-04-22T13:37:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-04-22T13:37:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-04-22T13:37:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-04-22T13:37:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:37:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-04-22T13:37:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T13:37:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.392 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.394 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.402 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.405 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.408 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.412 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.417 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.419 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.420 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.422 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.424 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.425 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-04-22T13:37:39.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T13:37:39.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.437 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T13:37:39.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.446 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-04-22T13:37:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-04-22T13:37:39.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-04-22T13:37:39.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.456 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.462 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.463 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.473 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.477 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-04-22T13:37:39.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.505 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.510 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.517 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.524 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.525 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-04-22T13:37:39.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-04-22T13:37:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-04-22T13:37:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-04-22T13:37:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-04-22T13:37:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-04-22T13:37:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T13:37:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25058, 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-04-22T13:37:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-04-22T13:37:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-04-22T13:37:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-04-22T13:37:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-04-22T13:37:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-04-22T13:37:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T13:37:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-04-22T13:37:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-04-22T13:37:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T13:37:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-04-22T13:37:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T13:37:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T13:37:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-04-22T13:37:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-04-22T13:37:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T13:37:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T13:37:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T13:37:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T13:37:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T13:37:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.703 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T13:37:39.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T13:37:39.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T13:37:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-04-22T13:37:39.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.717 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-04-22T13:37:39.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-22T13:37:39.730 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.732 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.734 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.746 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.755 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.769 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.773 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.774 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.782 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.795 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.796 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.801 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.815 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.817 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-22T13:37:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-04-22T13:37:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-04-22T13:37:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T13:37:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-04-22T13:37:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-22T13:37:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-04-22T13:37:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T13:37:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T13:37:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-22T13:37:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-04-22T13:37:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2024-04-22T13:37:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-04-22T13:37:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-04-22T13:37:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-04-22T13:37:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-04-22T13:37:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-04-22T13:37:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:37:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-04-22T13:37:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-22T13:37:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T13:37:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T13:37:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-04-22T13:37:40.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.006 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.017 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-04-22T13:37:40.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.019 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.020 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-04-22T13:37:40.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.022 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-04-22T13:37:40.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-04-22T13:37:40.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-04-22T13:37:40.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-04-22T13:37:40.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:40.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.039 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T13:37:40.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T13:37:40.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T13:37:40.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T13:37:40.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T13:37:40.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.056 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-04-22T13:37:40.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-04-22T13:37:40.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.061 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.062 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.064 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.068 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.069 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T13:37:40.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T13:37:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T13:37:40.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T13:37:40.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T13:37:40.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.101 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.102 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T13:37:40.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T13:37:40.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T13:37:40.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T13:37:40.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T13:37:40.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.117 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-22T13:37:40.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.127 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-22T13:37:40.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-04-22T13:37:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-04-22T13:37:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-04-22T13:37:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-04-22T13:37:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-04-22T13:37:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-04-22T13:37:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.221 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.225 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.226 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.230 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.231 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-04-22T13:37:40.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-04-22T13:37:40.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-04-22T13:37:40.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.247 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T13:37:40.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.249 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.250 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.252 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-04-22T13:37:40.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.255 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-04-22T13:37:40.261 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-22T13:37:40.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.263 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.264 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.269 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-22T13:37:40.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.274 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.275 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.278 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.283 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.286 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-04-22T13:37:40.288 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.292 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.296 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-04-22T13:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.321 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.322 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-04-22T13:37:40.327 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.328 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.331 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-04-22T13:37:40.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.339 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-04-22T13:37:40.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.345 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.357 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.360 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-04-22T13:37:40.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-04-22T13:37:40.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.373 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-04-22T13:37:40.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-22T13:37:40.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-22T13:37:40.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.387 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-22T13:37:40.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-22T13:37:40.396 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-22T13:37:40.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.398 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-04-22T13:37:40.399 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.400 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 92, 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1638411898 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-04-22T13:37:40.401 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 806506508 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3279233079 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-22T13:37:40.403 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3223714694 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1912392307 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 310770953 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-22T13:37:40.405 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:40.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:39 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/2864614290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:37:40.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:39 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/624668379' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T13:37:40.408 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:39 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/2864614290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:37:40.408 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:39 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/624668379' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T13:37:40.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f6c0a3d20 msgr2=0x7f8f6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:40.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009b40 tx=0x7f8f7c005e50 comp rx=0 tx=0).stop 2024-04-22T13:37:40.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 msgr2=0x7f8f8019d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:37:40.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 0x7f8f8019d4b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005e30 tx=0x7f8f78034760 comp rx=0 tx=0).stop 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 shutdown_connections 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8019d000 0x7f8f8019d4b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f80107270 0x7f8f80198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 --2- 172.21.15.5:0/624668379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f801021a0 0x7f8f80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 >> 172.21.15.5:0/624668379 conn(0x7f8f800fda40 msgr2=0x7f8f800fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 shutdown_connections 2024-04-22T13:37:40.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:37:39.160+0000 7f8f877ce640 1 -- 172.21.15.5:0/624668379 wait complete. 2024-04-22T13:37:40.416 DEBUG:tasks.check_counter:Getting stats from a 2024-04-22T13:37:40.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-22T13:37:40.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:39 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/2864614290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:37:40.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:39 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/624668379' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T13:37:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:37:40.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:40 smithi005 ceph-mon[28006]: pgmap v327: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:41.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:40 smithi049 ceph-mon[31379]: pgmap v327: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:41.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:40 smithi107 ceph-mon[31859]: pgmap v327: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:42.734 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:42.734 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 754, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 754, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 200995, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 239012, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.061597590, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.019786012, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.016351693, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.018825664, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 753, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.022178245, 2024-04-22T13:37:42.735 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000029453 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 622, 2024-04-22T13:37:42.736 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 776, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 152862, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 638429, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.069779101, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.028359193, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.014471184, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.020287076, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 773, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.028307087, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000036619 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:42.737 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 441264 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 979, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 966, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1874608, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 365198, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 3, 2024-04-22T13:37:42.738 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.131849247, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.050814165, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.054615497, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.027399285, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 965, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.047624608, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000049351 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.739 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1624288, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 128288 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3601, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.099247581, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000027561 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.740 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 0, 2024-04-22T13:37:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-04-22T13:37:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:37:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-04-22T13:37:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-04-22T13:37:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 3, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-04-22T13:37:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4195200, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4194892, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 3 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 3, 2024-04-22T13:37:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 3, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 1, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 1, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 50048, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215416 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.754 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.755 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-04-22T13:37:42.755 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.755 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.755 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.756 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.758 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-04-22T13:37:42.760 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.761 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 1, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 0, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 1, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 1901, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:42.764 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:42.765 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 189450, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 53, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8032, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 167, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8512, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 13, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-04-22T13:37:42.767 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 2115, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 2115, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2115, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1.004977764, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000475166 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2115, 2024-04-22T13:37:42.768 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 2116 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 2115, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 2115, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 1363, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 750, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-04-22T13:37:42.769 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-04-22T13:37:42.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-04-22T13:37:42.771 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 101, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 12, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 5, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 5, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-04-22T13:37:42.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-04-22T13:37:42.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 2351, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 2001900, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 2351, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:42.774 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2351, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2001900, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:42.775 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2115, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 901, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2115, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 901, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:42.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2115, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 2115, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2115, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2115, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.777 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:42.778 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55c2c1e746e0": { 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:42.779 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:42.780 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:43.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:42 smithi049 ceph-mon[31379]: pgmap v328: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:43.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:42 smithi107 ceph-mon[31859]: pgmap v328: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:43.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:42 smithi005 ceph-mon[28006]: pgmap v328: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:43.521 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-22T13:37:43.521 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-22T13:37:43.521 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-22T13:37:43.521 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:37:43.521 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:37:43.521 DEBUG:tasks.check_counter:Getting stats from d 2024-04-22T13:37:43.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-22T13:37:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:37:44.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:44 smithi005 ceph-mon[28006]: pgmap v329: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:45.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:44 smithi049 ceph-mon[31379]: pgmap v329: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:45.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:44 smithi107 ceph-mon[31859]: pgmap v329: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 749, 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 751, 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 251609, 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 238183, 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-04-22T13:37:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.087052392, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.035983357, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.019057728, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.022802765, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 749, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.036196028, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000048325 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1630, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1617, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1982025, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 574047, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.145062581, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.042900776, 2024-04-22T13:37:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.064697885, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.040588446, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1616, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.052257721, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000032337 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1626256, 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 132288 2024-04-22T13:37:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 9, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 165, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 4660, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 443381, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.013762478, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.008762234, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002898889, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000064496, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 163, 2024-04-22T13:37:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.015889931, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000097484 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 442416 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-22T13:37:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3630, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.095668638, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000026354 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-04-22T13:37:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-04-22T13:37:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:37:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-04-22T13:37:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-04-22T13:37:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-04-22T13:37:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 7, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-04-22T13:37:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4196347, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4196206, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 7 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 3, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 3, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 0, 2024-04-22T13:37:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 1, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 1, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 50524, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215416 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-04-22T13:37:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-04-22T13:37:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 3, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 2, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 3, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 4457, 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-04-22T13:37:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 197507, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 57, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-22T13:37:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8032, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 167, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8512, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 13, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-22T13:37:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 2141, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 2141, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2141, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1.147974583, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000536186 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2141, 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 2142 2024-04-22T13:37:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 2141, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 2141, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 1383, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 756, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-04-22T13:37:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:37:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-04-22T13:37:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 101, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 12, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 6, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 6, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-04-22T13:37:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 2384, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 2008395, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 2384, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2384, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2008395, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2141, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 2048, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2141, 2024-04-22T13:37:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2048, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2141, 2024-04-22T13:37:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 2141, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2141, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2141, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55a94e0e66e0": { 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:45.496 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:46.167 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-04-22T13:37:46.167 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-04-22T13:37:46.167 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-04-22T13:37:46.167 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:37:46.167 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:37:46.167 DEBUG:tasks.check_counter:Getting stats from g 2024-04-22T13:37:46.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-22T13:37:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:37:46.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:46 smithi005 ceph-mon[28006]: pgmap v330: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:47.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:46 smithi049 ceph-mon[31379]: pgmap v330: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:47.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:46 smithi107 ceph-mon[31859]: pgmap v330: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1507, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1509, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 452608, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 479227, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.121639074, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.041030718, 2024-04-22T13:37:48.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.030731200, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.037868439, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1507, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.040633701, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000026963 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:48.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 645, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 801, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 159150, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 647774, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.066725708, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.024134678, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.014437981, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.021083065, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 798, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.033388314, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000041839 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.192 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 443232 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 243, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 228, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1628700, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 131118, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 3, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.045605982, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.014589300, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.036397288, 2024-04-22T13:37:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.004513402, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 228, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.012558768, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000055082 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1628432, 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 130800 2024-04-22T13:37:48.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3582, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.089312111, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000024933 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-22T13:37:48.195 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-04-22T13:37:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-04-22T13:37:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:37:48.199 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-04-22T13:37:48.200 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-04-22T13:37:48.201 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 8, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-04-22T13:37:48.202 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4196655, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4196514, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 8 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 3, 2024-04-22T13:37:48.203 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 3, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 1, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 1, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 49788, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215416 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-04-22T13:37:48.204 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-04-22T13:37:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.206 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.207 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-04-22T13:37:48.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-04-22T13:37:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.210 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.211 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.212 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.213 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 4, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 2, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 4, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-04-22T13:37:48.215 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 3183, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:48.217 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 189433, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 52, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8032, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 167, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8512, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 13, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-04-22T13:37:48.219 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 2151, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 2151, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2151, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1.194973506, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000555543 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2151, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 2152 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 2151, 2024-04-22T13:37:48.220 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 2151, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 1390, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 759, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-04-22T13:37:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:37:48.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-04-22T13:37:48.223 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 101, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 12, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 5, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 5, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-04-22T13:37:48.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:48.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 2391, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 2010085, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 2391, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2391, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2010085, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2151, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 2356, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2151, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2356, 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:48.227 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2151, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 2151, 2024-04-22T13:37:48.228 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2151, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2151, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:48.229 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55d96941e6e0": { 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:48.230 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:48.231 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:48.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:48 smithi005 ceph-mon[28006]: pgmap v331: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:48.930 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-22T13:37:48.930 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-22T13:37:48.930 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-22T13:37:48.930 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:37:48.930 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:37:48.930 DEBUG:tasks.check_counter:Getting stats from j 2024-04-22T13:37:48.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-22T13:37:49.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:48 smithi049 ceph-mon[31379]: pgmap v331: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:49.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:48 smithi107 ceph-mon[31859]: pgmap v331: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:37:50.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:50 smithi005 ceph-mon[28006]: pgmap v332: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:51.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:50 smithi049 ceph-mon[31379]: pgmap v332: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 224, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 213, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1590464, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 130608, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.045367702, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.014431141, 2024-04-22T13:37:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.034854039, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.004447601, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 212, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.010319816, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000048678 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1590464, 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 130608 2024-04-22T13:37:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 6, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 165, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3312, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 185216, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.014092949, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.009216036, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002273839, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 163, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.013731884, 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000084244 2024-04-22T13:37:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 185216 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-04-22T13:37:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000458834, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000390914, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 149017, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-04-22T13:37:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 226, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1558875, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 226, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 226, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1558875, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-22T13:37:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-22T13:37:51.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-22T13:37:51.041 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:50 smithi107 ceph-mon[31859]: pgmap v332: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:51.720 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-22T13:37:51.720 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-22T13:37:51.720 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-22T13:37:51.720 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-22T13:37:51.721 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-22T13:37:51.721 DEBUG:tasks.check_counter:Getting stats from b 2024-04-22T13:37:51.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-22T13:37:51.984 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:37:53.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:52 smithi049 ceph-mon[31379]: pgmap v333: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:53.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:52 smithi107 ceph-mon[31859]: pgmap v333: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:52 smithi005 ceph-mon[28006]: pgmap v333: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:53.667 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1204, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1195, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 426384, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 1550329, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 32, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 6, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.142819616, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.047214962, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.047165380, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.028414207, 2024-04-22T13:37:53.668 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1193, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.616533104, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000516792 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1015, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000311463, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000306 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 321728 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.669 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1884, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1544, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 2354082, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 552855, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 10, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.214598443, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.070234788, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.105053265, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.050637429, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1547, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.870865963, 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000562938 2024-04-22T13:37:53.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2226, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000507774, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000228 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1706128, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 159632 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2828, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1323, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1072019, 2024-04-22T13:37:53.671 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 2755407, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 201, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 8, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.383847755, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.083179378, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.171003903, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.115454467, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1317, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.554285196, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000420869 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3293, 2024-04-22T13:37:53.672 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000672687, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000204 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 454400, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 248720 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 664, 2024-04-22T13:37:53.673 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.178893921, 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000269418 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 87, 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.003593049, 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000041299 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-04-22T13:37:53.674 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 166, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 166, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 166, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 17.053352279, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.102731037 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 46, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-04-22T13:37:53.675 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 39, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 7, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 32, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 29, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 3, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 3, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 2, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 237, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 225, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-04-22T13:37:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-04-22T13:37:53.677 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 235, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 4, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 234, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 6, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 235, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 147, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 20, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 127, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 88, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 3, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 29, 2024-04-22T13:37:53.678 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 46, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 88, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 57, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 5, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 28 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-04-22T13:37:53.679 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 9, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 2, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 6, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 4, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-04-22T13:37:53.680 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 4, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 53, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 20, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 193, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 109, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 109, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 84, 2024-04-22T13:37:53.681 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 14, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 14, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 13, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 13, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 1, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 1, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 1, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4595162, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4834692, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 0, 2024-04-22T13:37:53.682 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 159, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.817503191, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.005141529 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 0 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 29, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 53, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 24, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 20, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 20, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 0, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 39, 2024-04-22T13:37:53.683 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 43, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 4, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 3, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 25, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 22, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 55748, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 231800 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 216, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 45, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 166, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 36, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 90, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 8, 2024-04-22T13:37:53.684 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.006527875, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000815984 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 97, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.018206648, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000187697 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.685 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 7, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.002107144, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000301020 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.686 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000148021, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000148021 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.687 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 10, 2024-04-22T13:37:53.688 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.001673304, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000167330 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 8, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 16.910120559, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 2.113765069 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.689 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 10, 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.052883090, 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.005288309 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1, 2024-04-22T13:37:53.690 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000200130, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000200130 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.004292346, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001430782 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.002030692, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001015346 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.691 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.009920667, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.004960333 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 4, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.018380350, 2024-04-22T13:37:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.004595087 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.001094029, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001094029 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 8, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.019687311, 2024-04-22T13:37:53.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002460913 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 4, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.006080113, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001520028 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:37:53.695 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 6, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 4, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 27, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 13, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 616, 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-04-22T13:37:53.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:53.697 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:53.698 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 319125, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 117, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8032, 2024-04-22T13:37:53.699 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 167, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 134197, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 1193, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 1192, 2024-04-22T13:37:53.700 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 715122, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 19, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 1192, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1192, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 24.838340486, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.020837533 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1192, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 1337 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 1192, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 277, 2024-04-22T13:37:53.701 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 915, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 26, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 28, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 22, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 161, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 59, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 579, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-04-22T13:37:53.702 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 255, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 56, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-04-22T13:37:53.703 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 101, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 13, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 12, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 12, 2024-04-22T13:37:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 138, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 0, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 13 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 1, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 102 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-04-22T13:37:53.705 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 64, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 3, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 67, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 4, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 5912, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 3250329, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 5912, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:53.706 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 5912, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 3250329, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:53.707 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 1192, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 715558, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 1192, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 715558, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:53.708 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 1192, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 1192, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 1192, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1192, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 4, 2024-04-22T13:37:53.709 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 436, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 4, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 436, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x55d77c3d2460": { 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-04-22T13:37:53.710 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 193, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 640388, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 193, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 159, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 640388, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:53.711 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-22T13:37:54.414 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-22T13:37:54.415 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-22T13:37:54.415 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-22T13:37:54.415 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-22T13:37:54.415 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-22T13:37:54.415 DEBUG:tasks.check_counter:Getting stats from e 2024-04-22T13:37:54.415 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-22T13:37:54.675 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:37:55.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:54 smithi049 ceph-mon[31379]: pgmap v334: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:55.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:54 smithi107 ceph-mon[31859]: pgmap v334: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:55.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:54 smithi005 ceph-mon[28006]: pgmap v334: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 589, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 691, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 167755, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 783506, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 45, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.090026148, 2024-04-22T13:37:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.035190361, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.045514338, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.004599280, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 694, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.054920164, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000079135 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1144, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000277990, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000242 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-04-22T13:37:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 1424 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 41, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 88, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 10290, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 193052, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 14, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.011299139, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.002886953, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.007225538, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000425884, 2024-04-22T13:37:56.308 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 85, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.019212839, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000226033 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 55, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000012423, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000225 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 4672, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 137216 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 307, 2024-04-22T13:37:56.309 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 458, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1648508, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 550609, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 13, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 6, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.070797268, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.024400934, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.050003547, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.005721266, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 453, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.045479835, 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000100396 2024-04-22T13:37:56.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 66, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000019685, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000298 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1642272, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 459728 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-04-22T13:37:56.311 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 62, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.001454357, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000023457 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000004516, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000002258 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-04-22T13:37:56.312 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 6, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 4, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 4, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.002902632, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000725658 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 2, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 11, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-04-22T13:37:56.313 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 14, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 0, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 14, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 14, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 1, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 1, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 1, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 2, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 19, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 13, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 1, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 2, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:37:56.314 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 4, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 234, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 6, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-04-22T13:37:56.315 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 7, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 6, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:37:56.316 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 8, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 1 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 3, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-04-22T13:37:56.317 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 4, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 20, 2024-04-22T13:37:56.318 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 8, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 8, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-04-22T13:37:56.319 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 2, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4196655, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 0, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 7, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.012314421, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001759203 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 0 2024-04-22T13:37:56.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 17, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 17, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 0, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 15, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 16, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 1, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 14, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 15, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 1, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 1, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 3, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 2, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 56000, 2024-04-22T13:37:56.321 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 223608 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 11, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 6, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 25, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 21, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-04-22T13:37:56.323 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2, 2024-04-22T13:37:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.002403530, 2024-04-22T13:37:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001201765 2024-04-22T13:37:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-04-22T13:37:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.325 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000499102, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000249551 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-04-22T13:37:56.326 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-04-22T13:37:56.327 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-04-22T13:37:56.328 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.329 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.329 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.329 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-04-22T13:37:56.329 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.329 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.330 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.331 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.332 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.333 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.334 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:37:56.335 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 4, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 2, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 3, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 1, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 617, 2024-04-22T13:37:56.336 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:56.337 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:56.338 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 262423, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 85, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-04-22T13:37:56.339 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8032, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 167, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 68141, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 301, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-04-22T13:37:56.340 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 50, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 13529, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 33, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 50, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 50, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.240993643, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.004819872 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.341 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 50, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 78 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 50, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 0, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 50, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 0, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 10, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 0, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 7, 2024-04-22T13:37:56.342 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 8, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 6, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 20, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-04-22T13:37:56.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-04-22T13:37:56.344 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-04-22T13:37:56.345 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 101, 2024-04-22T13:37:56.346 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-04-22T13:37:56.346 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 12, 2024-04-22T13:37:56.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 9, 2024-04-22T13:37:56.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 9, 2024-04-22T13:37:56.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-04-22T13:37:56.347 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-04-22T13:37:56.347 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 25, 2024-04-22T13:37:56.347 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 0, 2024-04-22T13:37:56.347 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 2 2024-04-22T13:37:56.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.347 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-04-22T13:37:56.348 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 1, 2024-04-22T13:37:56.348 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:37:56.348 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-04-22T13:37:56.348 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-04-22T13:37:56.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.348 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-04-22T13:37:56.349 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 24 2024-04-22T13:37:56.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.349 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-04-22T13:37:56.349 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-04-22T13:37:56.349 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:37:56.349 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-04-22T13:37:56.350 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-04-22T13:37:56.350 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-04-22T13:37:56.350 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-04-22T13:37:56.350 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-04-22T13:37:56.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.350 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:56.351 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:56.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:37:56.351 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:56.351 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 933, 2024-04-22T13:37:56.351 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1724623, 2024-04-22T13:37:56.351 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 933, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 933, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1724623, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:56.352 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 50, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 13529, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 50, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 13529, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.353 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 50, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 50, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 50, 2024-04-22T13:37:56.354 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 50, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:56.355 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x5616b2948460": { 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:56.356 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 8, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 2351, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 8, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 7, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2351, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:56.357 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.358 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:56.358 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-22T13:37:57.014 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:56 smithi049 ceph-mon[31379]: pgmap v335: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:57.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:56 smithi107 ceph-mon[31859]: pgmap v335: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:57.042 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-22T13:37:57.042 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-22T13:37:57.042 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-22T13:37:57.042 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:37:57.043 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:37:57.043 DEBUG:tasks.check_counter:Getting stats from h 2024-04-22T13:37:57.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-22T13:37:57.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:56 smithi005 ceph-mon[28006]: pgmap v335: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:37:57.291 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:37:58.924 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:37:58 smithi049 ceph-mon[31379]: pgmap v336: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 738, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 857, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1785585, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 404290, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 9, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.119050983, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.045557261, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.068866597, 2024-04-22T13:37:58.950 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.010400004, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 857, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.046428362, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000054175 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 976, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000248729, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000254 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1653728, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 138176 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 61, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 221, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 11468, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 901959, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 49, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.028447450, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.009762481, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.015565811, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000350377, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 218, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.032418426, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000148708 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.952 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 59, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000006454, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000109 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 451168 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 31, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 47, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 7038, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 89419, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 13, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.008769033, 2024-04-22T13:37:58.953 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.001447121, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.005733323, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.001246665, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 45, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.015651654, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000347814 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 36, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000009288, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000258 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 5152 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.954 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 46, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.002187570, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000047555 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-04-22T13:37:58.955 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000007355, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000003677 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 3, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 1, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000251969, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000251969 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 2, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-04-22T13:37:58.956 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 11, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 11, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 11, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 10, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 2, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 13, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 10, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 1, 2024-04-22T13:37:58.957 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 4, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 234, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 6, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-04-22T13:37:58.958 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 2 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.959 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 2, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-04-22T13:37:58.960 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 4, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 8, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 7, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 7, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-04-22T13:37:58.961 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 2, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4196347, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 0, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 6, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.010953344, 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.001825557 2024-04-22T13:37:58.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 0 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 14, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 14, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 0, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 12, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 13, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 1, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 11, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 11, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 0, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 1, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 1, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 52460, 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 223608 2024-04-22T13:37:58.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 4, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 3, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 16, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 8, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000251969, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000251969 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-04-22T13:37:58.964 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.965 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.966 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-04-22T13:37:58.967 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.968 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.969 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.970 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-04-22T13:37:58.971 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.972 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-04-22T13:37:58.973 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 1, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 3, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 2, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 1, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 1, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 1, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 594, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:37:58.974 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:37:58.975 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:37:58.976 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 255127, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 87, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-04-22T13:37:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8032, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 167, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 55088, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 214, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-04-22T13:37:58.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 44, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 12525, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 33, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 44, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 44, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.246993456, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.005613487 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 44, 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 66 2024-04-22T13:37:58.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 44, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 44, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 10, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 6, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 9, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 6, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-04-22T13:37:58.980 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 20, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-04-22T13:37:58.981 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 101, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 12, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 9, 2024-04-22T13:37:58.982 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 9, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 0, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 14, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 0, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 1 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 1, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 12 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-04-22T13:37:58.983 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 826, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1711842, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 826, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:58.984 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 826, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1711842, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:58.985 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 44, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 12525, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 44, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 12525, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:58.986 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 44, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 44, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 44, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 44, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-04-22T13:37:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x5609b35d8460": { 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:37:58.988 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 7, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 2043, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 7, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 6, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2043, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:37:58.989 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-22T13:37:59.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:37:58 smithi107 ceph-mon[31859]: pgmap v336: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:59.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:37:58 smithi005 ceph-mon[28006]: pgmap v336: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:37:59.889 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-22T13:37:59.889 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-22T13:37:59.889 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-22T13:37:59.889 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:37:59.889 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:37:59.889 DEBUG:tasks.check_counter:Getting stats from k 2024-04-22T13:37:59.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-22T13:38:00.139 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:38:00.770 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:00 smithi049 ceph-mon[31379]: pgmap v337: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:38:01.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:00 smithi107 ceph-mon[31859]: pgmap v337: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:38:01.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:00 smithi005 ceph-mon[28006]: pgmap v337: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:38:01.895 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-22T13:38:01.895 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:38:01.895 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001567334, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000532953, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:38:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 165, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 320, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 185120, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-04-22T13:38:01.897 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.012373809, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.008074814, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001815196, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 162, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.016312626, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000100695 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.898 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 185120 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 231, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 218, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1594272, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 132992, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.046799947, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.011605206, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.037818935, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.006012216, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 218, 2024-04-22T13:38:01.899 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.012402657, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000056892 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1594272, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 132992 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.900 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:38:01.901 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:38:01.902 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148977, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:38:01.903 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-04-22T13:38:01.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 229, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1559265, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 229, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 229, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1559265, 2024-04-22T13:38:01.905 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-04-22T13:38:01.906 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-04-22T13:38:01.906 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-04-22T13:38:01.906 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-04-22T13:38:01.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:38:01.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-22T13:38:01.906 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-22T13:38:02.565 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-22T13:38:02.565 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-22T13:38:02.565 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-22T13:38:02.565 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-22T13:38:02.565 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-22T13:38:02.565 DEBUG:tasks.check_counter:Getting stats from c 2024-04-22T13:38:02.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-22T13:38:02.815 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:38:02.932 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:02 smithi107 ceph-mon[31859]: pgmap v338: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:38:03.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:02 smithi049 ceph-mon[31379]: pgmap v338: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:38:03.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:02 smithi005 ceph-mon[28006]: pgmap v338: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:38:04.522 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 479, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 603, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 137049, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 236203, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 9, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.061039540, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.023271080, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.027456945, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004682163, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 602, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.040951462, 2024-04-22T13:38:04.523 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000068025 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 996, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000257822, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000258 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 39, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 232, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 8952, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 567035, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 19, 2024-04-22T13:38:04.524 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.017602122, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.008148734, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.007449043, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000221695, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 229, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.020273397, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000088530 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 72, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000011790, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000163 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 5296, 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 462496 2024-04-22T13:38:04.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 334, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 322, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1647844, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 596036, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 44, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.064118166, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.016662860, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.049595104, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.006049965, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 321, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.028064212, 2024-04-22T13:38:04.526 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000087427 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 60, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000006563, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000109 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1638272, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 143216 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:04.527 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 43, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.002143512, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000049849 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000007626, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000003813 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 10, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 6, 2024-04-22T13:38:04.528 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001743758, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000290626 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 4, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 11, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 12, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 12, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 11, 2024-04-22T13:38:04.529 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 1, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 23, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 20, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 2, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-04-22T13:38:04.530 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-04-22T13:38:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 4, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 4, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 3, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-04-22T13:38:04.532 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 4, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 12, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:38:04.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 7, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 7, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 2, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-04-22T13:38:04.534 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4196347, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.010249981, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001708330 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 15, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 15, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-04-22T13:38:04.535 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 13, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 14, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 1, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 12, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 12, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 1, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 4, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 3, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 52996, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 223608 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 12, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 10, 2024-04-22T13:38:04.536 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 17, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 12, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000763526, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000190881 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000980232, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000490116 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-04-22T13:38:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 3, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 2, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-04-22T13:38:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 4, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 1, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 601, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:38:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-22T13:38:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 263287, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 92, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-22T13:38:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8032, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 167, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 59696, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 235, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-22T13:38:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 45, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 12660, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 28, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 45, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 45, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.215996364, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004799919 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 45, 2024-04-22T13:38:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 69 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 45, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 45, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 6, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 8, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 6, 2024-04-22T13:38:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:38:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-04-22T13:38:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 101, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 12, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 10, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 10, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 17, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 2 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-04-22T13:38:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 16 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-04-22T13:38:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 848, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1699435, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 848, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 848, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1699435, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 45, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 12660, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 45, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 12660, 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 45, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 45, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 45, 2024-04-22T13:38:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 45, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-04-22T13:38:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55d98ffae460": { 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 7, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 2043, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 7, 2024-04-22T13:38:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 6, 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2043, 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:04.563 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T13:38:04.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:04 smithi107 ceph-mon[31859]: pgmap v339: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:38:05.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:04 smithi049 ceph-mon[31379]: pgmap v339: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:38:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:04 smithi005 ceph-mon[28006]: pgmap v339: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:38:05.302 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-22T13:38:05.302 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-22T13:38:05.302 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-22T13:38:05.302 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:38:05.302 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:38:05.302 DEBUG:tasks.check_counter:Getting stats from f 2024-04-22T13:38:05.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-22T13:38:05.548 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:38:06.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:06 smithi107 ceph-mon[31859]: pgmap v340: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:38:06.784 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:06 smithi107 ceph-mon[31859]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T13:38:07.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:06 smithi049 ceph-mon[31379]: pgmap v340: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:38:07.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:06 smithi049 ceph-mon[31379]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T13:38:07.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:06 smithi005 ceph-mon[28006]: pgmap v340: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:38:07.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:06 smithi005 ceph-mon[28006]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1131, 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1133, 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 885987, 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 359668, 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-04-22T13:38:07.077 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.110587628, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.035518615, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.028750510, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.035373078, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1131, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.047193812, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000041727 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.078 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 5, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 113, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 2511, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 279050, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.007408269, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.004828438, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001295681, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000104091, 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:07.079 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 111, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.008743555, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000078770 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 277712 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 693, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 691, 2024-04-22T13:38:07.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 337049, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 259597, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.074886176, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.028396448, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.024087342, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.020520775, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 691, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.029598564, 2024-04-22T13:38:07.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000042834 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 206432, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 91056 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-22T13:38:07.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2724, 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.077786733, 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000028556 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:07.083 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 23, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 23, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 23, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-04-22T13:38:07.085 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-04-22T13:38:07.086 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-04-22T13:38:07.087 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:38:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-04-22T13:38:07.089 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-04-22T13:38:07.090 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-04-22T13:38:07.091 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 143, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 5, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4595162, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4837950, 2024-04-22T13:38:07.092 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 4837890, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 143 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 14, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 18, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 4, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 9, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 9, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-04-22T13:38:07.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 23, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 23, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 52924, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215416 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-04-22T13:38:07.094 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-04-22T13:38:07.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.099 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.099 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-04-22T13:38:07.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.100 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-04-22T13:38:07.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-04-22T13:38:07.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-04-22T13:38:07.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.104 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.105 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 0, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 1908, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-22T13:38:07.107 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:38:07.108 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:38:07.109 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 189756, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 60, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:38:07.110 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8152, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 170, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 71216, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 639, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-22T13:38:07.111 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 1665, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 26, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 4, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 1665, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1665, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.065981094, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000640228 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-04-22T13:38:07.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1665, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1666 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 1665, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 1665, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1058, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 604, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-04-22T13:38:07.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 2, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-04-22T13:38:07.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-04-22T13:38:07.115 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-04-22T13:38:07.116 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 102, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 9, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 5, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 5, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-04-22T13:38:07.117 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-04-22T13:38:07.118 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 1825, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1050152, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 1825, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 1825, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1050152, 2024-04-22T13:38:07.119 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:07.120 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 1665, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 550910, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 1665, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 550910, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-04-22T13:38:07.121 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 1665, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 1665, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 1665, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1665, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.122 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:07.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x56550b3d4640": { 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-04-22T13:38:07.124 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:07.125 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T13:38:07.759 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-22T13:38:07.759 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-22T13:38:07.759 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-22T13:38:07.759 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:38:07.759 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:38:07.759 DEBUG:tasks.check_counter:Getting stats from i 2024-04-22T13:38:07.759 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-22T13:38:07.998 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:38:09.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:08 smithi049 ceph-mon[31379]: pgmap v342: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:38:09.033 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:08 smithi107 ceph-mon[31859]: pgmap v342: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:38:09.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:08 smithi005 ceph-mon[28006]: pgmap v342: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 308, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 325, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1653997, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 201508, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 12, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.061372274, 2024-04-22T13:38:09.584 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.018944719, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.040221264, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.005741936, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 321, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.022849320, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000071181 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 50, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000011340, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000226 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1648176, 2024-04-22T13:38:09.585 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 139184 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 62, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 220, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 13168, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 955626, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 50, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.025981794, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.007727877, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.014930876, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000413459, 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:09.586 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 218, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.026879027, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000123298 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 42, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000003942, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000093 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 450848 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 423, 2024-04-22T13:38:09.587 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 573, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 126007, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 208768, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.057471590, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.023451385, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.026560819, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.005224309, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 574, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.037295031, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000064973 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 945, 2024-04-22T13:38:09.588 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000306780, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000324 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 48, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001414771, 2024-04-22T13:38:09.589 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000029474 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000009144, 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000004572 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 2, 2024-04-22T13:38:09.590 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 1, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000158517, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000158517 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 1, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 11, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 10, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-04-22T13:38:09.591 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 10, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 10, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 1, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 8, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 8, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-04-22T13:38:09.592 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 1, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:38:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 1, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 1, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-04-22T13:38:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 1, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:38:09.595 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 4, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 4, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 3, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-04-22T13:38:09.596 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 3, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 2, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195200, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-04-22T13:38:09.597 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003929550, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001964775 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 13, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 13, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 11, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 12, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 1, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 10, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 10, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-04-22T13:38:09.598 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 1, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 1, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 52528, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 223608 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 2, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 2, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 9, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 4, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.599 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000158517, 2024-04-22T13:38:09.601 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000158517 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.602 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.603 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-04-22T13:38:09.605 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.606 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.607 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.608 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.610 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.611 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 1, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-04-22T13:38:09.612 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 1, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 1, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 642, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:38:09.613 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:38:09.614 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 238087, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 72, 2024-04-22T13:38:09.615 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8152, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 170, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 51248, 2024-04-22T13:38:09.616 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 201, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 37, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 11108, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 33, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 37, 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-04-22T13:38:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 37, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.215996379, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005837739 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 37, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 59 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 37, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 0, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 37, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-04-22T13:38:09.618 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 2, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 6, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 6, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-04-22T13:38:09.619 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-04-22T13:38:09.620 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-04-22T13:38:09.621 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 102, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 13, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 8, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 8, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 14, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 1 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-04-22T13:38:09.622 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 8 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-04-22T13:38:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 789, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1705004, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 789, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 789, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1705004, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:09.625 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 37, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 11108, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 37, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 11108, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.626 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 37, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 37, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 37, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 37, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.627 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-04-22T13:38:09.628 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x56014d266460": { 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 896, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:09.629 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2, 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 896, 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:09.630 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T13:38:10.313 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-22T13:38:10.314 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-22T13:38:10.314 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-22T13:38:10.314 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:38:10.314 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:38:10.314 DEBUG:tasks.check_counter:Getting stats from l 2024-04-22T13:38:10.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-22T13:38:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:38:11.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:10 smithi049 ceph-mon[31379]: pgmap v343: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:38:11.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:10 smithi107 ceph-mon[31859]: pgmap v343: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:38:11.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:10 smithi005 ceph-mon[28006]: pgmap v343: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2242, 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2244, 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 636242, 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 712963, 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-04-22T13:38:12.196 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.192779344, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.065796118, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.046036623, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.065480045, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2242, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.083966472, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000037451 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:38:12.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 170, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 876, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 453770, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.010366640, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.006603398, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001847053, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000102926, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 167, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.012934168, 2024-04-22T13:38:12.198 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000077450 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 453072 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 256, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 241, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1632828, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 138990, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-04-22T13:38:12.199 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.049375059, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.015229101, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.037469633, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.005676760, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 241, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.014668396, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000060864 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1632560, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 138672 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.200 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3795, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.103123815, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000027173 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.201 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-04-22T13:38:12.202 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-04-22T13:38:12.203 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:38:12.204 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:38:12.205 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-04-22T13:38:12.206 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-04-22T13:38:12.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 7, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-04-22T13:38:12.208 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4196347, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 4196206, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 7 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 3, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 3, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-04-22T13:38:12.209 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 1, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 49984, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215416 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-04-22T13:38:12.210 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-04-22T13:38:12.211 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.213 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-04-22T13:38:12.214 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.215 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-04-22T13:38:12.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-04-22T13:38:12.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-04-22T13:38:12.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-04-22T13:38:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-04-22T13:38:12.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 3, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 2, 2024-04-22T13:38:12.221 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 2, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 3174, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:38:12.222 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-22T13:38:12.223 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:38:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 181330, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 48, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8152, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 170, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-22T13:38:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 8512, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 13, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 2245, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-04-22T13:38:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 2245, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2245, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.994982320, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000443199 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2245, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2246 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 2245, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 2245, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-04-22T13:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1461, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 782, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-04-22T13:38:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-04-22T13:38:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 102, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 13, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 4, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 4, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-04-22T13:38:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-04-22T13:38:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2498, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 2029268, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2498, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2498, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2029268, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 2245, 2024-04-22T13:38:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 2048, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2245, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2048, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 2245, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 2245, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2245, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2245, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:38:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55af3ac646e0": { 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-22T13:38:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T13:38:12.238 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T13:38:12.926 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:12 smithi107 ceph-mon[31859]: pgmap v344: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:38:12.926 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:12 smithi107 ceph-mon[31859]: osdmap e103: 12 total, 12 up, 12 in 2024-04-22T13:38:12.952 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-22T13:38:12.952 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-22T13:38:12.952 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-22T13:38:12.953 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:38:12.953 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:38:12.953 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-22T13:38:12.965 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-22T13:38:12.965 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:38:12.965 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T13:38:12.987 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-22T13:38:12.988 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T13:38:13.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:12 smithi049 ceph-mon[31379]: pgmap v344: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:38:13.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:12 smithi049 ceph-mon[31379]: osdmap e103: 12 total, 12 up, 12 in 2024-04-22T13:38:13.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:38:13.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:12 smithi005 ceph-mon[28006]: pgmap v344: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:38:13.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:12 smithi005 ceph-mon[28006]: osdmap e103: 12 total, 12 up, 12 in 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.169+0000 7f3fe029a640 1 -- 172.21.15.5:0/2714988439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 msgr2=0x7f3fd810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.169+0000 7f3fe029a640 1 --2- 172.21.15.5:0/2714988439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd810ee70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009ca0 tx=0x7f3fd002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 -- 172.21.15.5:0/2714988439 shutdown_connections 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 --2- 172.21.15.5:0/2714988439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd810ee70 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 --2- 172.21.15.5:0/2714988439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069f10 0x7f3fd8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 --2- 172.21.15.5:0/2714988439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd8069560 0x7f3fd8069940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 -- 172.21.15.5:0/2714988439 >> 172.21.15.5:0/2714988439 conn(0x7f3fd80771d0 msgr2=0x7f3fd80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 -- 172.21.15.5:0/2714988439 shutdown_connections 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 -- 172.21.15.5:0/2714988439 wait complete. 2024-04-22T13:38:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 Processor -- start 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 -- start start 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069560 0x7f3fd8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd819cf10 0x7f3fd819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.170+0000 7f3fe029a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd81121b0 con 0x7f3fd81072c0 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fe029a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8112030 con 0x7f3fd819cf10 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fe029a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8112330 con 0x7f3fd8069560 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49446/0 (socket says 172.21.15.5:49446) 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 -- 172.21.15.5:0/1444726895 learned_addr learned my addr 172.21.15.5:0/1444726895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fde837640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd819cf10 0x7f3fd819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fde036640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069560 0x7f3fd8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 -- 172.21.15.5:0/1444726895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069560 msgr2=0x7f3fd8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069560 0x7f3fd8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 -- 172.21.15.5:0/1444726895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd819cf10 msgr2=0x7f3fd819d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd819cf10 0x7f3fd819d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 -- 172.21.15.5:0/1444726895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd819da00 con 0x7f3fd81072c0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fde036640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069560 0x7f3fd8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fdd835640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd8198b90 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400d740 tx=0x7f3fd400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4017070 con 0x7f3fd81072c0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd400de50 con 0x7f3fd81072c0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd819dcf0 con 0x7f3fd81072c0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.171+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400f620 con 0x7f3fd81072c0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.172+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd81a54e0 con 0x7f3fd81072c0 2024-04-22T13:38:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.172+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd8068e00 con 0x7f3fd81072c0 2024-04-22T13:38:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.174+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f3fd401a070 con 0x7f3fd81072c0 2024-04-22T13:38:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.175+0000 7f3fcb7fe640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fc40a3d20 0x7f3fc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.175+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f3fd40d0270 con 0x7f3fd81072c0 2024-04-22T13:38:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.175+0000 7f3fde036640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fc40a3d20 0x7f3fc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.176+0000 7f3fde036640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fc40a3d20 0x7f3fc40a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0058c0 tx=0x7f3fcc005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.180+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fd409c8b0 con 0x7f3fd81072c0 2024-04-22T13:38:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.369+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fd8069f10 con 0x7f3fd81072c0 2024-04-22T13:38:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.371+0000 7f3fcb7fe640 1 -- 172.21.15.5:0/1444726895 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23014 (secure 0 0 0) 0x7f3fd409c1b0 con 0x7f3fd81072c0 2024-04-22T13:38:13.368 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:38:13.368 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":103,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:38:12.403440+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:38:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fc40a3d20 msgr2=0x7f3fc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fc40a3d20 0x7f3fc40a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0058c0 tx=0x7f3fcc005850 comp rx=0 tx=0).stop 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 msgr2=0x7f3fd8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd8198b90 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400d740 tx=0x7f3fd400dc00 comp rx=0 tx=0).stop 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 shutdown_connections 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd81072c0 0x7f3fd8198b90 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f3fc40a3d20 0x7f3fc40a61e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fd819cf10 0x7f3fd819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 --2- 172.21.15.5:0/1444726895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fd8069560 0x7f3fd8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 >> 172.21.15.5:0/1444726895 conn(0x7f3fd80771d0 msgr2=0x7f3fd80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 shutdown_connections 2024-04-22T13:38:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.373+0000 7f3fe029a640 1 -- 172.21.15.5:0/1444726895 wait complete. 2024-04-22T13:38:13.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:20:42.191953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:25:08.431891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:25:09.192645+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:38:13.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:38:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.496+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3013558374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 msgr2=0x7f97d010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3013558374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d010e9a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009ca0 tx=0x7f97c002c180 comp rx=0 tx=0).stop 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3013558374 shutdown_connections 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3013558374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d010e9a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3013558374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d0103ae0 0x7f97d010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3013558374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d01031c0 0x7f97d01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3013558374 >> 172.21.15.5:0/3013558374 conn(0x7f97d0078000 msgr2=0x7f97d0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3013558374 shutdown_connections 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3013558374 wait complete. 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 Processor -- start 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.497+0000 7f97d4ac1640 1 -- start start 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97d4ac1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d01031c0 0x7f97d01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97d4ac1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d0103ae0 0x7f97d0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97d4ac1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97d4ac1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0112190 con 0x7f97d010c500 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97d4ac1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0112010 con 0x7f97d01031c0 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97d4ac1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0112310 con 0x7f97d0103ae0 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49462/0 (socket says 172.21.15.5:49462) 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 -- 172.21.15.5:0/3660987357 learned_addr learned my addr 172.21.15.5:0/3660987357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97cdd9b640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d0103ae0 0x7f97d0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ce59c640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d01031c0 0x7f97d01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 -- 172.21.15.5:0/3660987357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d0103ae0 msgr2=0x7f97d0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d0103ae0 0x7f97d0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 -- 172.21.15.5:0/3660987357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d01031c0 msgr2=0x7f97d01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d01031c0 0x7f97d01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 -- 172.21.15.5:0/3660987357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d019d6c0 con 0x7f97d010c500 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97ced9d640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d019cf90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f97c002c790 tx=0x7f97c0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.498+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0039040 con 0x7f97d010c500 2024-04-22T13:38:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.499+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97c0044070 con 0x7f97d010c500 2024-04-22T13:38:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.499+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d019d950 con 0x7f97d010c500 2024-04-22T13:38:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.499+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0032850 con 0x7f97d010c500 2024-04-22T13:38:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.499+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d01a5240 con 0x7f97d010c500 2024-04-22T13:38:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.500+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d0068e00 con 0x7f97d010c500 2024-04-22T13:38:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.501+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f97c0004240 con 0x7f97d010c500 2024-04-22T13:38:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.502+0000 7f97bf7fe640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f97b80a3d20 0x7f97b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.502+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f97c00f33e0 con 0x7f97d010c500 2024-04-22T13:38:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.502+0000 7f97ce59c640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f97b80a3d20 0x7f97b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.503+0000 7f97ce59c640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f97b80a3d20 0x7f97b80a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97c8005fd0 tx=0x7f97c8005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.506+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97c00bfaf0 con 0x7f97d010c500 2024-04-22T13:38:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.694+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97d0107680 con 0x7f97d010c500 2024-04-22T13:38:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.695+0000 7f97bf7fe640 1 -- 172.21.15.5:0/3660987357 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f97c00bf3f0 con 0x7f97d010c500 2024-04-22T13:38:13.691 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-22T13:38:13.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f97b80a3d20 msgr2=0x7f97b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f97b80a3d20 0x7f97b80a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97c8005fd0 tx=0x7f97c8005850 comp rx=0 tx=0).stop 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 msgr2=0x7f97d019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d019cf90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f97c002c790 tx=0x7f97c0003c20 comp rx=0 tx=0).stop 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 shutdown_connections 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d010c500 0x7f97d019cf90 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f97b80a3d20 0x7f97b80a61e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d0103ae0 0x7f97d0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 --2- 172.21.15.5:0/3660987357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d01031c0 0x7f97d01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 >> 172.21.15.5:0/3660987357 conn(0x7f97d0078000 msgr2=0x7f97d00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 shutdown_connections 2024-04-22T13:38:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.697+0000 7f97d4ac1640 1 -- 172.21.15.5:0/3660987357 wait complete. 2024-04-22T13:38:13.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:38:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.814+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3274427431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d78101090 msgr2=0x7f6d78101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.814+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3274427431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d78101090 0x7f6d78101470 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009900 tx=0x7f6d6802bc30 comp rx=0 tx=0).stop 2024-04-22T13:38:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3274427431 shutdown_connections 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3274427431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d78106160 0x7f6d7810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3274427431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d78101a40 0x7f6d78105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3274427431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d78101090 0x7f6d78101470 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3274427431 >> 172.21.15.5:0/3274427431 conn(0x7f6d78078000 msgr2=0x7f6d78078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3274427431 shutdown_connections 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3274427431 wait complete. 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 Processor -- start 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.815+0000 7f6d7daf2640 1 -- start start 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d7daf2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d78101090 0x7f6d7819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d7daf2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d78106160 0x7f6d7819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d7daf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 0x7f6d78198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d7daf2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781121e0 con 0x7f6d781985a0 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d7daf2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78112060 con 0x7f6d78101090 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d7daf2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78112360 con 0x7f6d78106160 2024-04-22T13:38:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 0x7f6d78198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 0x7f6d78198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49478/0 (socket says 172.21.15.5:49478) 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 -- 172.21.15.5:0/3159760201 learned_addr learned my addr 172.21.15.5:0/3159760201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d777fe640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d78101090 0x7f6d7819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d76ffd640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d78106160 0x7f6d7819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 -- 172.21.15.5:0/3159760201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d78106160 msgr2=0x7f6d7819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d78106160 0x7f6d7819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 -- 172.21.15.5:0/3159760201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d78101090 msgr2=0x7f6d7819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d78101090 0x7f6d7819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 -- 172.21.15.5:0/3159760201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d781990c0 con 0x7f6d781985a0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d77fff640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 0x7f6d78198a50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000da40 tx=0x7f6d7000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.816+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70017070 con 0x7f6d781985a0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.817+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d781993b0 con 0x7f6d781985a0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.817+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d70003e20 con 0x7f6d781985a0 2024-04-22T13:38:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.817+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70010460 con 0x7f6d781985a0 2024-04-22T13:38:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.817+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d781a5600 con 0x7f6d781985a0 2024-04-22T13:38:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.818+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d78068e00 con 0x7f6d781985a0 2024-04-22T13:38:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.819+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6d70004020 con 0x7f6d781985a0 2024-04-22T13:38:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.819+0000 7f6d74ff9640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6d600a3d20 0x7f6d600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.820+0000 7f6d777fe640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6d600a3d20 0x7f6d600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.820+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f6d700d03e0 con 0x7f6d781985a0 2024-04-22T13:38:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.820+0000 7f6d777fe640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6d600a3d20 0x7f6d600a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6d68006010 tx=0x7f6d68005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:13.823+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d7009c430 con 0x7f6d781985a0 2024-04-22T13:38:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.012+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d78101a40 con 0x7f6d781985a0 2024-04-22T13:38:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.012+0000 7f6d74ff9640 1 -- 172.21.15.5:0/3159760201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f6d700a0730 con 0x7f6d781985a0 2024-04-22T13:38:14.008 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6d600a3d20 msgr2=0x7f6d600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6d600a3d20 0x7f6d600a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6d68006010 tx=0x7f6d68005e40 comp rx=0 tx=0).stop 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 msgr2=0x7f6d78198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 0x7f6d78198a50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000da40 tx=0x7f6d7000df00 comp rx=0 tx=0).stop 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 shutdown_connections 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d781985a0 0x7f6d78198a50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6d600a3d20 0x7f6d600a61e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d78106160 0x7f6d7819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 --2- 172.21.15.5:0/3159760201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d78101090 0x7f6d7819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 >> 172.21.15.5:0/3159760201 conn(0x7f6d78078000 msgr2=0x7f6d780ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 shutdown_connections 2024-04-22T13:38:14.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.014+0000 7f6d7daf2640 1 -- 172.21.15.5:0/3159760201 wait complete. 2024-04-22T13:38:14.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1444726895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:38:14.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:13 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3660987357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:38:14.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1444726895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:38:14.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:13 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3660987357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:38:14.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:38:14.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1444726895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:38:14.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:13 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3660987357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:38:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 -- 172.21.15.5:0/2080925953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 msgr2=0x7f11c4110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/2080925953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c4110a90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009ca0 tx=0x7f11bc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 -- 172.21.15.5:0/2080925953 shutdown_connections 2024-04-22T13:38:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/2080925953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c4110a90 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/2080925953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c41014a0 0x7f11c410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/2080925953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11c4100af0 0x7f11c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 -- 172.21.15.5:0/2080925953 >> 172.21.15.5:0/2080925953 conn(0x7f11c4077a00 msgr2=0x7f11c40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 -- 172.21.15.5:0/2080925953 shutdown_connections 2024-04-22T13:38:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.130+0000 7f11c9b9f640 1 -- 172.21.15.5:0/2080925953 wait complete. 2024-04-22T13:38:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 Processor -- start 2024-04-22T13:38:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 -- start start 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c4100af0 0x7f11c419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11c41014a0 0x7f11c419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41141a0 con 0x7f11c410e6a0 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4114020 con 0x7f11c4100af0 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c9b9f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4114320 con 0x7f11c41014a0 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c939e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c939e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49508/0 (socket says 172.21.15.5:49508) 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.131+0000 7f11c939e640 1 -- 172.21.15.5:0/4013460177 learned_addr learned my addr 172.21.15.5:0/4013460177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c8b9d640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c4100af0 0x7f11c419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c3fff640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11c41014a0 0x7f11c419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c939e640 1 -- 172.21.15.5:0/4013460177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11c41014a0 msgr2=0x7f11c419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c939e640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11c41014a0 0x7f11c419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c939e640 1 -- 172.21.15.5:0/4013460177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c4100af0 msgr2=0x7f11c419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c939e640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c4100af0 0x7f11c419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c939e640 1 -- 172.21.15.5:0/4013460177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c419f6b0 con 0x7f11c410e6a0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c8b9d640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c4100af0 0x7f11c419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c939e640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c419ef80 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f11bc033000 tx=0x7f11bc0346e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc041020 con 0x7f11c410e6a0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11bc032970 con 0x7f11c410e6a0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc039040 con 0x7f11c410e6a0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c419f940 con 0x7f11c410e6a0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.132+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c41a7230 con 0x7f11c410e6a0 2024-04-22T13:38:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.133+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c404fce0 con 0x7f11c410e6a0 2024-04-22T13:38:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.135+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f11bc044020 con 0x7f11c410e6a0 2024-04-22T13:38:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.135+0000 7f11c1ffb640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11b40a3d20 0x7f11b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.136+0000 7f11c8b9d640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11b40a3d20 0x7f11b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.136+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f11bc0f29b0 con 0x7f11c410e6a0 2024-04-22T13:38:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.136+0000 7f11c8b9d640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11b40a3d20 0x7f11b40a61e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11b80058c0 tx=0x7f11b8005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.143+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11bc04b020 con 0x7f11c410e6a0 2024-04-22T13:38:14.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.330+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f11c4109830 con 0x7f11c410e6a0 2024-04-22T13:38:14.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.331+0000 7f11c1ffb640 1 -- 172.21.15.5:0/4013460177 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f11bc0bf220 con 0x7f11c410e6a0 2024-04-22T13:38:14.328 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-22T13:38:14.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11b40a3d20 msgr2=0x7f11b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11b40a3d20 0x7f11b40a61e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11b80058c0 tx=0x7f11b8005850 comp rx=0 tx=0).stop 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 msgr2=0x7f11c419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c419ef80 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f11bc033000 tx=0x7f11bc0346e0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 shutdown_connections 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c410e6a0 0x7f11c419ef80 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f11b40a3d20 0x7f11b40a61e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11c41014a0 0x7f11c419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 --2- 172.21.15.5:0/4013460177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11c4100af0 0x7f11c419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.333+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 >> 172.21.15.5:0/4013460177 conn(0x7f11c4077a00 msgr2=0x7f11c40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.334+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 shutdown_connections 2024-04-22T13:38:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.334+0000 7f11c9b9f640 1 -- 172.21.15.5:0/4013460177 wait complete. 2024-04-22T13:38:14.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 -- 172.21.15.5:0/289813757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38103ae0 msgr2=0x7f2a3810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/289813757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38103ae0 0x7f2a3810bfc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009900 tx=0x7f2a2802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 -- 172.21.15.5:0/289813757 shutdown_connections 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/289813757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a38103ae0 0x7f2a3810bfc0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/289813757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a3810c500 0x7f2a3810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/289813757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a381031c0 0x7f2a381035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 -- 172.21.15.5:0/289813757 >> 172.21.15.5:0/289813757 conn(0x7f2a38078000 msgr2=0x7f2a38078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 -- 172.21.15.5:0/289813757 shutdown_connections 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 -- 172.21.15.5:0/289813757 wait complete. 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 Processor -- start 2024-04-22T13:38:14.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.449+0000 7f2a3fd75640 1 -- start start 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 0x7f2a38198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38103ae0 0x7f2a38198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a3810c500 0x7f2a3819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38112190 con 0x7f2a381031c0 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38112010 con 0x7f2a38103ae0 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38112310 con 0x7f2a3810c500 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 0x7f2a38198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 0x7f2a38198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49524/0 (socket says 172.21.15.5:49524) 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 -- 172.21.15.5:0/3700817460 learned_addr learned my addr 172.21.15.5:0/3700817460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3d310640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38103ae0 0x7f2a38198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3e312640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a3810c500 0x7f2a3819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 -- 172.21.15.5:0/3700817460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a3810c500 msgr2=0x7f2a3819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a3810c500 0x7f2a3819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 -- 172.21.15.5:0/3700817460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38103ae0 msgr2=0x7f2a38198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38103ae0 0x7f2a38198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 -- 172.21.15.5:0/3700817460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3819d700 con 0x7f2a381031c0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3db11640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 0x7f2a38198710 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9f0 tx=0x7f2a3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400cd80 con 0x7f2a381031c0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a340108e0 con 0x7f2a381031c0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3819d9f0 con 0x7f2a381031c0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34018980 con 0x7f2a381031c0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.450+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a381a5280 con 0x7f2a381031c0 2024-04-22T13:38:14.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.451+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a38068e00 con 0x7f2a381031c0 2024-04-22T13:38:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.452+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2a34010430 con 0x7f2a381031c0 2024-04-22T13:38:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.453+0000 7f2a2effd640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2a240a3da0 0x7f2a240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.453+0000 7f2a3d310640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2a240a3da0 0x7f2a240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.453+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f2a340d1070 con 0x7f2a381031c0 2024-04-22T13:38:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.454+0000 7f2a3d310640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2a240a3da0 0x7f2a240a6260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a38100910 tx=0x7f2a2802c730 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.457+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a3409d710 con 0x7f2a381031c0 2024-04-22T13:38:14.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.657+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a381076b0 con 0x7f2a381031c0 2024-04-22T13:38:14.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.658+0000 7f2a2effd640 1 -- 172.21.15.5:0/3700817460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+930 (secure 0 0 0) 0x7f2a3409d010 con 0x7f2a381031c0 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2674774949 2024-04-23T13:27:09.442003+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2674774949 2024-04-23T13:27:09.442003+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3327695745 2024-04-23T13:27:09.442003+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1240644936 2024-04-23T13:27:09.442003+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1744519677 2024-04-23T13:18:14.098934+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3837385355 2024-04-23T13:27:09.442003+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1556657643 2024-04-23T13:18:14.098934+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2261038189 2024-04-23T13:17:46.974612+0000 2024-04-22T13:38:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/4095818300 2024-04-23T13:18:14.098934+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1954020614 2024-04-23T13:17:46.974612+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1797990010 2024-04-23T13:18:14.098934+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3443029620 2024-04-23T13:17:46.974612+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3443029620 2024-04-23T13:17:46.974612+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1011902624 2024-04-23T13:17:46.974612+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2682698140 2024-04-23T13:27:09.442003+0000 2024-04-22T13:38:14.656 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/4095818300 2024-04-23T13:18:14.098934+0000 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2a240a3da0 msgr2=0x7f2a240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2a240a3da0 0x7f2a240a6260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a38100910 tx=0x7f2a2802c730 comp rx=0 tx=0).stop 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 msgr2=0x7f2a38198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 0x7f2a38198710 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9f0 tx=0x7f2a3400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 shutdown_connections 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f2a240a3da0 0x7f2a240a6260 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a3810c500 0x7f2a3819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a38103ae0 0x7f2a38198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 --2- 172.21.15.5:0/3700817460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a381031c0 0x7f2a38198710 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 >> 172.21.15.5:0/3700817460 conn(0x7f2a38078000 msgr2=0x7f2a380fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 shutdown_connections 2024-04-22T13:38:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:14.660+0000 7f2a3fd75640 1 -- 172.21.15.5:0/3700817460 wait complete. 2024-04-22T13:38:14.668 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-22T13:38:14.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:38:14.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-22T13:38:14.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:38:14.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:38:14.709 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T13:38:14.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:38:14.771 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-22T13:38:14.827 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T13:38:14.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:38:14.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T13:38:14.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:38:14.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-22T13:38:14.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:14 smithi005 ceph-mon[28006]: pgmap v346: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:38:14.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3159760201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:38:14.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/4013460177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:38:14.956 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:14 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3700817460' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T13:38:14.994 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-22T13:38:15.006 INFO:tasks.cephadm:Teardown begin 2024-04-22T13:38:15.006 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-22T13:38:15.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:38:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:14 smithi049 ceph-mon[31379]: pgmap v346: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:38:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:14 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3159760201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:38:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:14 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/4013460177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:38:15.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:14 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3700817460' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T13:38:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:14 smithi107 ceph-mon[31859]: pgmap v346: 97 pgs: 97 active+clean; 1.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:38:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3159760201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:38:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/4013460177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:38:15.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:14 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3700817460' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T13:38:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 -- 172.21.15.5:0/2308211409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc108060 msgr2=0x7f6bdc108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 --2- 172.21.15.5:0/2308211409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc108060 0x7f6bdc108440 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009930 tx=0x7f6bcc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:38:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 -- 172.21.15.5:0/2308211409 shutdown_connections 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 --2- 172.21.15.5:0/2308211409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc06b000 0x7f6bdc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 --2- 172.21.15.5:0/2308211409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bdc06a640 0x7f6bdc06aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 --2- 172.21.15.5:0/2308211409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc108060 0x7f6bdc108440 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 -- 172.21.15.5:0/2308211409 >> 172.21.15.5:0/2308211409 conn(0x7f6bdc077c40 msgr2=0x7f6bdc078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 -- 172.21.15.5:0/2308211409 shutdown_connections 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.140+0000 7f6be25b2640 1 -- 172.21.15.5:0/2308211409 wait complete. 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 Processor -- start 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 -- start start 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc06a640 0x7f6bdc101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 0x7f6bdc1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bdc108060 0x7f6bdc101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1121a0 con 0x7f6bdc06b000 2024-04-22T13:38:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc112020 con 0x7f6bdc06a640 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6be25b2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc112320 con 0x7f6bdc108060 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 0x7f6bdc1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 0x7f6bdc1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49550/0 (socket says 172.21.15.5:49550) 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746381179 learned_addr learned my addr 172.21.15.5:0/1746381179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.141+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc06a640 0x7f6bdc101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6be0b4f640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bdc108060 0x7f6bdc101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746381179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bdc108060 msgr2=0x7f6bdc101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bdc108060 0x7f6bdc101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746381179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc06a640 msgr2=0x7f6bdc101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc06a640 0x7f6bdc101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746381179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a6bc0 con 0x7f6bdc06b000 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 0x7f6bdc1016b0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000d770 tx=0x7f6bd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0017070 con 0x7f6bdc06b000 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd00054d0 con 0x7f6bdc06b000 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a6eb0 con 0x7f6bdc06b000 2024-04-22T13:38:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0004c30 con 0x7f6bdc06b000 2024-04-22T13:38:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.142+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a7320 con 0x7f6bdc06b000 2024-04-22T13:38:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.143+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc103650 con 0x7f6bdc06b000 2024-04-22T13:38:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.144+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6bd0004e10 con 0x7f6bdc06b000 2024-04-22T13:38:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.145+0000 7f6bd97fa640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bc40a3d20 0x7f6bc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.145+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bc40a3d20 0x7f6bc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.145+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f6bd00d02a0 con 0x7f6bdc06b000 2024-04-22T13:38:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.146+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bc40a3d20 0x7f6bc40a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005ab0 tx=0x7f6bcc005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.150+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bd009c940 con 0x7f6bdc06b000 2024-04-22T13:38:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.378+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bdc02d5c0 con 0x7f6bdc06b000 2024-04-22T13:38:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.378+0000 7f6bd97fa640 1 -- 172.21.15.5:0/1746381179 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7f6bd009c240 con 0x7f6bdc06b000 2024-04-22T13:38:15.375 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:38:15.375 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:38:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:38:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.380+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bc40a3d20 msgr2=0x7f6bc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.380+0000 7f6be25b2640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bc40a3d20 0x7f6bc40a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005ab0 tx=0x7f6bcc005fd0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.380+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 msgr2=0x7f6bdc1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.380+0000 7f6be25b2640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 0x7f6bdc1016b0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000d770 tx=0x7f6bd000dc30 comp rx=0 tx=0).stop 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 shutdown_connections 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc06b000 0x7f6bdc1016b0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f6bc40a3d20 0x7f6bc40a61e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bdc108060 0x7f6bdc101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 --2- 172.21.15.5:0/1746381179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc06a640 0x7f6bdc101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 >> 172.21.15.5:0/1746381179 conn(0x7f6bdc077c40 msgr2=0x7f6bdc0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 shutdown_connections 2024-04-22T13:38:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.381+0000 7f6be25b2640 1 -- 172.21.15.5:0/1746381179 wait complete. 2024-04-22T13:38:15.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:38:15.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3476612288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff418103ad0 msgr2=0x7ff41810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3476612288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff418103ad0 0x7ff41810bfb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009900 tx=0x7ff40c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3476612288 shutdown_connections 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3476612288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff418103ad0 0x7ff41810bfb0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3476612288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff41810c580 0x7ff41810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3476612288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4181031b0 0x7ff418103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3476612288 >> 172.21.15.5:0/3476612288 conn(0x7ff4180771d0 msgr2=0x7ff4180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3476612288 shutdown_connections 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3476612288 wait complete. 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 Processor -- start 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.494+0000 7ff41e6f1640 1 -- start start 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41e6f1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4181031b0 0x7ff418100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41e6f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff418103ad0 0x7ff418101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41e6f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 0x7ff418101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41e6f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4181121c0 con 0x7ff41810c580 2024-04-22T13:38:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41e6f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff418112040 con 0x7ff418103ad0 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41e6f1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff418112340 con 0x7ff4181031b0 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 0x7ff418101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 0x7ff418101880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49558/0 (socket says 172.21.15.5:49558) 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 -- 172.21.15.5:0/3671752743 learned_addr learned my addr 172.21.15.5:0/3671752743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff417fff640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4181031b0 0x7ff418100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff4177fe640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff418103ad0 0x7ff418101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 -- 172.21.15.5:0/3671752743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4181031b0 msgr2=0x7ff418100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4181031b0 0x7ff418100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 -- 172.21.15.5:0/3671752743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff418103ad0 msgr2=0x7ff418101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff418103ad0 0x7ff418101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 -- 172.21.15.5:0/3671752743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4181a6be0 con 0x7ff41810c580 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.495+0000 7ff41cc8e640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 0x7ff418101880 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff41000ccb0 tx=0x7ff41000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.496+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41000f450 con 0x7ff41810c580 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.496+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41000faa0 con 0x7ff41810c580 2024-04-22T13:38:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.496+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4181a6ed0 con 0x7ff41810c580 2024-04-22T13:38:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.496+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410018b00 con 0x7ff41810c580 2024-04-22T13:38:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.496+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4181a73a0 con 0x7ff41810c580 2024-04-22T13:38:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.497+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41806ae60 con 0x7ff41810c580 2024-04-22T13:38:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.498+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff41000fc50 con 0x7ff41810c580 2024-04-22T13:38:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.499+0000 7ff4157fa640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4000a3d20 0x7ff4000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.499+0000 7ff417fff640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4000a3d20 0x7ff4000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.499+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7ff4100d0e50 con 0x7ff41810c580 2024-04-22T13:38:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.499+0000 7ff417fff640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4000a3d20 0x7ff4000a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff408009bd0 tx=0x7ff408005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.502+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff410014030 con 0x7ff41810c580 2024-04-22T13:38:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.732+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff41802d5c0 con 0x7ff41810c580 2024-04-22T13:38:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.733+0000 7ff4157fa640 1 -- 172.21.15.5:0/3671752743 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11902 (secure 0 0 0) 0x7ff41009d580 con 0x7ff41810c580 2024-04-22T13:38:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:38:15.730 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1569295173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1569295173},{"type":"v1","addr":"172.21.15.49:6833","nonce":1569295173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/219923343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":219923343},{"type":"v1","addr":"172.21.15.5:6835","nonce":219923343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:25:10.226899+0000","modified":"2024-04-22T13:29:17.280118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24496,"mds_3":24485,"mds_4":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14550":{"gid":14550,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/1096407808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1096407808},{"type":"v1","addr":"172.21.15.5:6837","nonce":1096407808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/373669901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":373669901},{"type":"v1","addr":"172.21.15.5:6839","nonce":373669901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14637":{"gid":14637,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/4083506255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":4083506255},{"type":"v1","addr":"172.21.15.5:6841","nonce":4083506255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1584198825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1584198825},{"type":"v1","addr":"172.21.15.107:6833","nonce":1584198825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6837/207360580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":207360580},{"type":"v1","addr":"172.21.15.107:6837","nonce":207360580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.49:6835/827214764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":827214764},{"type":"v1","addr":"172.21.15.49:6835","nonce":827214764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6839/3107882391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3107882391},{"type":"v1","addr":"172.21.15.107:6839","nonce":3107882391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6837/164218196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":164218196},{"type":"v1","addr":"172.21.15.49:6837","nonce":164218196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6839/1393810832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":1393810832},{"type":"v1","addr":"172.21.15.49:6839","nonce":1393810832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26113":{"gid":26113,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/603974888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":603974888},{"type":"v1","addr":"172.21.15.107:6835","nonce":603974888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24469,24485,24496,24512,24557]},"id":1}]} 2024-04-22T13:38:15.730 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-22T13:38:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4000a3d20 msgr2=0x7ff4000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4000a3d20 0x7ff4000a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff408009bd0 tx=0x7ff408005f90 comp rx=0 tx=0).stop 2024-04-22T13:38:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 msgr2=0x7ff418101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 0x7ff418101880 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff41000ccb0 tx=0x7ff41000a620 comp rx=0 tx=0).stop 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 shutdown_connections 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41810c580 0x7ff418101880 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7ff4000a3d20 0x7ff4000a61e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff418103ad0 0x7ff418101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 --2- 172.21.15.5:0/3671752743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4181031b0 0x7ff418100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 >> 172.21.15.5:0/3671752743 conn(0x7ff4180771d0 msgr2=0x7ff4180fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.735+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 shutdown_connections 2024-04-22T13:38:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.736+0000 7ff41e6f1640 1 -- 172.21.15.5:0/3671752743 wait complete. 2024-04-22T13:38:15.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:38:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.848+0000 7fd4de150640 1 -- 172.21.15.5:0/2177946065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 msgr2=0x7fd4d8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.848+0000 7fd4de150640 1 --2- 172.21.15.5:0/2177946065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8101450 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009900 tx=0x7fd4c802bc30 comp rx=0 tx=0).stop 2024-04-22T13:38:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.848+0000 7fd4de150640 1 -- 172.21.15.5:0/2177946065 shutdown_connections 2024-04-22T13:38:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.848+0000 7fd4de150640 1 --2- 172.21.15.5:0/2177946065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d8106140 0x7fd4d810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.848+0000 7fd4de150640 1 --2- 172.21.15.5:0/2177946065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d8101a20 0x7fd4d8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.848+0000 7fd4de150640 1 --2- 172.21.15.5:0/2177946065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8101450 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.849+0000 7fd4de150640 1 -- 172.21.15.5:0/2177946065 >> 172.21.15.5:0/2177946065 conn(0x7fd4d80771d0 msgr2=0x7fd4d80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.849+0000 7fd4de150640 1 -- 172.21.15.5:0/2177946065 shutdown_connections 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.849+0000 7fd4de150640 1 -- 172.21.15.5:0/2177946065 wait complete. 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.849+0000 7fd4de150640 1 Processor -- start 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.849+0000 7fd4de150640 1 -- start start 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d8106140 0x7fd4d8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d819cfe0 0x7fd4d819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81110d0 con 0x7fd4d8101070 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8110f50 con 0x7fd4d8106140 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8111250 con 0x7fd4d819cfe0 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49574/0 (socket says 172.21.15.5:49574) 2024-04-22T13:38:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 -- 172.21.15.5:0/1188383253 learned_addr learned my addr 172.21.15.5:0/1188383253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d7fff640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d819cfe0 0x7fd4d819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d6ffd640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d8106140 0x7fd4d8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 -- 172.21.15.5:0/1188383253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d819cfe0 msgr2=0x7fd4d819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d819cfe0 0x7fd4d819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 -- 172.21.15.5:0/1188383253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d8106140 msgr2=0x7fd4d8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d8106140 0x7fd4d8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 -- 172.21.15.5:0/1188383253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d819dad0 con 0x7fd4d8101070 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d77fe640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8198720 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000c00 tx=0x7fd4c802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8043020 con 0x7fd4d8101070 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c80052b0 con 0x7fd4d8101070 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d819dd60 con 0x7fd4d8101070 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8003c60 con 0x7fd4d8101070 2024-04-22T13:38:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.850+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d81a5580 con 0x7fd4d8101070 2024-04-22T13:38:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.851+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d8068e00 con 0x7fd4d8101070 2024-04-22T13:38:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.853+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd4c8046020 con 0x7fd4d8101070 2024-04-22T13:38:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.853+0000 7fd4d4ff9640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4c00a3d20 0x7fd4c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.853+0000 7fd4d6ffd640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4c00a3d20 0x7fd4c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.853+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fd4c80f2e00 con 0x7fd4d8101070 2024-04-22T13:38:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.854+0000 7fd4d6ffd640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4c00a3d20 0x7fd4c00a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005fd0 tx=0x7fd4cc005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:15.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:15.857+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4c804d020 con 0x7fd4d8101070 2024-04-22T13:38:16.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1746381179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:38:16.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:15 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3671752743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:38:16.021 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1746381179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:38:16.021 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:15 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3671752743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:38:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.044+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4d810a380 con 0x7fd4d8101070 2024-04-22T13:38:16.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.046+0000 7fd4d4ff9640 1 -- 172.21.15.5:0/1188383253 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23014 (secure 0 0 0) 0x7fd4c80bf5b0 con 0x7fd4d8101070 2024-04-22T13:38:16.043 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-22T13:38:16.043 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":103,"fsid":"73e95e94-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:17:12.208053+0000","modified":"2024-04-22T13:38:12.403440+0000","last_up_change":"2024-04-22T13:24:22.966848+0000","last_in_change":"2024-04-22T13:24:03.377773+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:20:42.191953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:25:08.431891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:25:09.192645+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39983697-bdf7-4808-8aaf-f2ae5083be8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6803","nonce":236509498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6805","nonce":236509498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6809","nonce":236509498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":236509498},{"type":"v1","addr":"172.21.15.5:6807","nonce":236509498}]},"public_addr":"172.21.15.5:6803/236509498","cluster_addr":"172.21.15.5:6805/236509498","heartbeat_back_addr":"172.21.15.5:6809/236509498","heartbeat_front_addr":"172.21.15.5:6807/236509498","state":["exists","up"]},{"osd":1,"uuid":"555f251b-3c1d-48a2-a456-d45129ace7df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6801","nonce":3148163706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6803","nonce":3148163706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6807","nonce":3148163706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3148163706},{"type":"v1","addr":"172.21.15.49:6805","nonce":3148163706}]},"public_addr":"172.21.15.49:6801/3148163706","cluster_addr":"172.21.15.49:6803/3148163706","heartbeat_back_addr":"172.21.15.49:6807/3148163706","heartbeat_front_addr":"172.21.15.49:6805/3148163706","state":["exists","up"]},{"osd":2,"uuid":"7d526a13-cdf8-49ac-9871-b8a577898db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6801","nonce":1765086658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1765086658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1765086658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1765086658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1765086658}]},"public_addr":"172.21.15.107:6801/1765086658","cluster_addr":"172.21.15.107:6803/1765086658","heartbeat_back_addr":"172.21.15.107:6807/1765086658","heartbeat_front_addr":"172.21.15.107:6805/1765086658","state":["exists","up"]},{"osd":3,"uuid":"8732d14a-c8a2-4cbe-ac81-8d7eccbecd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6811","nonce":466638912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6813","nonce":466638912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6817","nonce":466638912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":466638912},{"type":"v1","addr":"172.21.15.5:6815","nonce":466638912}]},"public_addr":"172.21.15.5:6811/466638912","cluster_addr":"172.21.15.5:6813/466638912","heartbeat_back_addr":"172.21.15.5:6817/466638912","heartbeat_front_addr":"172.21.15.5:6815/466638912","state":["exists","up"]},{"osd":4,"uuid":"9dc694d4-956f-424a-a8a7-e56d4e3b1213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6809","nonce":939487791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6811","nonce":939487791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6815","nonce":939487791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":939487791},{"type":"v1","addr":"172.21.15.49:6813","nonce":939487791}]},"public_addr":"172.21.15.49:6809/939487791","cluster_addr":"172.21.15.49:6811/939487791","heartbeat_back_addr":"172.21.15.49:6815/939487791","heartbeat_front_addr":"172.21.15.49:6813/939487791","state":["exists","up"]},{"osd":5,"uuid":"2da21bf6-32ce-49f7-93ba-362c202d18cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6809","nonce":2834490578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6811","nonce":2834490578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6815","nonce":2834490578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2834490578},{"type":"v1","addr":"172.21.15.107:6813","nonce":2834490578}]},"public_addr":"172.21.15.107:6809/2834490578","cluster_addr":"172.21.15.107:6811/2834490578","heartbeat_back_addr":"172.21.15.107:6815/2834490578","heartbeat_front_addr":"172.21.15.107:6813/2834490578","state":["exists","up"]},{"osd":6,"uuid":"b51d6e8a-1cb9-4856-9945-94e6ecb354d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2491838519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2491838519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6825","nonce":2491838519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2491838519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2491838519}]},"public_addr":"172.21.15.5:6819/2491838519","cluster_addr":"172.21.15.5:6821/2491838519","heartbeat_back_addr":"172.21.15.5:6825/2491838519","heartbeat_front_addr":"172.21.15.5:6823/2491838519","state":["exists","up"]},{"osd":7,"uuid":"49b9738e-63b0-4716-8f64-d9508227837a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6817","nonce":3440466171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6819","nonce":3440466171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6823","nonce":3440466171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3440466171},{"type":"v1","addr":"172.21.15.49:6821","nonce":3440466171}]},"public_addr":"172.21.15.49:6817/3440466171","cluster_addr":"172.21.15.49:6819/3440466171","heartbeat_back_addr":"172.21.15.49:6823/3440466171","heartbeat_front_addr":"172.21.15.49:6821/3440466171","state":["exists","up"]},{"osd":8,"uuid":"1f91b08d-b3ad-49c4-9c0d-ac20c1f0f759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6817","nonce":4036561619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6819","nonce":4036561619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6823","nonce":4036561619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4036561619},{"type":"v1","addr":"172.21.15.107:6821","nonce":4036561619}]},"public_addr":"172.21.15.107:6817/4036561619","cluster_addr":"172.21.15.107:6819/4036561619","heartbeat_back_addr":"172.21.15.107:6823/4036561619","heartbeat_front_addr":"172.21.15.107:6821/4036561619","state":["exists","up"]},{"osd":9,"uuid":"2392b48b-bdb9-4875-8725-20edf192d686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6827","nonce":3613366882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6829","nonce":3613366882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6833","nonce":3613366882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3613366882},{"type":"v1","addr":"172.21.15.5:6831","nonce":3613366882}]},"public_addr":"172.21.15.5:6827/3613366882","cluster_addr":"172.21.15.5:6829/3613366882","heartbeat_back_addr":"172.21.15.5:6833/3613366882","heartbeat_front_addr":"172.21.15.5:6831/3613366882","state":["exists","up"]},{"osd":10,"uuid":"aaa1c6f2-7fa0-442d-89ca-2ef7e46b8d0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6825","nonce":2579440718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6827","nonce":2579440718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6831","nonce":2579440718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2579440718},{"type":"v1","addr":"172.21.15.49:6829","nonce":2579440718}]},"public_addr":"172.21.15.49:6825/2579440718","cluster_addr":"172.21.15.49:6827/2579440718","heartbeat_back_addr":"172.21.15.49:6831/2579440718","heartbeat_front_addr":"172.21.15.49:6829/2579440718","state":["exists","up"]},{"osd":11,"uuid":"1c592b75-f30f-4e7f-b38c-5b09e1252184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6825","nonce":1856713900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6827","nonce":1856713900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6831","nonce":1856713900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1856713900},{"type":"v1","addr":"172.21.15.107:6829","nonce":1856713900}]},"public_addr":"172.21.15.107:6825/1856713900","cluster_addr":"172.21.15.107:6827/1856713900","heartbeat_back_addr":"172.21.15.107:6831/1856713900","heartbeat_front_addr":"172.21.15.107:6829/1856713900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:49.405914+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:13.428568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:37.388616+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:02.142118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:26.883889+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:51.186354+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:16.152063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:22:40.428192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:05.054199+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:30.567033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:23:56.373806+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:24:21.175720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/2674774949":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/3327695745":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1240644936":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1744519677":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/3837385355":"2024-04-23T13:27:09.442003+0000","172.21.15.5:0/1556657643":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/2261038189":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/4095818300":"2024-04-23T13:18:14.098934+0000","172.21.15.5:0/1954020614":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1797990010":"2024-04-23T13:18:14.098934+0000","172.21.15.5:6800/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:6801/3443029620":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/1011902624":"2024-04-23T13:17:46.974612+0000","172.21.15.5:0/2682698140":"2024-04-23T13:27:09.442003+0000","172.21.15.5:6800/4095818300":"2024-04-23T13:18:14.098934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4c00a3d20 msgr2=0x7fd4c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4c00a3d20 0x7fd4c00a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005fd0 tx=0x7fd4cc005850 comp rx=0 tx=0).stop 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 msgr2=0x7fd4d8198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8198720 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000c00 tx=0x7fd4c802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 shutdown_connections 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fd4c00a3d20 0x7fd4c00a61e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d819cfe0 0x7fd4d819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4d8106140 0x7fd4d8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 --2- 172.21.15.5:0/1188383253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d8101070 0x7fd4d8198720 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 >> 172.21.15.5:0/1188383253 conn(0x7fd4d80771d0 msgr2=0x7fd4d8069610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 shutdown_connections 2024-04-22T13:38:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.048+0000 7fd4de150640 1 -- 172.21.15.5:0/1188383253 wait complete. 2024-04-22T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1746381179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:15 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3671752743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:38:16.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-22T13:38:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 -- 172.21.15.5:0/298041779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 msgr2=0x7f8b0810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/298041779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0810eef0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009930 tx=0x7f8af802bc30 comp rx=0 tx=0).stop 2024-04-22T13:38:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 -- 172.21.15.5:0/298041779 shutdown_connections 2024-04-22T13:38:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/298041779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0810eef0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/298041779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08103b50 0x7f8b08103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/298041779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b08102270 0x7f8b08102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.164+0000 7f8b0ec06640 1 -- 172.21.15.5:0/298041779 >> 172.21.15.5:0/298041779 conn(0x7f8b080fda10 msgr2=0x7f8b080ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 -- 172.21.15.5:0/298041779 shutdown_connections 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 -- 172.21.15.5:0/298041779 wait complete. 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 Processor -- start 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 -- start start 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08102270 0x7f8b081986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b08103b50 0x7f8b08198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08112270 con 0x7f8b081044f0 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b081120f0 con 0x7f8b08102270 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0ec06640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b081123f0 con 0x7f8b08103b50 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0d1a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0d1a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49588/0 (socket says 172.21.15.5:49588) 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.165+0000 7f8b0d1a3640 1 -- 172.21.15.5:0/796284063 learned_addr learned my addr 172.21.15.5:0/796284063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b07fff640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b08103b50 0x7f8b08198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0c9a2640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08102270 0x7f8b081986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0d1a3640 1 -- 172.21.15.5:0/796284063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b08103b50 msgr2=0x7f8b08198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0d1a3640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b08103b50 0x7f8b08198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0d1a3640 1 -- 172.21.15.5:0/796284063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08102270 msgr2=0x7f8b081986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0d1a3640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08102270 0x7f8b081986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0d1a3640 1 -- 172.21.15.5:0/796284063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0819d6c0 con 0x7f8b081044f0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0c9a2640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08102270 0x7f8b081986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0d1a3640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0819cf90 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8af8005940 tx=0x7f8af80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8043020 con 0x7f8b081044f0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0819d950 con 0x7f8b081044f0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af802ca70 con 0x7f8b081044f0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8003ab0 con 0x7f8b081044f0 2024-04-22T13:38:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.166+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b081a5240 con 0x7f8b081044f0 2024-04-22T13:38:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.167+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b08069250 con 0x7f8b081044f0 2024-04-22T13:38:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.169+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8af8046020 con 0x7f8b081044f0 2024-04-22T13:38:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.169+0000 7f8b05ffb640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8af00a3d20 0x7f8af00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.169+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f8af80f2c30 con 0x7f8b081044f0 2024-04-22T13:38:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.169+0000 7f8b0c9a2640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8af00a3d20 0x7f8af00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.170+0000 7f8b0c9a2640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8af00a3d20 0x7f8af00a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b00007e70 tx=0x7f8b00007850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.173+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8af804d020 con 0x7f8b081044f0 2024-04-22T13:38:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.399+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8b08199a20 con 0x7f8b081044f0 2024-04-22T13:38:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.402+0000 7f8b05ffb640 1 -- 172.21.15.5:0/796284063 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f8b08199a20 con 0x7f8b081044f0 2024-04-22T13:38:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8af00a3d20 msgr2=0x7f8af00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8af00a3d20 0x7f8af00a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b00007e70 tx=0x7f8b00007850 comp rx=0 tx=0).stop 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 msgr2=0x7f8b0819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0819cf90 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8af8005940 tx=0x7f8af80045e0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 shutdown_connections 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b081044f0 0x7f8b0819cf90 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f8af00a3d20 0x7f8af00a61e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b08103b50 0x7f8b08198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 --2- 172.21.15.5:0/796284063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b08102270 0x7f8b081986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 >> 172.21.15.5:0/796284063 conn(0x7f8b080fda10 msgr2=0x7f8b080fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 shutdown_connections 2024-04-22T13:38:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.406+0000 7f8b0ec06640 1 -- 172.21.15.5:0/796284063 wait complete. 2024-04-22T13:38:16.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-22T13:38:16.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.529+0000 7fdb3b255640 1 -- 172.21.15.5:0/2552180508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 msgr2=0x7fdb34101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.529+0000 7fdb3b255640 1 --2- 172.21.15.5:0/2552180508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34101430 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009ca0 tx=0x7fdb2c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.529+0000 7fdb3b255640 1 -- 172.21.15.5:0/2552180508 shutdown_connections 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.529+0000 7fdb3b255640 1 --2- 172.21.15.5:0/2552180508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34106120 0x7fdb3410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 --2- 172.21.15.5:0/2552180508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb34101a00 0x7fdb34105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 --2- 172.21.15.5:0/2552180508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34101430 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- 172.21.15.5:0/2552180508 >> 172.21.15.5:0/2552180508 conn(0x7fdb34077fa0 msgr2=0x7fdb340783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- 172.21.15.5:0/2552180508 shutdown_connections 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- 172.21.15.5:0/2552180508 wait complete. 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 Processor -- start 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- start start 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34106120 0x7fdb34198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3419cfd0 0x7fdb3419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34111060 con 0x7fdb34101050 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34110ee0 con 0x7fdb3419cfd0 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb3b255640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341111e0 con 0x7fdb34106120 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb38ff1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb38ff1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49662/0 (socket says 172.21.15.5:49662) 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.530+0000 7fdb38ff1640 1 -- 172.21.15.5:0/3250413305 learned_addr learned my addr 172.21.15.5:0/3250413305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb33fff640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34106120 0x7fdb34198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb397f2640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3419cfd0 0x7fdb3419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb38ff1640 1 -- 172.21.15.5:0/3250413305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34106120 msgr2=0x7fdb34198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb38ff1640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34106120 0x7fdb34198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb38ff1640 1 -- 172.21.15.5:0/3250413305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3419cfd0 msgr2=0x7fdb3419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb38ff1640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3419cfd0 0x7fdb3419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:16.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb38ff1640 1 -- 172.21.15.5:0/3250413305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3419dac0 con 0x7fdb34101050 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb38ff1640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34198710 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005e30 tx=0x7fdb2c0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c039040 con 0x7fdb34101050 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3419dd50 con 0x7fdb34101050 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2c044070 con 0x7fdb34101050 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0042a0 con 0x7fdb34101050 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.531+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb341a5510 con 0x7fdb34101050 2024-04-22T13:38:16.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.532+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb34102520 con 0x7fdb34101050 2024-04-22T13:38:16.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.533+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fdb2c004440 con 0x7fdb34101050 2024-04-22T13:38:16.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.534+0000 7fdb31ffb640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdb200a3d20 0x7fdb200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:16.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.534+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8899+0+0 (secure 0 0 0) 0x7fdb2c0f35e0 con 0x7fdb34101050 2024-04-22T13:38:16.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.534+0000 7fdb33fff640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdb200a3d20 0x7fdb200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:16.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.535+0000 7fdb33fff640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdb200a3d20 0x7fdb200a61e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb34105c10 tx=0x7fdb1c00b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:16.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.538+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb2c0bfc40 con 0x7fdb34101050 2024-04-22T13:38:16.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a[28002]: 2024-04-22T13:38:16.400+0000 7fb7c7283700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:38:16.755 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d[69425]: starting mds.d at 2024-04-22T13:38:16.755 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g[70022]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.755 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g[70022]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mds[70026]: -- 172.21.15.5:0/1745815493 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589209004e0 con 0x5589208d6c00 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mds[70026]: -- 172.21.15.5:0/1745815493 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558920903a40 con 0x5589208d6c00 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g[70022]: starting mds.g at 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a[70379]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a[70379]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mds[70383]: -- 172.21.15.5:0/1696908193 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636eadeda40 con 0x5636eadc1000 2024-04-22T13:38:16.756 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a[70379]: starting mds.a at 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e[60712]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e[60712]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mds[60716]: -- 172.21.15.49:0/3166483006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5619575c1a40 con 0x561957594800 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e[60712]: starting mds.e at 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c[63277]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c[63277]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.757 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c[63277]: starting mds.c at 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mds[61348]: -- 172.21.15.49:0/4102831661 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db557cda40 con 0x55db557a0c00 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: starting mds.b at 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[66920]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[66920]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mds[66943]: -- 172.21.15.107:0/1102693619 <== mon.1 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558269861a40 con 0x558269834800 2024-04-22T13:38:16.758 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[66920]: starting mds.f at 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h[61023]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h[61023]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mds[61027]: -- 172.21.15.49:0/4175017075 <== mon.2 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb13ca1a40 con 0x55eb13c74800 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h[61023]: starting mds.h at 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i[63589]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i[63589]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mds[63594]: -- 172.21.15.107:0/1705363471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3f1c1ba40 con 0x55d3f1bee800 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i[63589]: starting mds.i at 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: ignoring --setuser ceph since I am not root 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: ignoring --setgroup ceph since I am not root 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mds[62657]: -- 172.21.15.107:0/1503170617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef1063da40 con 0x55ef10610c00 2024-04-22T13:38:16.759 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: starting mds.l at 2024-04-22T13:38:16.760 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: 2024-04-22T13:38:16.462+0000 7f6b26413700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2024-04-22T13:38:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:16.764+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdb34199a60 con 0x7fdb34101050 2024-04-22T13:38:17.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: pgmap v347: 97 pgs: 97 active+clean; 1.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 827 B/s wr, 11 op/s 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/1188383253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/796284063' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/796284063' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: osdmap e104: 12 total, 12 up, 12 in 2024-04-22T13:38:17.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:16 smithi049 ceph-mon[31379]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: pgmap v347: 97 pgs: 97 active+clean; 1.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 827 B/s wr, 11 op/s 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/1188383253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/796284063' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/796284063' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: osdmap e104: 12 total, 12 up, 12 in 2024-04-22T13:38:17.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:16 smithi107 ceph-mon[31859]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: pgmap v347: 97 pgs: 97 active+clean; 1.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 827 B/s wr, 11 op/s 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/1188383253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/796284063' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/796284063' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: osdmap e104: 12 total, 12 up, 12 in 2024-04-22T13:38:17.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:16 smithi005 ceph-mon[28006]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-22T13:38:17.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.404+0000 7fdb31ffb640 1 -- 172.21.15.5:0/3250413305 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fdb2c0bf540 con 0x7fdb34101050 2024-04-22T13:38:17.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdb200a3d20 msgr2=0x7fdb200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:17.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdb200a3d20 0x7fdb200a61e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb34105c10 tx=0x7fdb1c00b390 comp rx=0 tx=0).stop 2024-04-22T13:38:17.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 msgr2=0x7fdb34198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:17.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34198710 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005e30 tx=0x7fdb2c0346f0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 shutdown_connections 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fdb200a3d20 0x7fdb200a61e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3419cfd0 0x7fdb3419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34106120 0x7fdb34198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 --2- 172.21.15.5:0/3250413305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34101050 0x7fdb34198710 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.407+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 >> 172.21.15.5:0/3250413305 conn(0x7fdb34077fa0 msgr2=0x7fdb34069650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.408+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 shutdown_connections 2024-04-22T13:38:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.408+0000 7fdb3b255640 1 -- 172.21.15.5:0/3250413305 wait complete. 2024-04-22T13:38:17.421 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 -- 172.21.15.5:0/1194233341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0106120 msgr2=0x7f81b010dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/1194233341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0106120 0x7f81b010dcf0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009ca0 tx=0x7f81a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 -- 172.21.15.5:0/1194233341 shutdown_connections 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/1194233341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0106120 0x7f81b010dcf0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/1194233341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b0101a00 0x7f81b0105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/1194233341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b0101050 0x7f81b0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 -- 172.21.15.5:0/1194233341 >> 172.21.15.5:0/1194233341 conn(0x7f81b0077fa0 msgr2=0x7f81b00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 -- 172.21.15.5:0/1194233341 shutdown_connections 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 -- 172.21.15.5:0/1194233341 wait complete. 2024-04-22T13:38:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 Processor -- start 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.533+0000 7f81b6d0e640 1 -- start start 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b6d0e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b0101050 0x7f81b0102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b6d0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b0106120 0x7f81b0103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b6d0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 0x7f81b01a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b6d0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0111070 con 0x7f81b0103560 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b6d0e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0110ef0 con 0x7f81b0106120 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b6d0e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01111f0 con 0x7f81b0101050 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 0x7f81b01a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 0x7f81b01a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49678/0 (socket says 172.21.15.5:49678) 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 -- 172.21.15.5:0/3391922072 learned_addr learned my addr 172.21.15.5:0/3391922072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b4aaa640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b0101050 0x7f81b0102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:17.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81affff640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b0106120 0x7f81b0103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 -- 172.21.15.5:0/3391922072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b0101050 msgr2=0x7f81b0102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b0101050 0x7f81b0102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 -- 172.21.15.5:0/3391922072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b0106120 msgr2=0x7f81b0103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b0106120 0x7f81b0103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 -- 172.21.15.5:0/3391922072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a6f90 con 0x7f81b0103560 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81b52ab640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 0x7f81b01a6890 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005e30 tx=0x7f81a8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8039040 con 0x7f81b0103560 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a8044070 con 0x7f81b0103560 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.534+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003e60 con 0x7f81b0103560 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.535+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a7220 con 0x7f81b0103560 2024-04-22T13:38:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.535+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a76c0 con 0x7f81b0103560 2024-04-22T13:38:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.536+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b0068e00 con 0x7f81b0103560 2024-04-22T13:38:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.537+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f81a80324c0 con 0x7f81b0103560 2024-04-22T13:38:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.537+0000 7f81adffb640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f81a00a3d50 0x7f81a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.538+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8899+0+0 (secure 0 0 0) 0x7f81a80f3580 con 0x7f81b0103560 2024-04-22T13:38:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.538+0000 7f81b4aaa640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f81a00a3d50 0x7f81a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.538+0000 7f81b4aaa640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f81a00a3d50 0x7f81a00a6210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005d00 tx=0x7f81a400b410 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.541+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81a80bfb50 con 0x7f81b0103560 2024-04-22T13:38:17.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:17.729+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f81b010a360 con 0x7f81b0103560 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3250413305' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: Cluster is now healthy 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3250413305' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6836/2674617210,v1:172.21.15.5:6839/2674617210] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6837/4144820927,v1:172.21.15.5:6840/4144820927] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.5:6838/3463602291,v1:172.21.15.5:6841/3463602291] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6834/3894422216,v1:172.21.15.49:6835/3894422216] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6833/2352947103,v1:172.21.15.107:6834/2352947103] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6832/1753073943,v1:172.21.15.107:6835/1753073943] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6836/993247027,v1:172.21.15.49:6838/993247027] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.49:6837/1836253948,v1:172.21.15.49:6839/1836253948] up:boot 2024-04-22T13:38:17.759 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6836/1402170801,v1:172.21.15.107:6838/1402170801] up:boot 2024-04-22T13:38:17.760 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: mds.? [v2:172.21.15.107:6837/3722169594,v1:172.21.15.107:6839/3722169594] up:boot 2024-04-22T13:38:17.760 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: fsmap 12 up:standby 2024-04-22T13:38:18.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:38:18.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:38:18.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:17 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3391922072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3250413305' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: Cluster is now healthy 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3250413305' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6836/2674617210,v1:172.21.15.5:6839/2674617210] up:boot 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6837/4144820927,v1:172.21.15.5:6840/4144820927] up:boot 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.5:6838/3463602291,v1:172.21.15.5:6841/3463602291] up:boot 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6834/3894422216,v1:172.21.15.49:6835/3894422216] up:boot 2024-04-22T13:38:18.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6833/2352947103,v1:172.21.15.107:6834/2352947103] up:boot 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6832/1753073943,v1:172.21.15.107:6835/1753073943] up:boot 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6836/993247027,v1:172.21.15.49:6838/993247027] up:boot 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.49:6837/1836253948,v1:172.21.15.49:6839/1836253948] up:boot 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6836/1402170801,v1:172.21.15.107:6838/1402170801] up:boot 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: mds.? [v2:172.21.15.107:6837/3722169594,v1:172.21.15.107:6839/3722169594] up:boot 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: fsmap 12 up:standby 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:17 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3391922072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3250413305' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: Cluster is now healthy 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3250413305' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6836/2674617210,v1:172.21.15.5:6839/2674617210] up:boot 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6837/4144820927,v1:172.21.15.5:6840/4144820927] up:boot 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.5:6838/3463602291,v1:172.21.15.5:6841/3463602291] up:boot 2024-04-22T13:38:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6834/3894422216,v1:172.21.15.49:6835/3894422216] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6833/2352947103,v1:172.21.15.107:6834/2352947103] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6832/1753073943,v1:172.21.15.107:6835/1753073943] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6836/993247027,v1:172.21.15.49:6838/993247027] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.49:6837/1836253948,v1:172.21.15.49:6839/1836253948] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6836/1402170801,v1:172.21.15.107:6838/1402170801] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: mds.? [v2:172.21.15.107:6837/3722169594,v1:172.21.15.107:6839/3722169594] up:boot 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: fsmap 12 up:standby 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:38:18.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='mgr.15354 172.21.15.5:0/2071587479' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:38:18.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:17 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3391922072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:38:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.410+0000 7f81adffb640 1 -- 172.21.15.5:0/3391922072 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f81a80bf450 con 0x7f81b0103560 2024-04-22T13:38:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-22T13:38:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.414+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f81a00a3d50 msgr2=0x7f81a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.414+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f81a00a3d50 0x7f81a00a6210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005d00 tx=0x7f81a400b410 comp rx=0 tx=0).stop 2024-04-22T13:38:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.414+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 msgr2=0x7f81b01a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.414+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 0x7f81b01a6890 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005e30 tx=0x7f81a8034760 comp rx=0 tx=0).stop 2024-04-22T13:38:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 shutdown_connections 2024-04-22T13:38:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b0103560 0x7f81b01a6890 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7f81a00a3d50 0x7f81a00a6210 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005d00 tx=0x7f81a400b410 comp rx=0 tx=0).stop 2024-04-22T13:38:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81b0106120 0x7f81b0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 --2- 172.21.15.5:0/3391922072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81b0101050 0x7f81b0102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 >> 172.21.15.5:0/3391922072 conn(0x7f81b0077fa0 msgr2=0x7f81b00696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 shutdown_connections 2024-04-22T13:38:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.415+0000 7f81b6d0e640 1 -- 172.21.15.5:0/3391922072 wait complete. 2024-04-22T13:38:18.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-22T13:38:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 -- 172.21.15.5:0/2532508042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4101090 msgr2=0x7fe4c4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/2532508042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4101090 0x7fe4c4101470 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:38:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 -- 172.21.15.5:0/2532508042 shutdown_connections 2024-04-22T13:38:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/2532508042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4106160 0x7fe4c410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/2532508042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c4101a40 0x7fe4c4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/2532508042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4101090 0x7fe4c4101470 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 -- 172.21.15.5:0/2532508042 >> 172.21.15.5:0/2532508042 conn(0x7fe4c4078000 msgr2=0x7fe4c4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 -- 172.21.15.5:0/2532508042 shutdown_connections 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.547+0000 7fe4cb87b640 1 -- 172.21.15.5:0/2532508042 wait complete. 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 Processor -- start 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 -- start start 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4101090 0x7fe4c4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 0x7fe4c4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c419a050 0x7fe4c41a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41110a0 con 0x7fe4c4106160 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4110f20 con 0x7fe4c419a050 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4cb87b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4111220 con 0x7fe4c4101090 2024-04-22T13:38:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 0x7fe4c4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 0x7fe4c4199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49686/0 (socket says 172.21.15.5:49686) 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 -- 172.21.15.5:0/984510081 learned_addr learned my addr 172.21.15.5:0/984510081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c9e18640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c419a050 0x7fe4c41a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c9617640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4101090 0x7fe4c4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 -- 172.21.15.5:0/984510081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4101090 msgr2=0x7fe4c4199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4101090 0x7fe4c4199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 -- 172.21.15.5:0/984510081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c419a050 msgr2=0x7fe4c41a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c419a050 0x7fe4c41a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 -- 172.21.15.5:0/984510081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41a4e00 con 0x7fe4c4106160 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c9617640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4101090 0x7fe4c4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c9e18640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c419a050 0x7fe4c41a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.548+0000 7fe4c8e16640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 0x7fe4c4199ac0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9c0 tx=0x7fe4b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.549+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400cd80 con 0x7fe4c4106160 2024-04-22T13:38:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.549+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b40054d0 con 0x7fe4c4106160 2024-04-22T13:38:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.549+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c41a50f0 con 0x7fe4c4106160 2024-04-22T13:38:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.549+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4010430 con 0x7fe4c4106160 2024-04-22T13:38:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.549+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41a5640 con 0x7fe4c4106160 2024-04-22T13:38:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.550+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c4068e00 con 0x7fe4c4106160 2024-04-22T13:38:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.551+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe4b4004fd0 con 0x7fe4c4106160 2024-04-22T13:38:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.552+0000 7fe4ba7fc640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4b00a3d20 0x7fe4b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:38:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.552+0000 7fe4c9617640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4b00a3d20 0x7fe4b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:38:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.552+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8403+0+0 (secure 0 0 0) 0x7fe4b40d0c40 con 0x7fe4c4106160 2024-04-22T13:38:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.552+0000 7fe4c9617640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4b00a3d20 0x7fe4b00a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005ab0 tx=0x7fe4c0005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:38:18.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.556+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4b409d350 con 0x7fe4c4106160 2024-04-22T13:38:18.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:18.744+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe4c4101a40 con 0x7fe4c4106160 2024-04-22T13:38:19.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:18 smithi049 ceph-mon[31379]: pgmap v349: 97 pgs: 97 active+clean; 1.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 14 op/s 2024-04-22T13:38:19.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:18 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/3391922072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T13:38:19.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:18 smithi049 ceph-mon[31379]: osdmap e105: 12 total, 12 up, 12 in 2024-04-22T13:38:19.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:18 smithi049 ceph-mon[31379]: from='client.? 172.21.15.5:0/984510081' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:38:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:18 smithi107 ceph-mon[31859]: pgmap v349: 97 pgs: 97 active+clean; 1.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 14 op/s 2024-04-22T13:38:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:18 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/3391922072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T13:38:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:18 smithi107 ceph-mon[31859]: osdmap e105: 12 total, 12 up, 12 in 2024-04-22T13:38:19.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:18 smithi107 ceph-mon[31859]: from='client.? 172.21.15.5:0/984510081' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:38:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:18 smithi005 ceph-mon[28006]: pgmap v349: 97 pgs: 97 active+clean; 1.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 14 op/s 2024-04-22T13:38:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:18 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/3391922072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T13:38:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:18 smithi005 ceph-mon[28006]: osdmap e105: 12 total, 12 up, 12 in 2024-04-22T13:38:19.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:18 smithi005 ceph-mon[28006]: from='client.? 172.21.15.5:0/984510081' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:38:19.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.411+0000 7fe4ba7fc640 1 -- 172.21.15.5:0/984510081 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7fe4b409cc50 con 0x7fe4c4106160 2024-04-22T13:38:19.408 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-22T13:38:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.415+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4b00a3d20 msgr2=0x7fe4b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.415+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4b00a3d20 0x7fe4b00a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005ab0 tx=0x7fe4c0005a20 comp rx=0 tx=0).stop 2024-04-22T13:38:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.415+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 msgr2=0x7fe4c4199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:38:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.415+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 0x7fe4c4199ac0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9c0 tx=0x7fe4b400ee80 comp rx=0 tx=0).stop 2024-04-22T13:38:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 shutdown_connections 2024-04-22T13:38:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c4106160 0x7fe4c4199ac0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.5:6800/751347710,v1:172.21.15.5:6801/751347710] conn(0x7fe4b00a3d20 0x7fe4b00a61e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4c419a050 0x7fe4c41a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 --2- 172.21.15.5:0/984510081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4101090 0x7fe4c4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:38:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 >> 172.21.15.5:0/984510081 conn(0x7fe4c4078000 msgr2=0x7fe4c40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:38:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 shutdown_connections 2024-04-22T13:38:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-22T13:38:19.416+0000 7fe4cb87b640 1 -- 172.21.15.5:0/984510081 wait complete. 2024-04-22T13:38:19.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:38:19.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:38:19.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:38:19.520 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-22T13:38:19.520 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:38:19.537 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:38:19.555 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:38:19.576 INFO:tasks.cephadm:Stopping all daemons... 2024-04-22T13:38:19.577 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-22T13:38:19.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a 2024-04-22T13:38:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:19 smithi005 systemd[1]: Stopping Ceph mon.a for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:20.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a[28002]: 2024-04-22T13:38:19.907+0000 7fb7cca8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:20.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a[28002]: 2024-04-22T13:38:19.907+0000 7fb7cca8e700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-22T13:38:20.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:20 smithi005 podman[101175]: 2024-04-22 13:38:20.173389082 +0000 UTC m=+0.393279986 container died ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T13:38:20.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:20 smithi005 podman[101175]: 2024-04-22 13:38:20.515940998 +0000 UTC m=+0.735831890 container cleanup ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-22T13:38:20.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:20 smithi005 bash[101175]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a 2024-04-22T13:38:20.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 22 13:38:20 smithi005 podman[101255]: 2024-04-22 13:38:20.742254224 +0000 UTC m=+0.560862381 container remove ecce5d1b31a90589a1225f63746d2d655f7295c089311ae417f495bfeace39a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-22T13:38:20.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.a.service' 2024-04-22T13:38:21.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:21.022 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-22T13:38:21.022 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-22T13:38:21.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.b 2024-04-22T13:38:21.330 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:21 smithi049 systemd[1]: Stopping Ceph mon.b for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:21.740 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b[31375]: 2024-04-22T13:38:21.335+0000 7fda1d971700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:21.740 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b[31375]: 2024-04-22T13:38:21.335+0000 7fda1d971700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-22T13:38:21.740 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:21 smithi049 podman[66683]: 2024-04-22 13:38:21.478881999 +0000 UTC m=+0.244978265 container died b37e1c5be6ecbdb9bb8ad023ab83784313f5e9b4db85a191494f49244c8cff54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-22T13:38:22.020 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:21 smithi049 podman[66683]: 2024-04-22 13:38:21.852131887 +0000 UTC m=+0.618228148 container cleanup b37e1c5be6ecbdb9bb8ad023ab83784313f5e9b4db85a191494f49244c8cff54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T13:38:22.021 INFO:journalctl@ceph.mon.b.smithi049.stdout:Apr 22 13:38:21 smithi049 bash[66683]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-b 2024-04-22T13:38:22.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.b.service' 2024-04-22T13:38:22.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:22.331 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-22T13:38:22.331 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-22T13:38:22.331 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.c 2024-04-22T13:38:22.637 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:22 smithi107 systemd[1]: Stopping Ceph mon.c for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c[31845]: 2024-04-22T13:38:22.641+0000 7f7942ef8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c[31845]: 2024-04-22T13:38:22.641+0000 7f7942ef8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-22T13:38:23.034 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:22 smithi107 podman[69206]: 2024-04-22 13:38:22.846854671 +0000 UTC m=+0.318093591 container died 47e7084a938f70fdcbad07f9bbb6d6cc4e0fae233dcd42ab497a1183ec714822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-22T13:38:23.404 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:23 smithi107 podman[69206]: 2024-04-22 13:38:23.166393267 +0000 UTC m=+0.637632188 container cleanup 47e7084a938f70fdcbad07f9bbb6d6cc4e0fae233dcd42ab497a1183ec714822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-22T13:38:23.404 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:23 smithi107 bash[69206]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c 2024-04-22T13:38:23.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.c.service' 2024-04-22T13:38:23.658 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:23 smithi107 podman[69220]: 2024-04-22 13:38:23.409374741 +0000 UTC m=+0.558209603 container remove 47e7084a938f70fdcbad07f9bbb6d6cc4e0fae233dcd42ab497a1183ec714822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-22T13:38:23.658 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:23 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-22T13:38:23.658 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:23 smithi107 systemd[1]: Stopped Ceph mon.c for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:38:23.658 INFO:journalctl@ceph.mon.c.smithi107.stdout:Apr 22 13:38:23 smithi107 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mon.c.service: Consumed 12.023s CPU time. 2024-04-22T13:38:23.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:23.681 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-22T13:38:23.681 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-22T13:38:23.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x 2024-04-22T13:38:24.008 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:38:23 smithi005 systemd[1]: Stopping Ceph mgr.x for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:24.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:38:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x[28236]: 2024-04-22T13:38:24.011+0000 7f3d44250700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:24.308 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:38:24 smithi005 podman[101385]: 2024-04-22 13:38:24.042914648 +0000 UTC m=+0.154940667 container died e9cd05080807eaeb3ec9f511ba4f47d354227ea9230beab6cf1e0793464ab00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T13:38:24.674 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:38:24 smithi005 podman[101385]: 2024-04-22 13:38:24.403514318 +0000 UTC m=+0.515540338 container cleanup e9cd05080807eaeb3ec9f511ba4f47d354227ea9230beab6cf1e0793464ab00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-22T13:38:24.675 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 22 13:38:24 smithi005 bash[101385]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-x 2024-04-22T13:38:24.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-22T13:38:24.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:24.820 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-22T13:38:24.820 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-22T13:38:24.820 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.y 2024-04-22T13:38:25.172 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:24 smithi049 systemd[1]: Stopping Ceph mgr.y for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:25.435 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:25 smithi049 podman[66803]: 2024-04-22 13:38:25.177903938 +0000 UTC m=+0.147700636 container died 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-22T13:38:25.770 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:25 smithi049 podman[66803]: 2024-04-22 13:38:25.623626617 +0000 UTC m=+0.593423312 container cleanup 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:38:25.771 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:25 smithi049 bash[66803]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y 2024-04-22T13:38:26.055 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:25 smithi049 podman[66817]: 2024-04-22 13:38:25.810395054 +0000 UTC m=+0.624182903 container remove 2852b3b02f722abc1dbc8f788db76f1f3c837064f6b0bef3f79bef6d0121a674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:38:26.055 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:25 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-22T13:38:26.056 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:26 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-22T13:38:26.056 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Apr 22 13:38:26 smithi049 systemd[1]: Stopped Ceph mgr.y for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:38:26.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-22T13:38:26.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:26.093 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-22T13:38:26.093 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-22T13:38:26.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.z 2024-04-22T13:38:26.428 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:38:26 smithi107 systemd[1]: Stopping Ceph mgr.z for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:26.724 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:38:26 smithi107 podman[69329]: 2024-04-22 13:38:26.432775931 +0000 UTC m=+0.141971953 container died ddc7808f282d04a13eec53e56cdc8199868b734a3a59b3c56e2bf132a9987fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-22T13:38:27.034 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:38:26 smithi107 podman[69329]: 2024-04-22 13:38:26.803332025 +0000 UTC m=+0.512528048 container cleanup ddc7808f282d04a13eec53e56cdc8199868b734a3a59b3c56e2bf132a9987fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-22T13:38:27.034 INFO:journalctl@ceph.mgr.z.smithi107.stdout:Apr 22 13:38:26 smithi107 bash[69329]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mgr-z 2024-04-22T13:38:27.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-22T13:38:27.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:27.247 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-22T13:38:27.247 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-22T13:38:27.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.0 2024-04-22T13:38:27.557 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:27 smithi005 systemd[1]: Stopping Ceph osd.0 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:28.058 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0[42998]: 2024-04-22T13:38:27.690+0000 7fdcf241a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:28.058 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0[42998]: 2024-04-22T13:38:27.690+0000 7fdcf241a700 -1 osd.0 106 *** Got signal Terminated *** 2024-04-22T13:38:28.058 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0[42998]: 2024-04-22T13:38:27.690+0000 7fdcf241a700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:38:33.308 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:33 smithi005 podman[101506]: 2024-04-22 13:38:33.060314551 +0000 UTC m=+5.606852845 container died 4dfba8397827cfe352c89bc15fb8fb1448faae12761d284a7a4ac8cf83cad0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-22T13:38:33.794 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:33 smithi005 podman[101506]: 2024-04-22 13:38:33.414879472 +0000 UTC m=+5.961417770 container cleanup 4dfba8397827cfe352c89bc15fb8fb1448faae12761d284a7a4ac8cf83cad0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:38:33.794 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:33 smithi005 bash[101506]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0 2024-04-22T13:38:34.057 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:33 smithi005 podman[101523]: 2024-04-22 13:38:33.799249236 +0000 UTC m=+0.734567419 container remove 4dfba8397827cfe352c89bc15fb8fb1448faae12761d284a7a4ac8cf83cad0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:38:34.337 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:34 smithi005 podman[101611]: 2024-04-22 13:38:34.112943128 +0000 UTC m=+0.018852349 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:38:34.337 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:34 smithi005 podman[101611]: 2024-04-22 13:38:34.217033642 +0000 UTC m=+0.122942896 container create e46c2d00c9dbc6a396e606ac897904792394bb0f7f33cfa9536b9015459d9173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T13:38:34.764 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:34 smithi005 podman[101611]: 2024-04-22 13:38:34.46105944 +0000 UTC m=+0.366968649 container init e46c2d00c9dbc6a396e606ac897904792394bb0f7f33cfa9536b9015459d9173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:38:34.764 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:34 smithi005 podman[101611]: 2024-04-22 13:38:34.464268504 +0000 UTC m=+0.370177716 container start e46c2d00c9dbc6a396e606ac897904792394bb0f7f33cfa9536b9015459d9173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:38:34.764 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:34 smithi005 podman[101611]: 2024-04-22 13:38:34.55263871 +0000 UTC m=+0.458547923 container attach e46c2d00c9dbc6a396e606ac897904792394bb0f7f33cfa9536b9015459d9173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-22T13:38:35.057 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 22 13:38:34 smithi005 podman[101611]: 2024-04-22 13:38:34.769295527 +0000 UTC m=+0.675204744 container died e46c2d00c9dbc6a396e606ac897904792394bb0f7f33cfa9536b9015459d9173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T13:38:35.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.0.service' 2024-04-22T13:38:35.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:35.554 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-22T13:38:35.555 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-22T13:38:35.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.1 2024-04-22T13:38:35.866 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:35 smithi049 systemd[1]: Stopping Ceph osd.1 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:36.270 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1[35957]: 2024-04-22T13:38:35.870+0000 7f668fd64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:36.271 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1[35957]: 2024-04-22T13:38:35.870+0000 7f668fd64700 -1 osd.1 106 *** Got signal Terminated *** 2024-04-22T13:38:36.271 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1[35957]: 2024-04-22T13:38:35.870+0000 7f668fd64700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:38:41.771 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:41 smithi049 podman[66925]: 2024-04-22 13:38:41.337909702 +0000 UTC m=+5.581172607 container died 99e9197c9ab3471898bfffb78f9015479de2281de8a022bfa21ffe8ad659e6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-22T13:38:42.050 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:41 smithi049 podman[66925]: 2024-04-22 13:38:41.870591354 +0000 UTC m=+6.113854300 container cleanup 99e9197c9ab3471898bfffb78f9015479de2281de8a022bfa21ffe8ad659e6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:38:42.050 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:41 smithi049 bash[66925]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1 2024-04-22T13:38:42.476 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:42 smithi049 podman[66955]: 2024-04-22 13:38:42.05513274 +0000 UTC m=+0.715122190 container remove 99e9197c9ab3471898bfffb78f9015479de2281de8a022bfa21ffe8ad659e6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:38:42.771 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:42 smithi049 podman[67048]: 2024-04-22 13:38:42.481038387 +0000 UTC m=+0.110424399 container create 3b756fbf859d1640fd1eadc310462b0bece523527b1e007f3c8305be504791e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-22T13:38:42.771 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:42 smithi049 podman[67048]: 2024-04-22 13:38:42.389217016 +0000 UTC m=+0.018603048 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:38:42.771 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:42 smithi049 podman[67048]: 2024-04-22 13:38:42.709164747 +0000 UTC m=+0.338550760 container init 3b756fbf859d1640fd1eadc310462b0bece523527b1e007f3c8305be504791e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-22T13:38:42.771 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:42 smithi049 podman[67048]: 2024-04-22 13:38:42.712694241 +0000 UTC m=+0.342080258 container start 3b756fbf859d1640fd1eadc310462b0bece523527b1e007f3c8305be504791e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-22T13:38:42.771 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:42 smithi049 podman[67048]: 2024-04-22 13:38:42.775791451 +0000 UTC m=+0.405177464 container attach 3b756fbf859d1640fd1eadc310462b0bece523527b1e007f3c8305be504791e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-22T13:38:43.029 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:43 smithi049 podman[67048]: 2024-04-22 13:38:43.014897991 +0000 UTC m=+0.644283994 container died 3b756fbf859d1640fd1eadc310462b0bece523527b1e007f3c8305be504791e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-22T13:38:43.539 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.1.service' 2024-04-22T13:38:43.562 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:43 smithi049 podman[67075]: 2024-04-22 13:38:43.521712666 +0000 UTC m=+0.500961747 container remove 3b756fbf859d1640fd1eadc310462b0bece523527b1e007f3c8305be504791e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-22T13:38:43.562 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:43 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-22T13:38:43.563 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:43 smithi049 systemd[1]: Stopped Ceph osd.1 for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:38:43.563 INFO:journalctl@ceph.osd.1.smithi049.stdout:Apr 22 13:38:43 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.1.service: Consumed 12.785s CPU time. 2024-04-22T13:38:43.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:43.580 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-22T13:38:43.580 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-22T13:38:43.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.2 2024-04-22T13:38:43.910 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:43 smithi107 systemd[1]: Stopping Ceph osd.2 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:44.284 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2[37029]: 2024-04-22T13:38:43.914+0000 7f18ae020700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:44.284 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2[37029]: 2024-04-22T13:38:43.914+0000 7f18ae020700 -1 osd.2 106 *** Got signal Terminated *** 2024-04-22T13:38:44.284 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2[37029]: 2024-04-22T13:38:43.914+0000 7f18ae020700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:38:49.534 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:49 smithi107 podman[69467]: 2024-04-22 13:38:49.247812008 +0000 UTC m=+5.464516065 container died 6533a63b67df898e6744856ba65577dcf568bb552768510407c1d5d139e2717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T13:38:50.034 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:49 smithi107 podman[69467]: 2024-04-22 13:38:49.563201112 +0000 UTC m=+5.779905180 container cleanup 6533a63b67df898e6744856ba65577dcf568bb552768510407c1d5d139e2717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-22T13:38:50.034 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:49 smithi107 bash[69467]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2 2024-04-22T13:38:50.034 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:49 smithi107 podman[69482]: 2024-04-22 13:38:49.739228228 +0000 UTC m=+0.485470793 container remove 6533a63b67df898e6744856ba65577dcf568bb552768510407c1d5d139e2717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T13:38:50.395 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:50 smithi107 podman[69576]: 2024-04-22 13:38:50.04162415 +0000 UTC m=+0.017793916 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:38:50.395 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:50 smithi107 podman[69576]: 2024-04-22 13:38:50.173587422 +0000 UTC m=+0.149757174 container create 05eb7b7dd60a8384ec4c3e752173155f6059c6e21acf5916ca11aea3cae12684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T13:38:50.396 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:50 smithi107 podman[69576]: 2024-04-22 13:38:50.400687632 +0000 UTC m=+0.376857382 container init 05eb7b7dd60a8384ec4c3e752173155f6059c6e21acf5916ca11aea3cae12684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-22T13:38:50.699 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:50 smithi107 podman[69576]: 2024-04-22 13:38:50.404276552 +0000 UTC m=+0.380446303 container start 05eb7b7dd60a8384ec4c3e752173155f6059c6e21acf5916ca11aea3cae12684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-22T13:38:50.700 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:50 smithi107 podman[69576]: 2024-04-22 13:38:50.500836851 +0000 UTC m=+0.477006604 container attach 05eb7b7dd60a8384ec4c3e752173155f6059c6e21acf5916ca11aea3cae12684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-22T13:38:50.990 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 13:38:50 smithi107 podman[69576]: 2024-04-22 13:38:50.704974241 +0000 UTC m=+0.681143994 container died 05eb7b7dd60a8384ec4c3e752173155f6059c6e21acf5916ca11aea3cae12684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-22T13:38:51.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.2.service' 2024-04-22T13:38:51.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:51.354 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-22T13:38:51.355 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-22T13:38:51.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.3 2024-04-22T13:38:51.678 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:51 smithi005 systemd[1]: Stopping Ceph osd.3 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:51.961 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:38:51.682+0000 7faf94dcc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:38:51.961 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:38:51.682+0000 7faf94dcc700 -1 osd.3 106 *** Got signal Terminated *** 2024-04-22T13:38:51.961 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:51 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:38:51.682+0000 7faf94dcc700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:38:54.058 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:54 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:38:54.014+0000 7faf8d3b8700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.382556+0000 front 2024-04-22T13:38:32.382516+0000 (oldest deadline 2024-04-22T13:38:53.482638+0000) 2024-04-22T13:38:54.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:53 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:53.808+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:38:55.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:38:54 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:38:54.703+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:38:55.308 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:55 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:38:55.008+0000 7faf8d3b8700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.382556+0000 front 2024-04-22T13:38:32.382516+0000 (oldest deadline 2024-04-22T13:38:53.482638+0000) 2024-04-22T13:38:55.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:54 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:54.816+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:38:55.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:38:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:38:55.326+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:38:56.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:38:55 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:38:55.709+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:38:56.057 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:56 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3[49507]: 2024-04-22T13:38:56.028+0000 7faf8d3b8700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.382556+0000 front 2024-04-22T13:38:32.382516+0000 (oldest deadline 2024-04-22T13:38:53.482638+0000) 2024-04-22T13:38:56.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:55 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:55.787+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:38:56.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:38:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:38:56.338+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:38:57.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:38:56 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:38:56.744+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:38:57.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:56 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:56.838+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:38:57.308 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:57 smithi005 podman[101719]: 2024-04-22 13:38:57.051219974 +0000 UTC m=+5.492255640 container died 24ba3df5a772985a923a213ae49dc8d75a0584d2d70547104fdc45eedf3f9fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T13:38:57.313 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:38:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:38:57.047+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:38:57.570 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:57 smithi005 podman[101719]: 2024-04-22 13:38:57.390642899 +0000 UTC m=+5.831678554 container cleanup 24ba3df5a772985a923a213ae49dc8d75a0584d2d70547104fdc45eedf3f9fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True) 2024-04-22T13:38:57.571 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:57 smithi005 bash[101719]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3 2024-04-22T13:38:57.571 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:38:57 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:38:57.404+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:38:57.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:38:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:38:57.317+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:38:58.009 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:57 smithi005 podman[101748]: 2024-04-22 13:38:57.575108263 +0000 UTC m=+0.513587870 container remove 24ba3df5a772985a923a213ae49dc8d75a0584d2d70547104fdc45eedf3f9fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:38:58.010 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:57 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:57.811+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:38:58.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:38:57 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:38:57.764+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:38:58.308 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:58 smithi005 podman[101838]: 2024-04-22 13:38:57.917792231 +0000 UTC m=+0.018810929 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:38:58.308 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:58 smithi005 podman[101838]: 2024-04-22 13:38:58.076209879 +0000 UTC m=+0.177228617 container create 9430138c93fa0a6858d14565fa99cfd0e015416a154b2278d77c95e73bfca832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-22T13:38:58.534 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:38:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:38:58.281+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:38:58.534 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:38:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:38:58.092+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:38:58.647 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:38:58 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:38:58.407+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:38:58.647 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:58 smithi005 podman[101838]: 2024-04-22 13:38:58.347304834 +0000 UTC m=+0.448323522 container init 9430138c93fa0a6858d14565fa99cfd0e015416a154b2278d77c95e73bfca832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-22T13:38:58.647 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:58 smithi005 podman[101838]: 2024-04-22 13:38:58.351163182 +0000 UTC m=+0.452181868 container start 9430138c93fa0a6858d14565fa99cfd0e015416a154b2278d77c95e73bfca832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-04-22T13:38:58.647 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:58 smithi005 podman[101838]: 2024-04-22 13:38:58.461661471 +0000 UTC m=+0.562680151 container attach 9430138c93fa0a6858d14565fa99cfd0e015416a154b2278d77c95e73bfca832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-22T13:38:59.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:38:58 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:38:58.723+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:38:59.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:58 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:58.785+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:38:59.058 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 22 13:38:58 smithi005 podman[101838]: 2024-04-22 13:38:58.652290689 +0000 UTC m=+0.753309383 container died 9430138c93fa0a6858d14565fa99cfd0e015416a154b2278d77c95e73bfca832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T13:38:59.310 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:38:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:38:59.052+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:38:59.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.3.service' 2024-04-22T13:38:59.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:38:59.452 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-22T13:38:59.453 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-22T13:38:59.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.4 2024-04-22T13:38:59.744 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:38:59 smithi049 systemd[1]: Stopping Ceph osd.4 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:38:59.782 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:38:59 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:38:59.373+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:38:59.783 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:38:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:38:59.314+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:00.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:38:59 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:38:59.749+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:00.021 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:38:59 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4[42642]: 2024-04-22T13:38:59.790+0000 7f0b2cdf3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:00.021 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:38:59 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4[42642]: 2024-04-22T13:38:59.790+0000 7f0b2cdf3700 -1 osd.4 106 *** Got signal Terminated *** 2024-04-22T13:39:00.021 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:38:59 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4[42642]: 2024-04-22T13:38:59.790+0000 7f0b2cdf3700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:00.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:38:59 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:38:59.786+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:00.534 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:00 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:00.057+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:00.534 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:00 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:00.284+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:00.785 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:00.332+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:01.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:00 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:00.752+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:01.057 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:00 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:00.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:01.308 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:01 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:01.041+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:01.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:01 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:01.312+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:01.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:01 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:01.357+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:02.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:01 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:01.762+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:02.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:02.016+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:02.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:01 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:01.835+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:02.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:02.332+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:02.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:02.332+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:02.800 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:02.395+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:03.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:02 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:02.804+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:03.270 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:02 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:02.808+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:03.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:02.972+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:03.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:02 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:02.972+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:03.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:03 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:03.382+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:03.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:03 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:03.382+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:03.786 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:03.426+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:03.786 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:03.426+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:04.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:03.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:04.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:03.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:04.148 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:03 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4[42642]: 2024-04-22T13:39:03.786+0000 7f0b253df700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:38.272653+0000 front 2024-04-22T13:38:38.272615+0000 (oldest deadline 2024-04-22T13:39:03.572751+0000) 2024-04-22T13:39:04.149 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:03 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:03.822+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:04.149 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:03 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:03.822+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:04.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:03 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:03.883+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:04.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:03 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:03.964+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:04.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:03 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:03.964+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:04.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:04.152+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:04.763 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:04.459+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:04.764 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:04.459+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:04.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:04.413+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:04.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:04.413+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:05.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:04.767+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:05.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:04 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:04.767+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:05.115 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:04.808+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:05.115 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:04 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:04.808+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:05.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:04.900+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:05.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:04.972+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:05.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:04 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:04.972+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:05.396 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:05 smithi049 podman[67153]: 2024-04-22 13:39:05.120472658 +0000 UTC m=+5.458444189 container died c8bb43c60ee23d812ee79e3338fc277fdb805deeee9e353d0b34d5b1fa0c2cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-22T13:39:05.397 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:05.175+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:05.554 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:05.430+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:05.554 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:05.430+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:05.770 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:05 smithi049 podman[67153]: 2024-04-22 13:39:05.499389452 +0000 UTC m=+5.837360983 container cleanup c8bb43c60ee23d812ee79e3338fc277fdb805deeee9e353d0b34d5b1fa0c2cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-22T13:39:05.770 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:05 smithi049 bash[67153]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4 2024-04-22T13:39:05.807 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:05.508+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:05.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:05.508+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:05.808 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:05.721+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:05.808 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:05.721+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:06.169 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:05 smithi049 podman[67170]: 2024-04-22 13:39:05.867159134 +0000 UTC m=+0.744489890 container remove c8bb43c60ee23d812ee79e3338fc277fdb805deeee9e353d0b34d5b1fa0c2cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:39:06.169 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:05.813+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:06.169 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:05 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:05.813+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:06.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:05.944+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:06.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:05.952+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:06.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:05 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:05.952+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:06.485 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:06 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:06.172+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:06.486 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:06 smithi049 podman[67258]: 2024-04-22 13:39:06.18177185 +0000 UTC m=+0.019323403 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:06.486 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:06 smithi049 podman[67258]: 2024-04-22 13:39:06.284772904 +0000 UTC m=+0.122324485 container create 3e1636b3a18a17395a7f0dec261a3ba46fee3d7aeaf5a930acdf0b64c569a3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-22T13:39:06.486 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:06 smithi049 podman[67258]: 2024-04-22 13:39:06.48856817 +0000 UTC m=+0.326119712 container init 3e1636b3a18a17395a7f0dec261a3ba46fee3d7aeaf5a930acdf0b64c569a3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T13:39:06.761 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:06 smithi049 podman[67258]: 2024-04-22 13:39:06.492033094 +0000 UTC m=+0.329584634 container start 3e1636b3a18a17395a7f0dec261a3ba46fee3d7aeaf5a930acdf0b64c569a3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-22T13:39:06.762 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:06 smithi049 podman[67258]: 2024-04-22 13:39:06.588115346 +0000 UTC m=+0.425666897 container attach 3e1636b3a18a17395a7f0dec261a3ba46fee3d7aeaf5a930acdf0b64c569a3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T13:39:06.762 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:06 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:06.764+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:06.762 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:06 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:06.764+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:06.783 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:06.380+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:06.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:06.380+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:06.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:06.463+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:06.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:06.463+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:06.808 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:06.697+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:06.808 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:06 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:06.697+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:07.020 INFO:journalctl@ceph.osd.4.smithi049.stdout:Apr 22 13:39:06 smithi049 podman[67258]: 2024-04-22 13:39:06.809178691 +0000 UTC m=+0.646730231 container died 3e1636b3a18a17395a7f0dec261a3ba46fee3d7aeaf5a930acdf0b64c569a3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-22T13:39:07.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:06.980+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:07.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:06.908+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:07.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:06 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:06.908+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:07.463 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:07.166+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:07.485 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.4.service' 2024-04-22T13:39:07.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:07.527 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-22T13:39:07.527 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-22T13:39:07.527 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.5 2024-04-22T13:39:07.728 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:07.480+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:07.728 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:07.480+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:07.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:07.366+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:07.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:07.366+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:07.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:07 smithi107 systemd[1]: Stopping Ceph osd.5 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:08.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:07.785+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:08.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:07 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:07.785+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:08.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:07.732+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:08.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:07 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:07.732+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:08.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:08.022+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:08.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:07.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:08.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:07.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:08.284 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:07.795+0000 7fb380674700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:08.284 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:07.795+0000 7fb380674700 -1 osd.5 106 *** Got signal Terminated *** 2024-04-22T13:39:08.285 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:07 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:07.795+0000 7fb380674700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:08.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:08 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:08.163+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:08.733 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:08.460+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:08.733 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:08.460+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:08.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:08.368+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:08.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:08.368+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:09.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:08 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:08.815+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:09.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:08 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:08.815+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:09.057 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:08.737+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:09.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:08 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:08.737+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:09.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:09 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:09.018+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:09.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:08.920+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:09.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:08 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:08.920+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:09.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:09 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:09.140+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:09.766 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:09.425+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:09.766 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:09.425+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:09.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:09 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:09.393+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:09.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:09 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:09.393+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:10.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:09.770+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:10.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:09.770+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:10.144 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:09 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:09.801+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:10.144 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:09 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:09.801+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:10.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:09 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:09.988+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:10.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:09 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:09.884+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:10.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:09 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:09.884+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:10.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:10 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:10.148+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:10.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:10 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:10.443+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:10.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:10 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:10.443+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:10.787 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:10.436+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:10.787 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:10.436+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:10.787 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:10.436+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:11.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:10.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:11.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:10 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:10.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:11.168 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:10 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:10.801+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:11.168 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:10 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:10.801+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:11.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:11 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:11.028+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:11.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:10 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:10.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:11.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:10 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:10.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:11.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:10 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:10.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:11.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:11 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:11.172+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:11.754 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:11.431+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:11.754 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:11.431+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:11.754 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:11.431+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:11.783 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:11 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:11.454+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:11.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:11 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:11.454+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:12.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:11.758+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:12.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:11 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:11.758+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:12.160 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:11 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:11.805+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:12.160 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:11 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:11.805+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:12.160 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:11 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:11.805+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:12.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:12.064+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:12.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:11 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:11.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:12.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:11 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:11.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:12.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:11 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:11.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:12.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:12 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:12.164+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:12.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:12.411+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:29.788104+0000 front 2024-04-22T13:38:29.787960+0000 (oldest deadline 2024-04-22T13:38:55.088106+0000) 2024-04-22T13:39:12.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:12.411+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.389039+0000 front 2024-04-22T13:38:37.389197+0000 (oldest deadline 2024-04-22T13:39:01.989749+0000) 2024-04-22T13:39:12.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5[43965]: 2024-04-22T13:39:12.411+0000 7fb378c60700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.791242+0000 front 2024-04-22T13:38:47.791513+0000 (oldest deadline 2024-04-22T13:39:11.891458+0000) 2024-04-22T13:39:12.801 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:12.451+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:12.801 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:12.451+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:12.801 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:12.451+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:13.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:12.805+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:13.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:12 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:12.805+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:13.192 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:12 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:12.807+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:13.192 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:12 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:12.807+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:13.192 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:12 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:12.807+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:13.280 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:13 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:13.066+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:13.280 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:13 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:13.066+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:13.280 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:12.882+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:13.280 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:12.882+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:13.280 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:12 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:12.882+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:13.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:13.196+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:13.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:13.196+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:13.534 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:13 smithi107 podman[69694]: 2024-04-22 13:39:13.284961737 +0000 UTC m=+5.567320765 container died 99abbce2d72e725998906c0d247b15305c49d10ae1ba5c1d0cb5beaff08eefac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-22T13:39:13.802 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:13.418+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:13.802 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:13.418+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:13.802 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:13.418+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:13.904 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:13 smithi107 podman[69694]: 2024-04-22 13:39:13.65408212 +0000 UTC m=+5.936441137 container cleanup 99abbce2d72e725998906c0d247b15305c49d10ae1ba5c1d0cb5beaff08eefac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:39:13.905 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:13 smithi107 bash[69694]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5 2024-04-22T13:39:14.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:13.806+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:14.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:13.806+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:14.212 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:13.805+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:14.212 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:13.805+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:14.212 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:13 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:13.805+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:14.276 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:13 smithi107 podman[69711]: 2024-04-22 13:39:13.921728886 +0000 UTC m=+0.635413978 container remove 99abbce2d72e725998906c0d247b15305c49d10ae1ba5c1d0cb5beaff08eefac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:39:14.276 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:14 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:14.071+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:14.277 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:14 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:14.071+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:14.277 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:13 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:13.908+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:14.277 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:13 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:13.908+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:14.277 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:13 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:13.908+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:14.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:14 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:14.216+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:14.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:14 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:14.216+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:14.527 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:14 smithi107 podman[69802]: 2024-04-22 13:39:14.184603029 +0000 UTC m=+0.017928460 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:14.527 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:14 smithi107 podman[69802]: 2024-04-22 13:39:14.297431891 +0000 UTC m=+0.130757340 container create e48f008660f5ec2c6530741546401a501a40c1b6e1e856401d630d8dfdce5ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-22T13:39:14.760 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:14.381+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:14.760 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:14.381+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:14.760 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:14.381+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:14.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:14 smithi107 podman[69802]: 2024-04-22 13:39:14.533061215 +0000 UTC m=+0.366386622 container init e48f008660f5ec2c6530741546401a501a40c1b6e1e856401d630d8dfdce5ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-22T13:39:14.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:14 smithi107 podman[69802]: 2024-04-22 13:39:14.536153192 +0000 UTC m=+0.369478608 container start e48f008660f5ec2c6530741546401a501a40c1b6e1e856401d630d8dfdce5ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T13:39:14.784 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:14 smithi107 podman[69802]: 2024-04-22 13:39:14.616288842 +0000 UTC m=+0.449614269 container attach e48f008660f5ec2c6530741546401a501a40c1b6e1e856401d630d8dfdce5ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-22T13:39:15.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:14.764+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:15.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:14.764+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:15.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:14 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:14.764+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:15.093 INFO:journalctl@ceph.osd.5.smithi107.stdout:Apr 22 13:39:14 smithi107 podman[69802]: 2024-04-22 13:39:14.830137574 +0000 UTC m=+0.663462973 container died e48f008660f5ec2c6530741546401a501a40c1b6e1e856401d630d8dfdce5ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:39:15.094 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:14 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:14.903+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:15.094 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:14 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:14.903+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:15.094 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:14 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:14.903+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:15.247 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:14 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:14.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:15.247 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:14 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:14.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:15.247 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:14 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:14.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:15.406 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:15 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:15.097+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:15.406 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:15 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:15.097+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:15.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.5.service' 2024-04-22T13:39:15.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:15.470 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-22T13:39:15.470 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-22T13:39:15.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.6 2024-04-22T13:39:15.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:15.251+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:15.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:15.251+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:15.779 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:15.342+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:15.779 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:15.342+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:15.779 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:15.342+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:15.779 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 systemd[1]: Stopping Ceph osd.6 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:16.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:15.749+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:16.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:15.749+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:16.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:15 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:15.749+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:16.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:15.798+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:16.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:15.798+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:16.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:15.798+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:16.058 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:15.783+0000 7f38d533f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:16.058 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:15.783+0000 7f38d533f700 -1 osd.6 106 *** Got signal Terminated *** 2024-04-22T13:39:16.058 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:15 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:15.783+0000 7f38d533f700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:16.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:15 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:15.880+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:16.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:15 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:15.880+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:16.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:15 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:15.880+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:16.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:16.262+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:16.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:16.262+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:16.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:16.142+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:16.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:16.142+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:16.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:16.370+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:16.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:16.370+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:16.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:16.370+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:17.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:16.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:17.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:16.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:17.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:16 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:16.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:17.159 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:16.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:17.160 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:16.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:17.160 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:16 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:16.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:17.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:16.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:17.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:16.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:17.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:16 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:16.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:17.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:17 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:17.164+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:17.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:17 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:17.164+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:17.753 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:17.278+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:17.753 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:17.278+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:17.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:17 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:17.325+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:17.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:17 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:17.325+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:17.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:17 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:17.325+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:18.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:17.757+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:18.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:17.757+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:18.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:17.757+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:18.197 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:17 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:17.871+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:18.197 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:17 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:17.871+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:18.198 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:17 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:17.871+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:18.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:17 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:17.839+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:18.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:17 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:17.839+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:18.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:17 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:17.839+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:18.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:18.289+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:18.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:18.289+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:18.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:18.289+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:18.533 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:18 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:18.202+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:18.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:18 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:18.202+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:18.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:18.372+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:18.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:18.372+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:18.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:18.372+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:18.809 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:18.372+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.663615+0000 front 2024-04-22T13:38:55.663648+0000 (oldest deadline 2024-04-22T13:39:17.363814+0000) 2024-04-22T13:39:19.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:18 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:18.898+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:19.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:18 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:18.898+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:19.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:18 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:18.898+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:19.163 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:18 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:18.898+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:19.270 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:18.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:19.270 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:18.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:19.271 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:18.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:19.271 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:18 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:18.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:19.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:18.842+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:19.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:18.842+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:19.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:18 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:18.842+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:19.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:19.167+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:19.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:19.167+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:19.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:19.167+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:19.763 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:19.304+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:19.763 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:19.304+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:19.763 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:19.304+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:19.800 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:19.333+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:19.800 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:19.333+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:19.801 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:19.333+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:19.801 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:19.333+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.663615+0000 front 2024-04-22T13:38:55.663648+0000 (oldest deadline 2024-04-22T13:39:17.363814+0000) 2024-04-22T13:39:20.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:19.767+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:20.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:19.767+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:20.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:19.767+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:20.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:19 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:19.767+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:20.057 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:19.804+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:20.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:19.804+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:20.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:19.804+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:20.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:19 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:19.804+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:20.190 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:19.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:20.191 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:19.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:20.191 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:19.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:20.191 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:19 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:19.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:20.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:20.258+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:20.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:20.258+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:20.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:20.258+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:20.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:20.195+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:20.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:20.195+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:20.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:20.195+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:20.557 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:20.294+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.459132+0000 front 2024-04-22T13:38:32.459046+0000 (oldest deadline 2024-04-22T13:38:56.559213+0000) 2024-04-22T13:39:20.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:20.294+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:36.559664+0000 front 2024-04-22T13:38:36.559824+0000 (oldest deadline 2024-04-22T13:39:02.459831+0000) 2024-04-22T13:39:20.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:20.294+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:43.560944+0000 front 2024-04-22T13:38:43.561201+0000 (oldest deadline 2024-04-22T13:39:09.461063+0000) 2024-04-22T13:39:20.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6[56527]: 2024-04-22T13:39:20.294+0000 7f38d1933700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.663615+0000 front 2024-04-22T13:38:55.663648+0000 (oldest deadline 2024-04-22T13:39:17.363814+0000) 2024-04-22T13:39:21.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:20.740+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:21.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:20.740+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:21.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:20.740+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:21.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:20 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:20.740+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:21.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:20.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:21.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:20.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:21.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:20.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:21.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:20 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:20.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:21.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:20.868+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:21.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:20.868+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:21.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:20.868+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:21.162 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:20 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:20.868+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:21.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:21.234+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:21.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:21.234+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:21.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:21.234+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:21.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:21.166+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:21.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:21.166+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:21.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:21.166+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:21.629 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:21 smithi005 podman[101942]: 2024-04-22 13:39:21.340538381 +0000 UTC m=+5.667553635 container died 44399764a50d9a7145bd98c737ffec43391cb4fe01c1c4d3d877f3830240733b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-22T13:39:21.954 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:21 smithi005 podman[101942]: 2024-04-22 13:39:21.724932356 +0000 UTC m=+6.051947618 container cleanup 44399764a50d9a7145bd98c737ffec43391cb4fe01c1c4d3d877f3830240733b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T13:39:21.954 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:21 smithi005 bash[101942]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6 2024-04-22T13:39:21.954 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:21.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:21.954 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:21.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:21.955 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:21.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:21.955 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:21 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:21.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:22.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:21.744+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:22.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:21.744+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:22.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:21.744+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:22.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:21.744+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:22.121 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:21.855+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:22.121 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:21.855+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:22.121 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:21.855+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:22.121 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:21 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:21.855+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:22.286 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:21 smithi005 podman[101961]: 2024-04-22 13:39:21.959323549 +0000 UTC m=+0.620672329 container remove 44399764a50d9a7145bd98c737ffec43391cb4fe01c1c4d3d877f3830240733b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-22T13:39:22.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:22.233+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:22.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:22.233+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:22.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:22.233+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:22.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:22.125+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:22.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:22.125+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:22.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:22.125+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:22.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:22 smithi005 podman[102062]: 2024-04-22 13:39:22.19391922 +0000 UTC m=+0.018956507 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:22.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:22 smithi005 podman[102062]: 2024-04-22 13:39:22.302026752 +0000 UTC m=+0.127064031 container create c270c5c0385393748fde0ec4a2682e021a70a56f1af6e0c064dc094846a48618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:39:22.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:22 smithi005 podman[102062]: 2024-04-22 13:39:22.504368896 +0000 UTC m=+0.329406174 container init c270c5c0385393748fde0ec4a2682e021a70a56f1af6e0c064dc094846a48618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-22T13:39:22.558 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:22 smithi005 podman[102062]: 2024-04-22 13:39:22.507687131 +0000 UTC m=+0.332724413 container start c270c5c0385393748fde0ec4a2682e021a70a56f1af6e0c064dc094846a48618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T13:39:23.058 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:22 smithi005 podman[102062]: 2024-04-22 13:39:22.581903719 +0000 UTC m=+0.406940994 container attach c270c5c0385393748fde0ec4a2682e021a70a56f1af6e0c064dc094846a48618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-22T13:39:23.059 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 22 13:39:22 smithi005 podman[102062]: 2024-04-22 13:39:22.811465544 +0000 UTC m=+0.636502813 container died c270c5c0385393748fde0ec4a2682e021a70a56f1af6e0c064dc094846a48618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T13:39:23.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:22.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:23.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:22.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:23.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:22.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:23.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:22 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:22.783+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:23.109 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:22.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:23.109 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:22.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:23.109 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:22.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:23.109 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:22 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:22.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:23.266 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:22.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:23.267 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:22.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:23.267 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:22.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:23.267 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:22 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:22.777+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:23.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.6.service' 2024-04-22T13:39:23.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:23.465 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-22T13:39:23.465 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-22T13:39:23.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.7 2024-04-22T13:39:23.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:23.271+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:23.520 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:23.271+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:23.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:23.271+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:23.521 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 systemd[1]: Stopping Ceph osd.7 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:23.533 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:23.113+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:23.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:23.113+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:23.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:23.113+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:24.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:23.806+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:24.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:23.806+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:24.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:23.806+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:24.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:23.806+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:24.021 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:23.754+0000 7f761cbb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:24.021 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:23.754+0000 7f761cbb5700 -1 osd.7 106 *** Got signal Terminated *** 2024-04-22T13:39:24.021 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:23 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:23.754+0000 7f761cbb5700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:24.057 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:23.786+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:24.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:23.786+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:24.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:23.786+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:24.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:23 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:23.786+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:24.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:24.098+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:24.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:24.098+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:24.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:24.098+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:24.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:23.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:24.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:23.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:24.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:23.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:24.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:23 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:23.891+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:24.766 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:24.294+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:24.766 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:24.294+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:24.766 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:24.294+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:25.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:24.770+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:25.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:24.770+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:25.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:24.770+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:25.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:24 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:24.770+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:25.057 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:24.746+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:25.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:24.746+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:25.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:24.746+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:25.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:24 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:24.746+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:25.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:25.073+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:25.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:25.073+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:25.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:25.073+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:25.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:24.887+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:25.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:24.887+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:25.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:24.887+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:25.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:24 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:24.887+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:25.742 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:25.314+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:25.742 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:25.314+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:25.742 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:25.314+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:26.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:25.746+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:26.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:25.746+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:26.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:25.746+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:26.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:25.746+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:26.057 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:25 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:25.770+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:26.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:25 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:25.770+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:26.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:25 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:25.770+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:26.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:25 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:25.770+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:26.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:26.100+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:26.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:26.100+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:26.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:26.100+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:26.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:26.100+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:26.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:25.927+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:26.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:25.927+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:26.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:25.927+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:26.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:25 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:25.927+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:26.703 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:26.347+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:26.703 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:26.347+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:26.704 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:26.347+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:27.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:26.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:27.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:26.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:27.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:26.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:27.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:26.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:27.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:26 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:26.707+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:27.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:26.768+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:27.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:26.768+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:27.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:26.768+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:27.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:26.768+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:27.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:26 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:26.768+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:27.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:27.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:27.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:27.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:26.956+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:26.956+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:27.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:26.956+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:27.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:26.956+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:27.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:26 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:26.956+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:27.659 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:27.308+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:27.659 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:27.308+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:27.659 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:27.308+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:28.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:27.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:28.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:27.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:28.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:27.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:28.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:27.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:28.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:27 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:27.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:28.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:28.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:28.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:28.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:28.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:28.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:28.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:28.096+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:28.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:27.909+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:28.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:27.909+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:28.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:27.909+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:28.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:27.909+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:28.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:27 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:27.909+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:28.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:27.819+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:28.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:27.819+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:28.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:27.819+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:28.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:27.819+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:28.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:27 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:27.819+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:28.638 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:28.327+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.021971+0000 front 2024-04-22T13:38:40.022103+0000 (oldest deadline 2024-04-22T13:39:04.122187+0000) 2024-04-22T13:39:28.638 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:28.327+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.223619+0000 front 2024-04-22T13:38:48.223441+0000 (oldest deadline 2024-04-22T13:39:12.923515+0000) 2024-04-22T13:39:28.638 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7[49201]: 2024-04-22T13:39:28.327+0000 7f76151a1700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.923932+0000 front 2024-04-22T13:38:52.923917+0000 (oldest deadline 2024-04-22T13:39:18.223981+0000) 2024-04-22T13:39:29.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:28.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:29.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:28.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:29.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:28.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:29.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:28.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:29.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:28 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:28.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:29.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:28 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:28.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:29.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:28 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:28.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:29.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:28 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:28.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:29.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:28 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:28.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:29.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:28 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:28.791+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:29.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:29.047+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:29.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:29.047+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:29.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:29.047+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:29.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:29.047+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:29.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:28.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:29.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:28.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:29.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:28.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:29.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:28.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:29.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:28 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:28.931+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:29.499 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:29 smithi049 podman[67374]: 2024-04-22 13:39:29.216119323 +0000 UTC m=+5.548594532 container died fd7333ac1fe0f9d2813d5579819f20b316f9088671286c7001469a2da4786166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T13:39:29.770 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:29 smithi049 podman[67374]: 2024-04-22 13:39:29.595407022 +0000 UTC m=+5.927882231 container cleanup fd7333ac1fe0f9d2813d5579819f20b316f9088671286c7001469a2da4786166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-22T13:39:29.771 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:29 smithi049 bash[67374]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7 2024-04-22T13:39:29.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:29.655+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:29.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:29.655+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:29.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:29.655+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:29.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:29.655+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:29.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:29.655+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:30.167 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:29 smithi049 podman[67392]: 2024-04-22 13:39:29.854776534 +0000 UTC m=+0.640303030 container remove fd7333ac1fe0f9d2813d5579819f20b316f9088671286c7001469a2da4786166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:39:30.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:30.051+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:30.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:30.051+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:30.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:30.051+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:30.285 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:30.051+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:30.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:29.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:30.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:29.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:30.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:29.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:30.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:29.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:30.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:29 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:29.918+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:30.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:29 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:29.812+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:30.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:29 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:29.812+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:30.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:29 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:29.812+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:30.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:29 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:29.812+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:30.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:29 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:29.812+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:30.504 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:30 smithi049 podman[67481]: 2024-04-22 13:39:30.172130066 +0000 UTC m=+0.105380056 container create afe622a1cc7f3328c22176afd658f5257e51564e8f1764dd13a68cc15bf7a55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:39:30.504 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:30 smithi049 podman[67481]: 2024-04-22 13:39:30.085542836 +0000 UTC m=+0.018792826 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:30.504 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:30 smithi049 podman[67481]: 2024-04-22 13:39:30.391235883 +0000 UTC m=+0.324485874 container init afe622a1cc7f3328c22176afd658f5257e51564e8f1764dd13a68cc15bf7a55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-22T13:39:30.504 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:30 smithi049 podman[67481]: 2024-04-22 13:39:30.394448417 +0000 UTC m=+0.327698408 container start afe622a1cc7f3328c22176afd658f5257e51564e8f1764dd13a68cc15bf7a55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-22T13:39:30.504 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:30 smithi049 podman[67481]: 2024-04-22 13:39:30.508926431 +0000 UTC m=+0.442176422 container attach afe622a1cc7f3328c22176afd658f5257e51564e8f1764dd13a68cc15bf7a55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-22T13:39:30.771 INFO:journalctl@ceph.osd.7.smithi049.stdout:Apr 22 13:39:30 smithi049 podman[67481]: 2024-04-22 13:39:30.703351189 +0000 UTC m=+0.636601179 container died afe622a1cc7f3328c22176afd658f5257e51564e8f1764dd13a68cc15bf7a55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-22T13:39:30.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:30.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:30.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:30.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:30.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:30.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:30.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:30.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:30.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:30 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:30.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:31.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.097+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:31.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.097+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:31.284 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.097+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:31.285 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.097+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:31.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:30.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:31.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:30.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:31.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:30.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:31.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:30.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:31.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:30 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:30.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:31.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:30 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:30.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:31.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:30 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:30.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:31.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:30 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:30.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:31.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:30 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:30.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:31.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:30 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:30.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:31.464 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.7.service' 2024-04-22T13:39:31.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:31.508 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-22T13:39:31.509 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-22T13:39:31.509 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.8 2024-04-22T13:39:31.735 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:31.608+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:31.735 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:31.608+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:31.735 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:31.608+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:31.735 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:31.608+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:31.735 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:31 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:31.608+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:31.798 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 systemd[1]: Stopping Ceph osd.8 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:32.137 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.802+0000 7f99881d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:32.137 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.802+0000 7f99881d3700 -1 osd.8 106 *** Got signal Terminated *** 2024-04-22T13:39:32.137 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:31.802+0000 7f99881d3700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:32.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:31.841+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:32.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:31.841+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:32.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:31.841+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:32.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:31.841+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:32.138 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:31 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:31.841+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:32.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:31.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:32.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:31.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:32.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:31.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:32.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:31.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:32.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:31 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:31.846+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:32.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:32.142+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:32.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:32.142+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:32.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:32.142+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:32.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:32.142+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:33.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:32.610+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:33.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:32.610+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:33.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:32.610+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:33.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:32.610+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:33.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:32 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:32.610+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:33.153 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:32.792+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:33.153 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:32.792+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:33.153 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:32.792+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:33.153 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:32.792+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:33.153 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:32 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:32.792+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:33.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:32 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:32.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:33.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:32 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:32.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:33.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:32 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:32.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:33.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:32 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:32.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:33.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:32 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:32.828+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:33.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:33.157+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:33.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:33.158+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:33.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:33.158+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:33.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:33.158+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:34.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:33.604+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:34.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:33.604+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:34.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:33.604+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:34.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:33.604+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:34.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:33 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:33.604+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:34.130 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:33.801+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:34.130 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:33.801+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:34.130 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:33.801+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:34.130 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:33.801+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:34.130 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:33 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:33.801+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:34.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:33 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:33.868+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:34.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:33 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:33.868+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:34.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:33 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:33.868+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:34.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:33 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:33.868+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:34.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:33 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:33.868+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:34.404 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:34.134+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:34.404 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:34.134+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:34.404 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:34.134+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:34.405 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:34.134+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:35.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:34 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:34.648+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:35.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:34 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:34.648+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:35.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:34 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:34.648+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:35.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:34 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:34.648+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:35.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:34 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:34.648+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:35.102 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:34.844+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:35.102 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:34.844+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:35.102 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:34.844+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:35.102 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:34.844+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:35.102 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:34.844+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:35.102 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:34.844+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:35.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:34 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:34.869+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:35.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:34 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:34.869+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:35.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:34 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:34.869+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:35.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:34 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:34.869+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:35.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:34 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:34.869+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:35.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:34 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:34.869+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:35.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:35.106+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:35.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:35.106+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:35.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:35.106+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:35.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:35.106+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:36.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:35.645+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:36.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:35.645+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:36.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:35.645+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:36.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:35.645+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:36.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:35 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:35.645+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:36.098 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:35.834+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:36.099 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:35.834+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:36.099 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:35.834+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:36.099 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:35.834+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:36.099 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:35.834+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:36.099 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:35 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:35.834+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:36.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:35 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:35.866+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:36.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:35 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:35.866+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:36.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:35 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:35.866+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:36.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:35 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:35.866+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:36.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:35 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:35.866+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:36.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:35 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:35.866+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:36.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:36.103+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:39.552995+0000 front 2024-04-22T13:38:39.553030+0000 (oldest deadline 2024-04-22T13:39:03.053025+0000) 2024-04-22T13:39:36.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:36.103+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.754024+0000 front 2024-04-22T13:38:47.754051+0000 (oldest deadline 2024-04-22T13:39:12.454241+0000) 2024-04-22T13:39:36.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:36.103+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:54.055850+0000 front 2024-04-22T13:38:54.055897+0000 (oldest deadline 2024-04-22T13:39:18.755963+0000) 2024-04-22T13:39:36.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:36.103+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.557480+0000 front 2024-04-22T13:39:04.557357+0000 (oldest deadline 2024-04-22T13:39:25.657421+0000) 2024-04-22T13:39:36.534 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8[50943]: 2024-04-22T13:39:36.103+0000 7f99807bf700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:09.658697+0000 front 2024-04-22T13:39:09.658759+0000 (oldest deadline 2024-04-22T13:39:35.558849+0000) 2024-04-22T13:39:36.922 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:36 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:36.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:36.922 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:36 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:36.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:36.922 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:36 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:36.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:36.922 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:36 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:36.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:36.922 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:36 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:36.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:36.923 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:36 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:36.612+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:37.174 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:36.794+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:37.174 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:36.794+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:37.174 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:36.794+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:37.174 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:36.794+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:37.174 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:36.794+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:37.175 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:36 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:36.794+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:37.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:36 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:36.882+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:37.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:36 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:36.882+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:37.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:36 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:36.882+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:37.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:36 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:36.882+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:37.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:36 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:36.882+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:37.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:36 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:36.882+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:37.432 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:37 smithi107 podman[69909]: 2024-04-22 13:39:37.178529323 +0000 UTC m=+5.469595618 container died bc5f5326c3d5615e1025883c118a89ec40768744f0592783f8f46781ce90ed0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T13:39:37.707 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:37 smithi107 podman[69909]: 2024-04-22 13:39:37.527413026 +0000 UTC m=+5.818479327 container cleanup bc5f5326c3d5615e1025883c118a89ec40768744f0592783f8f46781ce90ed0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:39:37.707 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:37 smithi107 bash[69909]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8 2024-04-22T13:39:38.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:37.563+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:38.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:37.563+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:38.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:37.563+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:38.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:37.563+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:38.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:37.563+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:38.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:37 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:37.563+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:37 smithi107 podman[69939]: 2024-04-22 13:39:37.711748442 +0000 UTC m=+0.521772171 container remove bc5f5326c3d5615e1025883c118a89ec40768744f0592783f8f46781ce90ed0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:37.837+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:37.837+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:37.837+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:37.837+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:37.837+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:38.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:37.837+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:38.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:37 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:37.861+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:38.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:37 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:37.861+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:38.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:37 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:37.861+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:38.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:37 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:37.861+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:38.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:37 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:37.861+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:38.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:37 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:37.861+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:38.451 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:38 smithi107 podman[70031]: 2024-04-22 13:39:38.042260504 +0000 UTC m=+0.018521953 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:38.452 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:38 smithi107 podman[70031]: 2024-04-22 13:39:38.14562275 +0000 UTC m=+0.121884163 container create 78d47709acf65ec27733e2390aec4b733c43a702a9ce024e04af71487038d21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:39:38.452 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:38 smithi107 podman[70031]: 2024-04-22 13:39:38.373114559 +0000 UTC m=+0.349375980 container init 78d47709acf65ec27733e2390aec4b733c43a702a9ce024e04af71487038d21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T13:39:38.452 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:38 smithi107 podman[70031]: 2024-04-22 13:39:38.376576927 +0000 UTC m=+0.352838338 container start 78d47709acf65ec27733e2390aec4b733c43a702a9ce024e04af71487038d21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-22T13:39:38.452 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:38 smithi107 podman[70031]: 2024-04-22 13:39:38.456342514 +0000 UTC m=+0.432603934 container attach 78d47709acf65ec27733e2390aec4b733c43a702a9ce024e04af71487038d21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:39:38.784 INFO:journalctl@ceph.osd.8.smithi107.stdout:Apr 22 13:39:38 smithi107 podman[70031]: 2024-04-22 13:39:38.67079766 +0000 UTC m=+0.647059072 container died 78d47709acf65ec27733e2390aec4b733c43a702a9ce024e04af71487038d21a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:39:39.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:38 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:38.577+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:39.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:38 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:38.577+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:39.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:38 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:38.577+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:39.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:38 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:38.577+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:39.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:38 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:38.577+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:39.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:38 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:38.577+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:39.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:38 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:38.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:39.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:38 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:38.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:39.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:38 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:38.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:39.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:38 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:38.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:39.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:38 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:38.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:39.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:38 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:38.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:39.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:38 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:38.827+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:39.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:38 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:38.827+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:39.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:38 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:38.827+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:39.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:38 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:38.827+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:39.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:38 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:38.827+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:39.309 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:38 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:38.827+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:39.309 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.8.service' 2024-04-22T13:39:39.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:39.349 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-22T13:39:39.349 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-22T13:39:39.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.9 2024-04-22T13:39:39.654 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 systemd[1]: Stopping Ceph osd.9 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:40.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:39 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:39.615+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:40.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:39 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:39.615+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:40.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:39 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:39.615+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:40.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:39 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:39.615+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:40.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:39 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:39.615+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:40.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:39 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:39.615+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:40.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.657+0000 7f8e70e8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:40.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.657+0000 7f8e70e8f700 -1 osd.9 106 *** Got signal Terminated *** 2024-04-22T13:39:40.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.657+0000 7f8e70e8f700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:40.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:40.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:40.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:40.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:40.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:40.059 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:39 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:39.782+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:40.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:39.881+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:40.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:39.881+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:40.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:39.881+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:40.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:39.881+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:40.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:39.881+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:40.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:39 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:39.881+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:41.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:40 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:40.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:41.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:40 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:40.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:41.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:40 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:40.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:41.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:40 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:40.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:41.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:40 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:40.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:41.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:40 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:40.642+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:41.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:40 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:40.913+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:41.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:40 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:40.913+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:41.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:40 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:40.913+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:41.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:40 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:40.913+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:41.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:40 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:40.913+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:41.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:40 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:40.913+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:41.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:40.830+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:41.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:40.830+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:41.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:40.830+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:41.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:40.830+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:41.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:40.830+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:41.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:40 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:40.830+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:42.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:41 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:41.656+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:42.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:41 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:41.656+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:42.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:41 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:41.656+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:42.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:41 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:41.656+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:42.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:41 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:41.656+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:42.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:41 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:41.656+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:42.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:41.879+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:42.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:41.879+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:42.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:41.879+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:42.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:41.879+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:42.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:41.879+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:42.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:41.879+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:42.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:41 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:41.862+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:42.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:41 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:41.862+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:42.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:41 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:41.862+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:42.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:41 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:41.862+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:42.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:41 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:41.862+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:42.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:41 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:41.862+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:43.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:43.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:43.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:43.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:43.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:43.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:43.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:42 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:42.653+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:43.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:42 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:42.857+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:43.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:42 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:42.857+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:43.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:42 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:42.857+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:43.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:42 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:42.857+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:43.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:42 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:42.857+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:43.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:42 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:42.857+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:43.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:42.824+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:43.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:42.824+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:43.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:42.824+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:43.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:42.824+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:43.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:42.824+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:43.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:42 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:42.824+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:44.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:44.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:44.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:44.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:44.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:44.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:44.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:43 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:43.630+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:44.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:43.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:27.322929+0000 front 2024-04-22T13:38:27.322885+0000 (oldest deadline 2024-04-22T13:38:53.222981+0000) 2024-04-22T13:39:44.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:43.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:40.824814+0000 front 2024-04-22T13:38:40.824909+0000 (oldest deadline 2024-04-22T13:39:03.724885+0000) 2024-04-22T13:39:44.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:43.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:48.227185+0000 front 2024-04-22T13:38:48.227065+0000 (oldest deadline 2024-04-22T13:39:14.127053+0000) 2024-04-22T13:39:44.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:43.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:55.828058+0000 front 2024-04-22T13:38:55.828086+0000 (oldest deadline 2024-04-22T13:39:19.328285+0000) 2024-04-22T13:39:44.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:43.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.229347+0000 front 2024-04-22T13:39:02.229244+0000 (oldest deadline 2024-04-22T13:39:26.329307+0000) 2024-04-22T13:39:44.058 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:43 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9[63570]: 2024-04-22T13:39:43.790+0000 7f8e6947b700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.630602+0000 front 2024-04-22T13:39:11.630697+0000 (oldest deadline 2024-04-22T13:39:34.530533+0000) 2024-04-22T13:39:44.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:43.875+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:44.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:43.875+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:44.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:43.875+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:44.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:43.875+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:44.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:43.875+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:44.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:43 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:43.875+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:45.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:45.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:45.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:45.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:45.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:45.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:45.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:44 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:44.680+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:45.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:44 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:44.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:45.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:44 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:44.869+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:45.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:44 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:44.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:45.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:44 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:44.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:45.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:44 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:44.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:45.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:44 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:44.870+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:45.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:45 smithi005 podman[102172]: 2024-04-22 13:39:45.079552211 +0000 UTC m=+5.528895983 container died 136304d1c66b5c02deec2981828531846ba112b572061db8a776eb933bb7473c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-22T13:39:45.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:45 smithi005 podman[102172]: 2024-04-22 13:39:45.400256761 +0000 UTC m=+5.849600532 container cleanup 136304d1c66b5c02deec2981828531846ba112b572061db8a776eb933bb7473c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T13:39:45.807 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:45 smithi005 bash[102172]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9 2024-04-22T13:39:45.808 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:45 smithi005 podman[102187]: 2024-04-22 13:39:45.576237758 +0000 UTC m=+0.498951358 container remove 136304d1c66b5c02deec2981828531846ba112b572061db8a776eb933bb7473c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T13:39:46.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:46.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:46.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:46.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:46.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:46.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:46.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:45 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:45.682+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:46.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:45.820+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:46.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:45.820+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:46.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:45.820+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:46.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:45.820+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:46.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:45.820+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:46.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:45.820+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:46.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:46 smithi005 podman[102278]: 2024-04-22 13:39:46.001908138 +0000 UTC m=+0.112226404 container create 338bdc15b06609d5e3c321df2d59ac5bf9383c961c957746741d7ee4452a7f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:39:46.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:46 smithi005 podman[102278]: 2024-04-22 13:39:45.907760511 +0000 UTC m=+0.018078791 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:46.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:46 smithi005 podman[102278]: 2024-04-22 13:39:46.229278773 +0000 UTC m=+0.339597034 container init 338bdc15b06609d5e3c321df2d59ac5bf9383c961c957746741d7ee4452a7f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-22T13:39:46.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:46 smithi005 podman[102278]: 2024-04-22 13:39:46.233143873 +0000 UTC m=+0.343462137 container start 338bdc15b06609d5e3c321df2d59ac5bf9383c961c957746741d7ee4452a7f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:39:46.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:46 smithi005 podman[102278]: 2024-04-22 13:39:46.312296882 +0000 UTC m=+0.422615154 container attach 338bdc15b06609d5e3c321df2d59ac5bf9383c961c957746741d7ee4452a7f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-22T13:39:46.558 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:46 smithi005 podman[102278]: 2024-04-22 13:39:46.534896614 +0000 UTC m=+0.645214874 container died 338bdc15b06609d5e3c321df2d59ac5bf9383c961c957746741d7ee4452a7f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T13:39:46.977 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:46.978 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:46.978 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:46.978 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:46.978 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:46.978 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:46.978 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:46 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:46.635+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:47.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:46 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:46.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:47.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:46 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:46.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:47.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:46 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:46.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:47.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:46 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:46.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:47.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:46 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:46.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:47.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:46 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:46.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:47.077 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 22 13:39:47 smithi005 podman[102304]: 2024-04-22 13:39:47.065865374 +0000 UTC m=+0.521438612 container remove 338bdc15b06609d5e3c321df2d59ac5bf9383c961c957746741d7ee4452a7f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T13:39:47.083 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.9.service' 2024-04-22T13:39:47.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:47.123 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-22T13:39:47.123 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-22T13:39:47.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.10 2024-04-22T13:39:47.270 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 systemd[1]: Stopping Ceph osd.10 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.437+0000 7f561e173700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.437+0000 7f561e173700 -1 osd.10 106 *** Got signal Terminated *** 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.437+0000 7f561e173700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:47.771 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:47 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:47.651+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:48.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:47 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:47.733+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:48.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:47 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:47.733+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:48.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:47 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:47.733+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:48.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:47 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:47.733+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:48.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:47 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:47.733+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:48.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:47 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:47.733+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.658+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.659+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.659+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.659+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.659+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.659+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:49.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:48 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:48.659+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:49.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:48 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:48.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:49.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:48 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:48.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:49.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:48 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:48.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:49.034 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:48 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:48.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:49.035 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:48 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:48.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:49.035 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:48 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:48.780+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:50.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:50.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:50.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:50.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:50.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:50.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:50.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:49 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:49.705+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:50.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:49.798+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:50.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:49.798+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:50.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:49.798+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:50.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:49.798+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:50.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:49.798+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:50.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:49 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:49.798+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:51.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:51.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:51.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:51.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:51.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:51.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:51.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:50 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:50.708+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:51.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:50 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:50.789+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:51.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:50 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:50.789+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:51.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:50 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:50.789+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:51.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:50 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:50.789+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:51.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:50 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:50.789+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:51.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:50 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:50.789+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:52.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:32.000242+0000 front 2024-04-22T13:38:32.000224+0000 (oldest deadline 2024-04-22T13:38:53.700298+0000) 2024-04-22T13:39:52.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.602553+0000 front 2024-04-22T13:38:37.602508+0000 (oldest deadline 2024-04-22T13:39:02.902641+0000) 2024-04-22T13:39:52.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:47.003881+0000 front 2024-04-22T13:38:47.003784+0000 (oldest deadline 2024-04-22T13:39:11.103944+0000) 2024-04-22T13:39:52.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:52.105645+0000 front 2024-04-22T13:38:52.105852+0000 (oldest deadline 2024-04-22T13:39:17.906452+0000) 2024-04-22T13:39:52.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:04.307913+0000 front 2024-04-22T13:39:04.307955+0000 (oldest deadline 2024-04-22T13:39:26.007937+0000) 2024-04-22T13:39:52.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:11.809745+0000 front 2024-04-22T13:39:11.809818+0000 (oldest deadline 2024-04-22T13:39:35.909722+0000) 2024-04-22T13:39:52.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:51 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10[55812]: 2024-04-22T13:39:51.663+0000 7f561675f700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-22T13:39:20.011055+0000 front 2024-04-22T13:39:20.010903+0000 (oldest deadline 2024-04-22T13:39:41.710749+0000) 2024-04-22T13:39:52.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:51.791+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:52.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:51.791+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:52.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:51.791+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:52.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:51.791+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:52.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:51.791+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:52.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:51 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:51.791+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:53.021 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:52 smithi049 podman[67598]: 2024-04-22 13:39:52.770906811 +0000 UTC m=+5.453067349 container died 917a5c6e2271de6d0200b6e09073fbc3560a19c084702dda3975966a24bded4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T13:39:53.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:53.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:53.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:53.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:53.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:53.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:53.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:52 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:52.824+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:39:53.520 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 podman[67598]: 2024-04-22 13:39:53.095451108 +0000 UTC m=+5.777611644 container cleanup 917a5c6e2271de6d0200b6e09073fbc3560a19c084702dda3975966a24bded4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:39:53.530 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 bash[67598]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10 2024-04-22T13:39:53.530 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 podman[67616]: 2024-04-22 13:39:53.288355679 +0000 UTC m=+0.518879166 container remove 917a5c6e2271de6d0200b6e09073fbc3560a19c084702dda3975966a24bded4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:39:54.019 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 podman[67709]: 2024-04-22 13:39:53.62150272 +0000 UTC m=+0.018420910 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:39:54.019 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 podman[67709]: 2024-04-22 13:39:53.722404594 +0000 UTC m=+0.119322775 container create e2cccea46c01a8b17b7d0dd6dc397a8a5a5d1b31843d89a87d234f462045b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-22T13:39:54.019 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 podman[67709]: 2024-04-22 13:39:53.932959856 +0000 UTC m=+0.329878037 container init e2cccea46c01a8b17b7d0dd6dc397a8a5a5d1b31843d89a87d234f462045b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:39:54.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:53 smithi049 podman[67709]: 2024-04-22 13:39:53.936173056 +0000 UTC m=+0.333091236 container start e2cccea46c01a8b17b7d0dd6dc397a8a5a5d1b31843d89a87d234f462045b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-22T13:39:54.020 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:54 smithi049 podman[67709]: 2024-04-22 13:39:54.024486141 +0000 UTC m=+0.421404331 container attach e2cccea46c01a8b17b7d0dd6dc397a8a5a5d1b31843d89a87d234f462045b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-22T13:39:54.270 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:54 smithi049 podman[67709]: 2024-04-22 13:39:54.239959163 +0000 UTC m=+0.636877335 container died e2cccea46c01a8b17b7d0dd6dc397a8a5a5d1b31843d89a87d234f462045b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:54.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:53 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:53.810+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:39:54.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.10.service' 2024-04-22T13:39:54.768 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:54 smithi049 podman[67736]: 2024-04-22 13:39:54.727628507 +0000 UTC m=+0.482604638 container remove e2cccea46c01a8b17b7d0dd6dc397a8a5a5d1b31843d89a87d234f462045b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T13:39:54.768 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:54 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-22T13:39:54.768 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:54 smithi049 systemd[1]: Stopped Ceph osd.10 for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:39:54.768 INFO:journalctl@ceph.osd.10.smithi049.stdout:Apr 22 13:39:54 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.10.service: Consumed 15.511s CPU time. 2024-04-22T13:39:54.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:39:54.783 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-22T13:39:54.783 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-22T13:39:54.784 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.11 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:54.796+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:39:55.081 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:54 smithi107 systemd[1]: Stopping Ceph osd.11 for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:39:55.082 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.084+0000 7f0ffb59c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:39:55.082 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.084+0000 7f0ffb59c700 -1 osd.11 106 *** Got signal Terminated *** 2024-04-22T13:39:55.082 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.084+0000 7f0ffb59c700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:39:56.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:56.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:56.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:56.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:56.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:56.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:56.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:55 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:55.813+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:57.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:56 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:56.825+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:58.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:57 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:57.812+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:39:59.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:39:59.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:39:59.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:39:59.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:39:59.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:39:59.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:39:59.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:58 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:58.861+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:40:00.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-22T13:38:31.356015+0000 front 2024-04-22T13:38:31.356104+0000 (oldest deadline 2024-04-22T13:38:56.656096+0000) 2024-04-22T13:40:00.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6804 osd.1 since back 2024-04-22T13:38:37.757182+0000 front 2024-04-22T13:38:37.757071+0000 (oldest deadline 2024-04-22T13:39:02.457191+0000) 2024-04-22T13:40:00.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6804 osd.2 since back 2024-04-22T13:38:45.958200+0000 front 2024-04-22T13:38:45.958120+0000 (oldest deadline 2024-04-22T13:39:10.058056+0000) 2024-04-22T13:40:00.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-22T13:38:53.559241+0000 front 2024-04-22T13:38:53.559185+0000 (oldest deadline 2024-04-22T13:39:18.259022+0000) 2024-04-22T13:40:00.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6812 osd.4 since back 2024-04-22T13:39:02.360030+0000 front 2024-04-22T13:39:02.360165+0000 (oldest deadline 2024-04-22T13:39:26.459916+0000) 2024-04-22T13:40:00.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.107:6812 osd.5 since back 2024-04-22T13:39:12.161912+0000 front 2024-04-22T13:39:12.162010+0000 (oldest deadline 2024-04-22T13:39:34.461939+0000) 2024-04-22T13:40:00.285 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:39:59 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11[57999]: 2024-04-22T13:39:59.853+0000 7f0ff7b90700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.49:6820 osd.7 since back 2024-04-22T13:39:26.564938+0000 front 2024-04-22T13:39:26.564960+0000 (oldest deadline 2024-04-22T13:39:51.864809+0000) 2024-04-22T13:40:00.872 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:00 smithi107 podman[70137]: 2024-04-22 13:40:00.62379083 +0000 UTC m=+5.638369512 container died dd3a4ed966e2c67ca03d481110b5dcde30d01a553190e4ba947cbd9beb724b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:40:01.156 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:00 smithi107 podman[70137]: 2024-04-22 13:40:00.967963515 +0000 UTC m=+5.982542188 container cleanup dd3a4ed966e2c67ca03d481110b5dcde30d01a553190e4ba947cbd9beb724b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-22T13:40:01.156 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:00 smithi107 bash[70137]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11 2024-04-22T13:40:01.534 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:01 smithi107 podman[70155]: 2024-04-22 13:40:01.160409393 +0000 UTC m=+0.536623742 container remove dd3a4ed966e2c67ca03d481110b5dcde30d01a553190e4ba947cbd9beb724b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-22T13:40:01.885 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:01 smithi107 podman[70244]: 2024-04-22 13:40:01.578226633 +0000 UTC m=+0.111664505 container create 79b9e467b9d4afaae190b3e18aa7ceabe1b344c77d48339e36890811673da529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-22T13:40:01.885 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:01 smithi107 podman[70244]: 2024-04-22 13:40:01.484594784 +0000 UTC m=+0.018032666 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:40:01.885 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:01 smithi107 podman[70244]: 2024-04-22 13:40:01.806458551 +0000 UTC m=+0.339896423 container init 79b9e467b9d4afaae190b3e18aa7ceabe1b344c77d48339e36890811673da529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-22T13:40:01.885 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:01 smithi107 podman[70244]: 2024-04-22 13:40:01.809653889 +0000 UTC m=+0.343091765 container start 79b9e467b9d4afaae190b3e18aa7ceabe1b344c77d48339e36890811673da529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:40:01.885 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:01 smithi107 podman[70244]: 2024-04-22 13:40:01.889858809 +0000 UTC m=+0.423296681 container attach 79b9e467b9d4afaae190b3e18aa7ceabe1b344c77d48339e36890811673da529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:40:02.284 INFO:journalctl@ceph.osd.11.smithi107.stdout:Apr 22 13:40:02 smithi107 podman[70244]: 2024-04-22 13:40:02.109439717 +0000 UTC m=+0.642877582 container died 79b9e467b9d4afaae190b3e18aa7ceabe1b344c77d48339e36890811673da529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:40:02.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@osd.11.service' 2024-04-22T13:40:02.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:02.708 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-22T13:40:02.708 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-22T13:40:02.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.a 2024-04-22T13:40:03.013 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:40:02 smithi005 systemd[1]: Stopping Ceph mds.a for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:03.307 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:40:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a[70379]: 2024-04-22T13:40:03.017+0000 7f5afd64f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:03.308 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:40:03 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a[70379]: 2024-04-22T13:40:03.017+0000 7f5afd64f700 -1 mds.a *** got signal Terminated *** 2024-04-22T13:40:04.754 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:40:04 smithi005 podman[102400]: 2024-04-22 13:40:04.471286156 +0000 UTC m=+1.563331115 container died f8402367009fcfd4298ad57d67bbba31a5df286e3a946bc7e4ec511a7e74dd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-22T13:40:05.005 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:40:04 smithi005 podman[102400]: 2024-04-22 13:40:04.758947784 +0000 UTC m=+1.850992737 container cleanup f8402367009fcfd4298ad57d67bbba31a5df286e3a946bc7e4ec511a7e74dd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:40:05.005 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 22 13:40:04 smithi005 bash[102400]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-a 2024-04-22T13:40:05.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.a.service' 2024-04-22T13:40:05.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:05.180 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-22T13:40:05.180 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-22T13:40:05.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.d 2024-04-22T13:40:05.558 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:05 smithi005 systemd[1]: Stopping Ceph mds.d for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:05.558 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d[69425]: 2024-04-22T13:40:05.512+0000 7f8818cfa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:05.558 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:05 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d[69425]: 2024-04-22T13:40:05.512+0000 7f8818cfa700 -1 mds.d *** got signal Terminated *** 2024-04-22T13:40:08.799 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:08 smithi005 podman[102521]: 2024-04-22 13:40:08.47283764 +0000 UTC m=+3.065072654 container died f871cdee6c7eabc1b3777f5e0d93647a778c758f7821c06ae1af1a28dd5f2637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:40:09.058 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:08 smithi005 podman[102521]: 2024-04-22 13:40:08.804392494 +0000 UTC m=+3.396627515 container cleanup f871cdee6c7eabc1b3777f5e0d93647a778c758f7821c06ae1af1a28dd5f2637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T13:40:09.058 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:08 smithi005 bash[102521]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d 2024-04-22T13:40:09.058 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 22 13:40:08 smithi005 podman[102535]: 2024-04-22 13:40:08.930293081 +0000 UTC m=+0.454360556 container remove f871cdee6c7eabc1b3777f5e0d93647a778c758f7821c06ae1af1a28dd5f2637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:40:09.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.d.service' 2024-04-22T13:40:09.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:09.195 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-22T13:40:09.196 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-22T13:40:09.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.g 2024-04-22T13:40:09.528 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:09 smithi005 systemd[1]: Stopping Ceph mds.g for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:09.807 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g[70022]: 2024-04-22T13:40:09.532+0000 7fc84633d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:09.808 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:09 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g[70022]: 2024-04-22T13:40:09.532+0000 7fc84633d700 -1 mds.g *** got signal Terminated *** 2024-04-22T13:40:12.728 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:12 smithi005 podman[102641]: 2024-04-22 13:40:12.470092391 +0000 UTC m=+3.041646444 container died d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:40:12.987 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:12 smithi005 podman[102641]: 2024-04-22 13:40:12.815654388 +0000 UTC m=+3.387208444 container cleanup d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-22T13:40:12.987 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:12 smithi005 bash[102641]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g 2024-04-22T13:40:13.245 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 22 13:40:12 smithi005 podman[102655]: 2024-04-22 13:40:12.991696139 +0000 UTC m=+0.521721155 container remove d544728105a2355183e061b0513a5db6edb12d842fb3a063d7080ef145f39aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-22T13:40:13.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.g.service' 2024-04-22T13:40:13.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:13.280 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-22T13:40:13.281 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-22T13:40:13.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.j 2024-04-22T13:40:13.558 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:13 smithi005 systemd[1]: Stopping Ceph mds.j for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:14.058 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j[69076]: 2024-04-22T13:40:13.602+0000 7fdf5e448700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:14.058 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:13 smithi005 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j[69076]: 2024-04-22T13:40:13.602+0000 7fdf5e448700 -1 mds.j *** got signal Terminated *** 2024-04-22T13:40:17.061 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:16 smithi005 podman[102761]: 2024-04-22 13:40:16.793128582 +0000 UTC m=+3.286652615 container died 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:40:17.403 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:17 smithi005 podman[102761]: 2024-04-22 13:40:17.068732084 +0000 UTC m=+3.562256130 container cleanup 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T13:40:17.403 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:17 smithi005 bash[102761]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j 2024-04-22T13:40:17.403 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 22 13:40:17 smithi005 podman[102775]: 2024-04-22 13:40:17.286520453 +0000 UTC m=+0.486445380 container remove 515ef78fe94a184485a7b9598364900ce6aea5f6306cd8a6638cbadfb6e1bf98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:40:17.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.j.service' 2024-04-22T13:40:17.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:17.455 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-22T13:40:17.455 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-22T13:40:17.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.b 2024-04-22T13:40:17.771 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:40:17 smithi049 systemd[1]: Stopping Ceph mds.b for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:18.270 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:40:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: 2024-04-22T13:40:17.776+0000 7fdd6fafd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:18.270 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:40:17 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b[61344]: 2024-04-22T13:40:17.776+0000 7fdd6fafd700 -1 mds.b *** got signal Terminated *** 2024-04-22T13:40:20.731 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:40:20 smithi049 podman[67816]: 2024-04-22 13:40:20.481164759 +0000 UTC m=+2.821716060 container died 6271c96a529e910575672b3f3ea56349ad8ddcee11091bf4365f3920659964ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:40:21.020 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:40:20 smithi049 podman[67816]: 2024-04-22 13:40:20.835553337 +0000 UTC m=+3.176104644 container cleanup 6271c96a529e910575672b3f3ea56349ad8ddcee11091bf4365f3920659964ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T13:40:21.020 INFO:journalctl@ceph.mds.b.smithi049.stdout:Apr 22 13:40:20 smithi049 bash[67816]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-b 2024-04-22T13:40:21.244 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.b.service' 2024-04-22T13:40:21.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:21.281 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-22T13:40:21.281 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-22T13:40:21.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.e 2024-04-22T13:40:21.520 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:21 smithi049 systemd[1]: Stopping Ceph mds.e for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:22.020 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e[60712]: 2024-04-22T13:40:21.605+0000 7fa529fa2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:22.021 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:21 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e[60712]: 2024-04-22T13:40:21.605+0000 7fa529fa2700 -1 mds.e *** got signal Terminated *** 2024-04-22T13:40:24.735 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:24 smithi049 podman[67950]: 2024-04-22 13:40:24.48189837 +0000 UTC m=+2.969672386 container died 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-22T13:40:24.993 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:24 smithi049 podman[67950]: 2024-04-22 13:40:24.830954107 +0000 UTC m=+3.318728128 container cleanup 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-22T13:40:24.993 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:24 smithi049 bash[67950]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e 2024-04-22T13:40:25.254 INFO:journalctl@ceph.mds.e.smithi049.stdout:Apr 22 13:40:24 smithi049 podman[67964]: 2024-04-22 13:40:24.998488089 +0000 UTC m=+0.511802930 container remove 360b54678b5fe14a16ab13e7c086d80b853362a58a5c37d8fd4bb9680b7e8de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:40:25.259 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.e.service' 2024-04-22T13:40:25.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:25.292 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-22T13:40:25.292 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-22T13:40:25.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.h 2024-04-22T13:40:25.520 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:25 smithi049 systemd[1]: Stopping Ceph mds.h for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:26.020 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h[61023]: 2024-04-22T13:40:25.617+0000 7fb0d4fc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:26.021 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:25 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h[61023]: 2024-04-22T13:40:25.617+0000 7fb0d4fc0700 -1 mds.h *** got signal Terminated *** 2024-04-22T13:40:28.730 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:28 smithi049 podman[68071]: 2024-04-22 13:40:28.469270687 +0000 UTC m=+2.947852923 container died fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:40:28.996 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:28 smithi049 podman[68071]: 2024-04-22 13:40:28.817310351 +0000 UTC m=+3.295892604 container cleanup fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-22T13:40:28.997 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:28 smithi049 bash[68071]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h 2024-04-22T13:40:29.239 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.h.service' 2024-04-22T13:40:29.260 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:29 smithi049 podman[68086]: 2024-04-22 13:40:29.001771493 +0000 UTC m=+0.526049153 container remove fbb58709ca370cd91dd111410e188bf0ca3ad004d2c646b42833544d0268ab48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-22T13:40:29.260 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:29 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.h.service: Deactivated successfully. 2024-04-22T13:40:29.260 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:29 smithi049 systemd[1]: Stopped Ceph mds.h for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:40:29.261 INFO:journalctl@ceph.mds.h.smithi049.stdout:Apr 22 13:40:29 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.h.service: Consumed 1.971s CPU time. 2024-04-22T13:40:29.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:29.271 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-22T13:40:29.271 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-22T13:40:29.272 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.k 2024-04-22T13:40:29.520 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:29 smithi049 systemd[1]: Stopping Ceph mds.k for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:30.020 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k[60393]: 2024-04-22T13:40:29.612+0000 7fea2ebb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:30.020 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:29 smithi049 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k[60393]: 2024-04-22T13:40:29.612+0000 7fea2ebb7700 -1 mds.k *** got signal Terminated *** 2024-04-22T13:40:33.883 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:33 smithi049 podman[68193]: 2024-04-22 13:40:33.624147435 +0000 UTC m=+4.134565394 container died 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:40:34.141 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:33 smithi049 podman[68193]: 2024-04-22 13:40:33.962620503 +0000 UTC m=+4.473038463 container cleanup 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:40:34.141 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:33 smithi049 bash[68193]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k 2024-04-22T13:40:34.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.k.service' 2024-04-22T13:40:34.410 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:34 smithi049 podman[68207]: 2024-04-22 13:40:34.146432769 +0000 UTC m=+0.517635894 container remove 4e8bbe93aae06aba2b8e5f94fd9e67b94863369f9e7910a808f415b1596f8aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-22T13:40:34.410 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:34 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-22T13:40:34.410 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:34 smithi049 systemd[1]: Stopped Ceph mds.k for 73e95e94-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:40:34.410 INFO:journalctl@ceph.mds.k.smithi049.stdout:Apr 22 13:40:34 smithi049 systemd[1]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.136s CPU time. 2024-04-22T13:40:34.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:34.420 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-22T13:40:34.421 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-22T13:40:34.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.c 2024-04-22T13:40:34.719 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:34 smithi107 systemd[1]: Stopping Ceph mds.c for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:34.997 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c[63277]: 2024-04-22T13:40:34.722+0000 7fe7a5b42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:34.997 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:34 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c[63277]: 2024-04-22T13:40:34.722+0000 7fe7a5b42700 -1 mds.c *** got signal Terminated *** 2024-04-22T13:40:36.784 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:36 smithi107 podman[70349]: 2024-04-22 13:40:36.477307225 +0000 UTC m=+1.858606622 container died cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:40:37.195 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:36 smithi107 podman[70349]: 2024-04-22 13:40:36.789887063 +0000 UTC m=+2.171186457 container cleanup cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-22T13:40:37.196 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:36 smithi107 bash[70349]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c 2024-04-22T13:40:37.196 INFO:journalctl@ceph.mds.c.smithi107.stdout:Apr 22 13:40:36 smithi107 podman[70374]: 2024-04-22 13:40:36.991252822 +0000 UTC m=+0.509648629 container remove cd1a96d06271db9ab38a6377178bf3158402d26c18970a6634fa6df897e15b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T13:40:37.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.c.service' 2024-04-22T13:40:37.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:37.244 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-22T13:40:37.244 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-22T13:40:37.244 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f 2024-04-22T13:40:37.534 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:40:37 smithi107 systemd[1]: Stopping Ceph mds.f for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:38.034 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:40:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[66920]: 2024-04-22T13:40:37.567+0000 7f6e3a984700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:38.034 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:40:37 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f[66920]: 2024-04-22T13:40:37.567+0000 7f6e3a984700 -1 mds.f *** got signal Terminated *** 2024-04-22T13:40:40.772 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:40:40 smithi107 podman[70482]: 2024-04-22 13:40:40.487734214 +0000 UTC m=+3.026393063 container died bf959c5c5c81b5032f18365ba6b9ae954aefd94c66a88391d80ae81ee1d0d324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T13:40:41.034 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:40:40 smithi107 podman[70482]: 2024-04-22 13:40:40.876750518 +0000 UTC m=+3.415409376 container cleanup bf959c5c5c81b5032f18365ba6b9ae954aefd94c66a88391d80ae81ee1d0d324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:40:41.034 INFO:journalctl@ceph.mds.f.smithi107.stdout:Apr 22 13:40:40 smithi107 bash[70482]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-f 2024-04-22T13:40:41.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.f.service' 2024-04-22T13:40:41.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:41.320 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-22T13:40:41.320 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-22T13:40:41.320 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.i 2024-04-22T13:40:41.784 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:40:41 smithi107 systemd[1]: Stopping Ceph mds.i for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:41.784 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:40:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i[63589]: 2024-04-22T13:40:41.630+0000 7fad4a52d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:41.784 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:40:41 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i[63589]: 2024-04-22T13:40:41.630+0000 7fad4a52d700 -1 mds.i *** got signal Terminated *** 2024-04-22T13:40:44.776 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:40:44 smithi107 podman[70601]: 2024-04-22 13:40:44.48767518 +0000 UTC m=+2.951188871 container died 3d90cf10b3400a4302b84c7fb1523e0f9d2984f9545a4259920d708b9ab3fb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-22T13:40:45.034 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:40:44 smithi107 podman[70601]: 2024-04-22 13:40:44.863342408 +0000 UTC m=+3.326856097 container cleanup 3d90cf10b3400a4302b84c7fb1523e0f9d2984f9545a4259920d708b9ab3fb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-22T13:40:45.034 INFO:journalctl@ceph.mds.i.smithi107.stdout:Apr 22 13:40:44 smithi107 bash[70601]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-i 2024-04-22T13:40:45.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.i.service' 2024-04-22T13:40:45.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:45.327 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-22T13:40:45.327 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-22T13:40:45.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.l 2024-04-22T13:40:45.646 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:40:45 smithi107 systemd[1]: Stopping Ceph mds.l for 73e95e94-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:40:46.034 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:40:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: 2024-04-22T13:40:45.650+0000 7f6b2440f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:40:46.035 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:40:45 smithi107 ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l[62653]: 2024-04-22T13:40:45.650+0000 7f6b2440f700 -1 mds.l *** got signal Terminated *** 2024-04-22T13:40:48.763 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:40:48 smithi107 podman[70722]: 2024-04-22 13:40:48.479530756 +0000 UTC m=+2.935185363 container died dfc486f84b81a22bb00aa920a851c8e5c162ed35314a769e927f526219871aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T13:40:49.034 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:40:48 smithi107 podman[70722]: 2024-04-22 13:40:48.850877984 +0000 UTC m=+3.306532588 container cleanup dfc486f84b81a22bb00aa920a851c8e5c162ed35314a769e927f526219871aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:40:49.034 INFO:journalctl@ceph.mds.l.smithi107.stdout:Apr 22 13:40:48 smithi107 bash[70722]: ceph-73e95e94-00aa-11ef-bc93-c7b262605968-mds-l 2024-04-22T13:40:49.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-73e95e94-00aa-11ef-bc93-c7b262605968@mds.l.service' 2024-04-22T13:40:49.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:40:49.301 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-22T13:40:49.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T13:40:49.500 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:40:52.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T13:40:52.246 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:40:54.752 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T13:40:54.939 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:40:57.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:40:57.300 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:40:57.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:40:57.358 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-22T13:40:57.361 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi005/crash 2024-04-22T13:40:57.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash -- . 2024-04-22T13:40:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T13:40:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-04-22T13:40:57.409 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi049/crash 2024-04-22T13:40:57.409 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash -- . 2024-04-22T13:40:57.448 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T13:40:57.448 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-04-22T13:40:57.450 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi107/crash 2024-04-22T13:40:57.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash -- . 2024-04-22T13:40:57.497 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T13:40:57.497 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-04-22T13:40:57.499 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-22T13:40:57.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-22T13:40:57.535 INFO:tasks.cephadm:Compressing logs... 2024-04-22T13:40:57.535 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T13:40:57.578 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T13:40:57.581 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T13:40:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log 2024-04-22T13:40:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:: No such file or directory 2024-04-22T13:40:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T13:40:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-22T13:40:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-22T13:40:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-22T13:40:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-22T13:40:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-22T13:40:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-22T13:40:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-22T13:40:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.a.log: 87.5% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T13:40:57.606 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.c.log: /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.5% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T13:40:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-22T13:40:57.611 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T13:40:57.611 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T13:40:57.615 INFO:teuthology.orchestra.run.smithi005.stderr: 94.5% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-22T13:40:57.615 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T13:40:57.615 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-22T13:40:57.615 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T13:40:57.615 INFO:teuthology.orchestra.run.smithi005.stderr: 84.0% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T13:40:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log 2024-04-22T13:40:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:: No such file or directory 2024-04-22T13:40:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T13:40:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-22T13:40:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5%/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.h.log: -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr: 90.3% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T13:40:57.619 INFO:teuthology.orchestra.run.smithi049.stderr: 91.8% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-22T13:40:57.621 INFO:teuthology.orchestra.run.smithi107.stderr: 91.9% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-22T13:40:57.621 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-22T13:40:57.621 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-22T13:40:57.621 INFO:teuthology.orchestra.run.smithi107.stderr: 83.6% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T13:40:57.621 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T13:40:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-22T13:40:57.625 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T13:40:57.625 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-22T13:40:57.625 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T13:40:57.625 INFO:teuthology.orchestra.run.smithi049.stderr: 83.6% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T13:40:57.631 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-22T13:40:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-22T13:40:57.638 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-22T13:40:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T13:40:57.643 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T13:40:57.645 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T13:40:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-22T13:40:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T13:40:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T13:40:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-22T13:40:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T13:40:57.668 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T13:40:57.668 INFO:teuthology.orchestra.run.smithi107.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T13:40:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 93.3% 94.4% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-22T13:40:57.671 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T13:40:57.671 INFO:teuthology.orchestra.run.smithi107.stderr: 92.8% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-22T13:40:57.673 INFO:teuthology.orchestra.run.smithi107.stderr: 92.9% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-22T13:40:57.681 INFO:teuthology.orchestra.run.smithi049.stderr: 92.8% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-22T13:40:57.684 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-22T13:40:57.710 INFO:teuthology.orchestra.run.smithi049.stderr: 92.7% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-22T13:40:57.711 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-22T13:40:57.713 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-22T13:40:57.723 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-22T13:40:57.878 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-22T13:40:57.933 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-22T13:40:57.937 INFO:teuthology.orchestra.run.smithi049.stderr: 93.0% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-22T13:40:58.057 INFO:teuthology.orchestra.run.smithi049.stderr: 92.1% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-22T13:40:58.290 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-22T13:40:58.337 INFO:teuthology.orchestra.run.smithi107.stderr: 92.9% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-22T13:40:58.358 INFO:teuthology.orchestra.run.smithi005.stderr: 92.9% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-22T13:40:58.382 INFO:teuthology.orchestra.run.smithi107.stderr: 92.8% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-22T13:40:58.398 INFO:teuthology.orchestra.run.smithi005.stderr: 93.0% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-22T13:40:58.428 INFO:teuthology.orchestra.run.smithi049.stderr: 92.8% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-22T13:40:58.434 INFO:teuthology.orchestra.run.smithi049.stderr: 93.1% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-22T13:40:58.518 INFO:teuthology.orchestra.run.smithi049.stderr: 93.1% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-22T13:40:58.560 INFO:teuthology.orchestra.run.smithi005.stderr: 90.6% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-22T13:40:58.612 INFO:teuthology.orchestra.run.smithi005.stderr: 93.1% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-22T13:40:58.665 INFO:teuthology.orchestra.run.smithi107.stderr: 93.0% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-22T13:40:58.667 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-22T13:40:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m1.072s 2024-04-22T13:40:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m3.722s 2024-04-22T13:40:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.153s 2024-04-22T13:40:58.713 INFO:teuthology.orchestra.run.smithi049.stderr: 93.2% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-22T13:40:58.715 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-22T13:40:58.715 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m1.119s 2024-04-22T13:40:58.715 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m4.407s 2024-04-22T13:40:58.715 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m0.174s 2024-04-22T13:40:58.715 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/73e95e94-00aa-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-22T13:40:58.717 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-22T13:40:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m1.126s 2024-04-22T13:40:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m4.919s 2024-04-22T13:40:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.161s 2024-04-22T13:40:58.718 INFO:tasks.cephadm:Archiving logs... 2024-04-22T13:40:58.718 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi005/log 2024-04-22T13:40:58.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T13:40:59.453 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi049/log 2024-04-22T13:40:59.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T13:41:00.104 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi107/log 2024-04-22T13:41:00.105 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T13:41:00.682 INFO:tasks.cephadm:Removing cluster... 2024-04-22T13:41:00.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --force 2024-04-22T13:41:00.878 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:41:01.146 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --force 2024-04-22T13:41:01.346 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:41:01.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 73e95e94-00aa-11ef-bc93-c7b262605968 --force 2024-04-22T13:41:01.799 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 73e95e94-00aa-11ef-bc93-c7b262605968 2024-04-22T13:41:02.115 INFO:tasks.cephadm:Removing cephadm ... 2024-04-22T13:41:02.115 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T13:41:02.134 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T13:41:02.151 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T13:41:02.166 INFO:tasks.cephadm:Teardown complete 2024-04-22T13:41:02.167 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T13:41:02.180 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T13:41:02.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T13:41:02.183 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T13:41:02.194 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T13:41:02.305 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T13:41:02.315 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T13:41:02.316 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:41:02.319 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:41:02.321 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:41:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:41:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:41:02.335 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:41:02.338 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:41:02.338 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-04-22T13:41:02.339 INFO:teuthology.orchestra.run.smithi005.stdout:^- hv01.front.sepia.ceph.com 2 6 377 13 +393us[ +393us] +/- 593ms 2024-04-22T13:41:02.339 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -2317us[-2284us] +/- 67ms 2024-04-22T13:41:02.339 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +3128us[+3128us] +/- 89ms 2024-04-22T13:41:02.339 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:^- hv01.front.sepia.ceph.com 2 6 377 14 +391us[ +383us] +/- 593ms 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -2304us[-2312us] +/- 67ms 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +3137us[+3137us] +/- 89ms 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:41:02.341 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-04-22T13:41:02.342 INFO:teuthology.orchestra.run.smithi049.stdout:^- hv01.front.sepia.ceph.com 2 6 377 13 +514us[ +514us] +/- 593ms 2024-04-22T13:41:02.342 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -2176us[-2152us] +/- 67ms 2024-04-22T13:41:02.342 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +3256us[+3281us] +/- 89ms 2024-04-22T13:41:02.342 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:41:02.342 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T13:41:02.353 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T13:41:02.354 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T13:41:02.364 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T13:41:02.467 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T13:41:02.538 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T13:41:02.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:41:02.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:41:02.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:41:02.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:41:02.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:41:02.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:41:02.688 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T13:41:02.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T13:41:02.709 INFO:teuthology.task.internal:Duration was 1894.852551 seconds 2024-04-22T13:41:02.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T13:41:02.720 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T13:41:02.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T13:41:02.723 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T13:41:02.725 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T13:41:02.761 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:41:02.762 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:41:02.764 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:41:03.093 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T13:41:03.093 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-04-22T13:41:03.094 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T13:41:03.155 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-04-22T13:41:03.155 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T13:41:03.188 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T13:41:03.188 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T13:41:03.216 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T13:41:03.216 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T13:41:03.219 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T13:41:03.230 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T13:41:03.281 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T13:41:03.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T13:41:03.288 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T13:41:03.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T13:41:03.708 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T13:41:03.720 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T13:41:03.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T13:41:03.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T13:41:03.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T13:41:03.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T13:41:03.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:41:03.809 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:41:03.812 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:41:03.829 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-04-22T13:41:03.833 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-04-22T13:41:03.855 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-04-22T13:41:03.879 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:41:03.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:41:03.897 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:41:03.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:41:03.913 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:41:03.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:41:03.933 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T13:41:03.947 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T13:41:03.947 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi005 2024-04-22T13:41:03.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T13:41:03.997 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi049 2024-04-22T13:41:03.997 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T13:41:04.033 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668269/remote/smithi107 2024-04-22T13:41:04.034 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T13:41:04.068 INFO:teuthology.task.internal:Removing archive directory... 2024-04-22T13:41:04.068 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T13:41:04.071 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T13:41:04.073 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T13:41:04.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T13:41:04.132 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T13:41:04.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T13:41:04.185 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T13:41:04.186 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T13:41:04.188 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T13:41:04.191 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T13:41:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 13:41 /home/ubuntu/cephtest 2024-04-22T13:41:04.203 INFO:teuthology.orchestra.run.smithi049.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 13:41 /home/ubuntu/cephtest 2024-04-22T13:41:04.204 INFO:teuthology.orchestra.run.smithi107.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 13:41 /home/ubuntu/cephtest 2024-04-22T13:41:04.205 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T13:41:04.218 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T13:41:04.292 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 1894.8525512218475 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-22T13:41:04.292 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T13:41:04.381 INFO:teuthology.run:pass